Remove first four rows from file
Moderators: chulett, rschirm, roy
Remove first four rows from file
Hi all,
I am trying a read a text file using sequential file stage. Column headers start from fifth line. And it contains 10 columns. I gave 10 column names in the column section. First four lines contain some information about the file. I want to remove that. Please help me in removing those four lines before proceeding with the load.
Thanks in advance
I am trying a read a text file using sequential file stage. Column headers start from fifth line. And it contains 10 columns. I gave 10 column names in the column section. First four lines contain some information about the file. I want to remove that. Please help me in removing those four lines before proceeding with the load.
Thanks in advance
-
- Participant
- Posts: 437
- Joined: Fri Oct 21, 2005 10:00 pm
Just put a reject link off of your sequential stage. The lines that don't meet the format of the file i.e. the 4 descriptive lines, will flow to the reject link and the rest of the data will flow through the job normally. Pretty simple.
Keith Williams
keith@peacefieldinc.com
keith@peacefieldinc.com
-
- Participant
- Posts: 437
- Joined: Fri Oct 21, 2005 10:00 pm
Of course ... It is probable that the delimeter (pipe, tab, comma) being used within the file is one that would preclude a header description from matching the content of the file in format. The only way a filter works is if the format matches to define a field by which to filter the first four line. Which is why I didn't suggest it.
If you don't want to do it this way, have the sequential stage read the entire file, use a transformer to constraint to eliminate the first four rows, and then use the column import stage to break the single line into multiple columns.
If you don't want to do it this way, have the sequential stage read the entire file, use a transformer to constraint to eliminate the first four rows, and then use the column import stage to break the single line into multiple columns.
Keith Williams
keith@peacefieldinc.com
keith@peacefieldinc.com
-
- Participant
- Posts: 437
- Joined: Fri Oct 21, 2005 10:00 pm
According to the documentation, the filter only works after reading the data.
Keith Williams
keith@peacefieldinc.com
keith@peacefieldinc.com
Hmmm, without testing, I would have hoped it passed the data through the filter before it attempted to parse it (like it does in Server jobs). As far as I remember (which admittedly isn't much), it should be a command line call to process the file prior to DataStage processing... still, will give the OP something to confirm if they so desire...
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 597
- Joined: Fri Apr 29, 2005 6:19 am
- Location: Singapore
Re: Remove first four rows from file
If it is a one time load, simply edit the file and remove first 4 lines manually
If not, there are many ways to remove first 4 lines using UNIX commands. for e.g. sed command. Execute this command in the respective place and then call the load job!!
If not, there are many ways to remove first 4 lines using UNIX commands. for e.g. sed command. Execute this command in the respective place and then call the load job!!
Kandy
_________________
Try and Try again…You will succeed atlast!!
_________________
Try and Try again…You will succeed atlast!!
-
- Participant
- Posts: 597
- Joined: Fri Apr 29, 2005 6:19 am
- Location: Singapore
Pandeesh, This command will stream the output? Don't you need to write to a temp file & replace the original file with the temp file?
Last edited by kandyshandy on Tue Feb 07, 2012 9:51 pm, edited 1 time in total.
Kandy
_________________
Try and Try again…You will succeed atlast!!
_________________
Try and Try again…You will succeed atlast!!