skip header and tail while reading from seq file
Moderators: chulett, rschirm, roy
skip header and tail while reading from seq file
Skip header and tail while reading from a flat file.
source.txt
eg:
CountryName|Capital
USA|Washington
UK|London
Austria|Vienna
Generated on: 04/04/2006
I need to skip the header and tail while reading.
resulted output is
USA|Washington
UK|London
Austria|Vienna
i searched in DSX, someone told to use @INROWNUM
what are the possibilities of doing this.
i am novice in this..
where exactly i need to write the code.....
Thanks
sunshine
source.txt
eg:
CountryName|Capital
USA|Washington
UK|London
Austria|Vienna
Generated on: 04/04/2006
I need to skip the header and tail while reading.
resulted output is
USA|Washington
UK|London
Austria|Vienna
i searched in DSX, someone told to use @INROWNUM
what are the possibilities of doing this.
i am novice in this..
where exactly i need to write the code.....
Thanks
sunshine
Re: skip header and tail while reading from seq file
n00b here too
@INROWNUM looks like the trick, but perhaps you could pair it with @OUTROWNUM
I'd build a job like this..
In transformer, add a constraint. The constraint would be something like:
@INROWNUM > 1 and @INROWNUM <> @OUTROWNUM
If the constraint evaluates to true, the output link should have the rows you want (i.e. not the first and not the last).
Hope it helps.
@INROWNUM looks like the trick, but perhaps you could pair it with @OUTROWNUM
I'd build a job like this..
In transformer, add a constraint. The constraint would be something like:
@INROWNUM > 1 and @INROWNUM <> @OUTROWNUM
If the constraint evaluates to true, the output link should have the rows you want (i.e. not the first and not the last).
Hope it helps.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
when i tried the command,got an exception:
i copied the command to a file called "filter.txt"
head -$(expr $(wc -l | awk '{ print $1 }') \- 1) Countries.txt | tail +2
and in the filter command,i entered the path of the file
but throws exception:
Error in filter command "home/username/filter.txt" -
job1.. Sequentional_File_Dslink3: ds_seqopen() - Error in execv() -
Permission Denied
job1.. Sequentional_File_Dslink3: DSD.SEQOpen GCI
$DS.SEQOPEN error 11
any idea?
Thanks
vsi
i copied the command to a file called "filter.txt"
head -$(expr $(wc -l | awk '{ print $1 }') \- 1) Countries.txt | tail +2
and in the filter command,i entered the path of the file
but throws exception:
Error in filter command "home/username/filter.txt" -
job1.. Sequentional_File_Dslink3: ds_seqopen() - Error in execv() -
Permission Denied
job1.. Sequentional_File_Dslink3: DSD.SEQOpen GCI
$DS.SEQOPEN error 11
any idea?
Thanks
vsi
You need to enter the head/tail/awk command in the filter command.
you can view the UNIX man pages for head & tail & wc and the awk command has whole books written about it.
you can view the UNIX man pages for head & tail & wc and the awk command has whole books written about it.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
if i write the below script in a file and execute it in filter command say
how can i parameterize the "filename" in the script
any ideas guys
thx[/quote]
"sh /home/kpraveen/test.sh"
how can i parameterize the "filename" in the script
so that it takes the file which is given in the"head -$(expr $(wc -l filename | awk '{ print $1 }') \- 1) filename | tail +2"
option of seqential filefilename
any ideas guys
thx[/quote]
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
How are you invoking the script? If it's via ExecSH you can simply make the file name a job parameter and use the parameter reference in both locations. If through DSExecute in one of your own routines, you can simply build the command into a variable, having passed the file name as a routine argument for example.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Re: skip header and tail while reading from seq file
Hi,zbethem wrote:n00b here too
@INROWNUM looks like the trick, but perhaps you could pair it with @OUTROWNUM
I'd build a job like this..
In transformer, add a constraint. The constraint would be something like:
@INROWNUM > 1 and @INROWNUM <> @OUTROWNUM
If the constraint evaluates to true, the output link should have the rows you want (i.e. not the first and not the last).
Hope it helps.
I tried with this logic, but only the header alone filtered out not the final record.
regards
vinod
Code: Select all
head -$(expr $(wc -l filename | awk '{ print $1 }') \- 1) filename | tail +2 > New_Filename.txt
Use te New_Filename.txt to read in the sequential stage.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'