DSXchange

Skip header and tail while reading from a flat file.

source.txt

eg:
CountryName|Capital
USA|Washington
UK|London
Austria|Vienna
Generated on: 04/04/2006


I need to skip the header and tail while reading.

resulted output is


USA|Washington
UK|London
Austria|Vienna

i searched in DSX, someone told to use @INROWNUM

what are the possibilities of doing this.
i am novice in this..
where exactly i need to write the code.....


Thanks
sunshine

n00b here too

@INROWNUM looks like the trick, but perhaps you could pair it with @OUTROWNUM

I'd build a job like this..

In transformer, add a constraint. The constraint would be something like:
@INROWNUM > 1 and @INROWNUM <> @OUTROWNUM

If the constraint evaluates to true, the output link should have the rows you want (i.e. not the first and not the last).

Hope it helps.

Use head tail and wc -l to strip off the header and trailer lines.

[quote="ray.wurlod"]Use [b]head [/b][b]tail [/b]and [b]wc -l [/b]to strip off the header and trailer lines. ...[/quote]

where i can use this head and tail

can you explain me detail

Thanks
sunshine

Hi,

You can do this in before/after job job subroutine.

head -$(expr $(wc -l | awk '{ print $1 }') \- 1) filename | tail +2

this would work

head -$(expr $(wc -l | awk '{ print $1 }') \- 1) filename | tail +2[/quote]

this would work,


ohhh, very confused...
If u have some time, Can you explain the command...

Thanks
vsi

when i tried the command,got an exception:

i copied the command to a file called "filter.txt"

head -$(expr $(wc -l | awk '{ print $1 }') \- 1) Countries.txt | tail +2

and in the filter command,i entered the path of the file

but throws exception:

Error in filter command "home/username/filter.txt" -
job1.. Sequentional_File_Dslink3: ds_seqopen() - Error in execv() -
Permission Denied
job1.. Sequentional_File_Dslink3: DSD.SEQOpen GCI
$DS.SEQOPEN error 11


any idea?


Thanks
vsi

You need to enter the head/tail/awk command in the filter command.

you can view the UNIX man pages for head & tail & wc and the awk command has whole books written about it.

Hi

i used command "sed"

eg:sed -e '1d' -e '$d' filename.

any comments?

Thanks
vsi

head -$(expr $(wc -l filename | awk '{ print $1 }') \- 1) filename | tail +2

try this [/quote]

if i write the below script in a file and execute it in filter command say

"sh /home/kpraveen/test.sh"

how can i parameterize the "filename" in the script

"head -$(expr $(wc -l filename | awk '{ print $1 }') \- 1) filename | tail +2"

so that it takes the file which is given in the

filename

option of seqential file

any ideas guys

thx[/quote]

How are you invoking the script? If it's via ExecSH you can simply make the file name a job parameter and use the parameter reference in both locations. If through DSExecute in one of your own routines, you can simply build the command into a variable, having passed the file name as a routine argument for example.

zbethem wrote:n00b here too

@INROWNUM looks like the trick, but perhaps you could pair it with @OUTROWNUM

I'd build a job like this..

In transformer, add a constraint. The constraint would be something like:
@INROWNUM > 1 and @INROWNUM <> @OUTROWNUM

If the constraint evaluates to true, the output link should have the rows you want (i.e. not the first and not the last).

Hope it helps.

Hi,

I tried with this logic, but only the header alone filtered out not the final record.

regards

vinod

Use the above in the before job subroutine with ExecSH.
Use te New_Filename.txt to read in the sequential stage.