Sequential file stage - Filter option

[saradakiran]

Hi All

Request your help in solving the issue with Filter command property of sequential file stage.

I need to read the header and data rows separately (header row has different structure than data rows). I am able to read the header row separately.But when I tried using the below command to read data rows it is not filtering any header row and not even throwing any syntax error.
grep -v -e "`head -1 FILENAME`"

Does this property of sequential stage has any restrictions on the UNIX commands used

Thanks in advance

[chulett]

No, not that I am aware of, other than the need to write to standard out. Did you test your expression from the command line? It just hangs for me.

[saradakiran]

Hi Chullet

Thanks for the quick response

Yes I tred that expresson in command line.For command line we need to use that command like this
grep -v -e "`head -1 FILENAME`" FILENAME
For DS as it is not mandatory to specify the file name again I haven't mentioned it. But I have tried this second expression also.But no luck

[chulett]

Try your full expression again in the Filter and put /dev/null in the Filename prompt (rather than the filename again) and see if that helps.

[saradakiran]

Hi Chullet
No luck .Its saying data source is empty

[gomez]

That would mean the file that you are testing with is empty. Please check

[ArndW]

Also remember that the working directory of the job is in the project directory, so you should always specify a full and absolute path to your filename.

[saradakiran]

Hi All

Thanks for the response.

First of all I clarify that file is not empty.When I tried using /dev/null then only its giving source is empty.

Secondly I have given the full path name in my job.Just for an example purpose I have mentioned FILENAME.

Any help is greately appriciated

[chulett]

So... you put /dev/null in the Filename prompt and this in the Filter with the full path for the filename (times two):

grep -v -e "`head -1 FILENAME`" FILENAME

And it told you it was empty? If so, then you may be a victim of the filter's bad handling of quotes in the command. I've seen it strip them or only execute up to the first quote and ignore the rest of the command.

Open a case with your official support provider and ask them if that should be working or not.