External Filter

[RPhani]

Hi experts,
I have few issues with external filter..
I have to capture the rejected detailed records of Sequential file Source.
Metadata mismatched ,Trailer and null records
should be rejected.But my requirement is to capture only detailed reject records.
I added an external filter to the reject link and tried to drop trailer and null records
and populate detailed rejected records to the dataset.
My trailer record starts with "Tr"
1)I have taken command in external filter = egrep -v '^Tr|^$'
job is aboretd because of fatal error 'external filter with exit1'
2)The same command worked fine when I taken rejected,Null Recs,trailer recs to a data set
and applied externalfilter to this dataset in another job.
Why the same command worked in one job and not worked in another job?
2)Then i tried with sed -e options.it failed because of '/./^$'.But Sed -e supports '/./!d'
Why ^$ works fine in egrep and why it is not worked for sed -e options?
Could you please help me to resolve this issue.

3) Is there any logic without using external filter to drop null recs and Trailer rec?

----------------------------------------------------------------------------
Thanks,
Phani

[bhasds]

Hi Rphani,

If you are using the sequential file stage , then you can use the Filter option in properties tab.You can try the below command in the Filter option to discard trailing records and null records-

Code: Select all

nawk -F"|" '{if ($1 !~ "Tr") print $0}' count | grep .

I have assumed that the file is pipe delimited.