How to build a External Filter Stage
Posted: Thu Oct 05, 2006 11:39 pm
Hi All,
I am making an attempt to understand how to use a external filter stage. I know the below narrated one is a simple situation and can be handled with a filter stage. For simplicity, i have assumed the below situation for the study. Request your assistance in doing it...
# Should i need to pass all the columns as argumnets to the shell script(stdin), even though the filter critera is on a single column(val) ?
# Should i need to output(stdout) all the columns from the shell script ?
I know i am missing things some where, pls help me in completing my understanding.
Bunch of thanks
,
-Sid
I am making an attempt to understand how to use a external filter stage. I know the below narrated one is a simple situation and can be handled with a filter stage. For simplicity, i have assumed the below situation for the study. Request your assistance in doing it...
Code: Select all
Input
-----
id val
-- ---
1 A
2 B
3 C
4 A
5 B
6 A
7 A
8 C
9 C
I want to pass the records with val="A" alone
Output
------
id val
-- ---
1 A
4 A
6 A
7 ACode: Select all
> My DS Job Seq File ----> External Filter Stage -----> Peek
> I have planned to use a shell script(grepcmd.sh) as a filter command in the External Filter Stage.
repcmd.sh => cat $1 | grep "A"
> In the External Filter Stage, following are the optios set,
Filter command = grepcmd.sh
Arguments = val# Should i need to output(stdout) all the columns from the shell script ?
Code: Select all
With the above setup, i get the following error,
External_Filter_1,0: cat: val: The system cannot find the file specified.
External_Filter_1,0: Wrapped Unix command 'grepcmd.sh val' terminated with error, exit code 1.External_Filter_1
External_Filter_1,0: subprocess failed with exit code 1,External_Filter_1
External_Filter_1,0: Operator's runLocally() failed.Bunch of thanks
,-Sid