filter commands

[New2DS]

Hello All,

I need some help in understanding the filter command option on the seqfile stage. When do we use these filter commands and what kind of filter commands we can use in the Filter command text box. Can we use any kind of UNIX commands?

I tried to use the sort command using the -t"|" -k1 option to sort the input file on column1 but it did not work meaning the job was successful but it did not sort the file.

What is difference in using a sort command in the before routine and in the filter command in the seq file stage. What I can think of at this time is, if the seq file stage is in between the job design (not as source) and want to use some commands on the file we can make use of this filter command option. Is that true?

thanks

[Sainath.Srinivasan]

How did you come to the conclusion that it did not sort the file?

Try to view data to confirm the sort.

[chulett]

The Filter can take any O/S command that handles Standard In and Standard Out well. Sort should be one of them.

There is a bug in the version I am running (7.0.1) where filter commands with quotes or pipes get stripped after the first occurance. You can see this by typing a filter command in the stage, closing it and reopening it. If you are a victim of this, you'll see your truncated command the next time you open the stage. For me it means I have to make sure the record delimiter matches the default for sort as I can't pass in a "-t" with mine.

[New2DS]

Craig,

Thanks for the explanation you are correct, it will strip of the quote characters and the rest when you reopen the stage. I tried to parameterize all the sort options like sort #opts# #FileName#.txt where opts = -t"|" -r -k1 and it works, meaning it will not strip when you open it next time.

I am trying to reverse sort using the -r option but it is not sorting in the order I want. Can't we use a '-r' option in the filter command? I tried to use the same command on the UNIX it worked. What is that option we use for the sort command if the data is alpha numeric is it '-n'?

Sai,
I saw the output and it is not in the reverse order I want.

Thanks

[chulett]

Technically, any valid UNIX command should work in the Filter.

By default, you get a string (alpha numeric) sort and you have to use the -n option to force a numeric sort. This should all be spelled out in your man pages.

I don't know if it's valid to use Job Parameters in the filter, I've never tried it. You may not be getting the command stripped but the parameters may not be helping you. You'd have to experiment to see if they can be used there or not.