Stage Filter in sequentil file stage command line arguments

[ranga1970]

Hi folk;
As for my discussion on this forum Ray advised me use srot - u to remove duplicates from sequential file.
1. For this I had checked the box stage uses filter Commands
2. then in the general tab filter command I ahs specified

Sort - U - PROVIDER_ID, STREET_LINE_1, STREET_LINE_2, CITY, STATE, ZIP_CODE, ZIP_CODE_EXTENSION_1, SOURCE_SYSTEM_ID

where PROVIDER_ID, STREET_LINE_1, STREET_LINE_2, CITY, STATE, ZIP_CODE, ZIP_CODE_EXTENSION_1, SOURCE_SYSTEM_ID are the columns on which I want to check unique ness, is this the curect way to specify the command line arguments, your help will be appreciated

thanks

[chulett]

No, it's not. The exact way would depend on your flavor of UNIX and can be found by reading the 'man' pages for the sort command on your system.

If you wanted to sort based on all fields (the entire record) you could just use sort -u and be fine. However, if you only want to sort on certain field(s), you need to tell the sort command how they are delimited and which column numbers to use.

It's all in the man pages, or you can search the forum for specific examples that people have posted.

[ranga1970]

I just did that by putting Sort - U in text box for in General tab, filter command:

now the job is throwing warnings

Warning 1:
LodProviderAddr.NICE.ProviderAddr.lnkProviders_in: ds_seqopen() - Error in filter command "Sort - U" -
LodProviderAddr.NICE.ProviderAddr.lnkProviders_in: ds_seqopen() - Error in execv() - No such file or directory

Warning 2:
LodProviderAddr.NICE.ProviderAddr.lnkProviders_in: DSD.SEQOpen GCI $DS.SEQOPEN error 11.

Fatal: LodProviderAddr.NICE.xfmProvider: |LodProviderAddr.NICE.ProviderAddr.lnkProviders_in: DSD.SEQOpen GCI $DS.SEQOPEN error 11.|


and job aborted, could some help where I went wrong, this is the first time I am trying to use stage uses filer commands otpion

in the DS manual, it says once give filter command , view data should be desabled, but in my job it is not happening so, could some one help me on this


thanks

[ray.wurlod]

For starters, the option must be in lower case: sort -u

Secondly, you will almost certainly need to specify at least one sort key (using a -k option).

[ranga1970]

Thanks ray,
Yes I realaised my mistake aboute case, it was too late, I apolazise
coming to second point
- k option, if I want the sort to be performed on the enire record uniqueness, is sort -u is enough or still -k is required,
thanks in advance

[ranga1970]

I had recitified the case sensitivity and my command now is

sort - u

but this time different kind of warning

LodProviderAddr.RIMS.ProviderAddr.lnkProviders_in: ds_seqopen() - Error in filter command "/usr/bin/sort - u" -
sort: Warning: A newli


any help on this for me...


Thanks

[chulett]

It's sort -u not sort - u

(no space between the dash and the 'u')

Make sure it runs from the command line before you try it in a job.

[kduke]

The sort command needs a separator. It cannot use column names. There is a sort filter example on ADN. Download it and figure it out. Also there are lots of posts on this so search for them.