sort filter problem

[ranga1970]

Hi all;
I browsed trough forum and with different post input
here is what I am doing
sort -u
Which i suppose will take care of duplicates by entire record

but i am encountering strange problem which I think at unix level

the error I am getting is

LodProviderAddr.ILIAD.ProviderAddr.lnkProviders_in: ds_seqopen() - Error in filter command "/usr/bin/sort -u" -
sort: Warning: A newli

is this some thing to do with unix level previlages or soem thing else

I am sure I did not make any mistake of case or spaces

it is just sort -u

any help is appreciated

[Sainath.Srinivasan]

Did you run the same command from Unix prompt? It maybe the sort is returning an error which DataStage raises as the warning.

By-the-way, did you give the filename you wish to sort?

[ranga1970]

I gave the file neme in the file name text box and sort -u in the filter command text box, I am doing it right?????

[ranga1970]

I think the proble is hear

LodProviderAddr.ILIAD.ProviderAddr.lnkProviders_in: ds_seqopen() - Error in filter command "/usr/bin/sort -u /etldata/cpdr/CPDR_DEV1/common/data/work/ILIADProviderAddress.dat" -
sort: Warning: A newline character was added to the end of file

Could any experts help me on this

does this mean it is problem with file?
or some thing else

desperate to learn a new thing, any inputs will be greatly appreciated

[alhamilton]

It sounds like sort is expecting the newline character at the end of the file and not finding it. So it adds it for you and warns you about that. If you cannot correct this at the source of the file or in some prep before the DataStage job (maybe move the sort there), then you might try redirecting the standard out to /dev/null.

sort -u 2>/dev/null

You'll have to weigh the risk that something really wrong with the sort might go undetected. Good Luck.