filter

[mcs_suman]

In filter stage i am filtering the particular record which has many duplicates.

which gives all the rows,in order to the first row I set row out put only once. instead of getting only one row i am getting all the row.
could any one help me.

[Maveric]

Looks like a remove duplicates logic to me. Do you want to remove all the duplicates from the data? Then use a Remove Duplicates stage instead of a filter stage. Hash partition and sort the data on the key fields.

[mcs_suman]

Looks like a remove duplicates logic to me. Do you want to remove all the duplicates from the data? Then use a Remove Duplicates stage instead of a filter stage. Hash partition and sort the data on the key fields.

i didn't mean removing the duplicates. i set 'output row once', option in filter stage,and still i didn't notice any change in the output.

will this option allow only one record from the filtered output.

[BugFree]

This is what the documentation says:

Code: Select all

Set this to true to specify that rows are only output down the link of the first Where clause they satisfy. Set to false to have rows output down the links of all Where clauses that they satisfy

Setting this to true will pass the record only once to any output links.