In filter stage i am filtering the particular record which has many duplicates.
which gives all the rows,in order to the first row I set row out put only once. instead of getting only one row i am getting all the row.
could any one help me.
filter
Moderators: chulett, rschirm, roy
filter
suman
Looks like a remove duplicates logic to me. Do you want to remove all the duplicates from the data? Then use a Remove Duplicates stage instead of a filter stage. Hash partition and sort the data on the key fields.
It is our choices that show what we truly are, far more than our abilities - J.K.Rowling
Maveric wrote:Looks like a remove duplicates logic to me. Do you want to remove all the duplicates from the data? Then use a Remove Duplicates stage instead of a filter stage. Hash partition and sort the data on the key fields.
i didn't mean removing the duplicates. i set 'output row once', option in filter stage,and still i didn't notice any change in the output.
will this option allow only one record from the filtered output.
suman
This is what the documentation says:
Setting this to true will pass the record only once to any output links.
Code: Select all
Set this to true to specify that rows are only output down the link of the first Where clause they satisfy. Set to false to have rows output down the links of all Where clauses that they satisfy
Ping me if I am wrong...