How to avoid this warning

[bollinenik]

Hi,

I am reading data from seq.file and next i am using Sort Stage and next using Transformer for avoid duplicates and next lookup i am getting this warning for these 3 stages like this
where ever i am using seq.file as target in this job ia m getting warning like " Trgt_Rej_data_FF: When checking operator: A sequential operator cannot preserve the partitioning
of the parallel data set on input port 0." any can help me
thanks in advance

[kris007]

I didn't quite get the clear picture of what you are doing in your job. The order of stages are confusing.But what you will need to do get rid of the warning is set your PreservePartioning flag to "Clear" in one of the stages downstream of the Sort stage because sequential file writes records only sequentially. How exactly are you removing duplicates from Transformer. Using Constraints? Wouldn't Remove Duplicates stage help you in this case? Just wondering.

P.S. Please delete the other duplicate post you have in place before anyone starts replying to it.

[vmcburney]

Assuming you are sending reject rows from the transformer (via Trgt_Rej_data_FF) to a sequential file the job is warning you that you are switching from partitioned data to sequential data. Each transformer across each node will have to send rejects to a link that repartitions (or unpartitions?) the data into a sequential stream to write it to a single file.

This is not a big deal since you would hope to get a very low number of rejects.

You could switch this message across to an information message via the message handler. Would be interested to hear from other programmers as to whether there is an easy way to avoid the warning altogether. Maybe if you switch the partitioning output link from SAME to sequential.

[ray.wurlod]

How about this post[url]?

Another post by the OP with identical topic. That will really help the Search facility!