I have two tables. Current and previous.I am joining these two tables and writing it a Sequential file. The table definition for the two tables is same. I am getting the following errors
Join_Stage: When checking operator: Dropping component "column1" because of a prior component with the same name.
Sequential_File_Stage: When checking operator: A sequential operator cannot preserve the partitioning
of the parallel data set on input port 0.
Can anyone please explain me how I can get rid of these warnings?
Join stage warnings
Moderators: chulett, rschirm, roy
Any time you get a 'dropping component' warning it means that DS thinks it has two columns of the same name going downstream; and in most cases this is because you have ntime column propagation turned on. You should fix this in your code to get rid of the warning.
Sequential files can only be processed sequentially and not in parallel, so when you go from a PX stage executed in parallel (pretty much any of them) to a sequential stage and you keep the partitioning flag set to "Auto" you will get this warning at runtime. There are two ways to get rid of this message. The most common is to use a message handler to demote this message to informational or drop it from the log altogether. The other method is to set the propagate partitioning flag on the stage leading to the sequential stage to read 'clear'
Sequential files can only be processed sequentially and not in parallel, so when you go from a PX stage executed in parallel (pretty much any of them) to a sequential stage and you keep the partitioning flag set to "Auto" you will get this warning at runtime. There are two ways to get rid of this message. The most common is to use a message handler to demote this message to informational or drop it from the log altogether. The other method is to set the propagate partitioning flag on the stage leading to the sequential stage to read 'clear'
</a>For the first warning you can try with RCP switched off. This will stop DataStage from propagating all the the fields from source to target. Only the fields which are mapped will be propagated. For the second warning, set the Preserve Partitioning property to Clear. This property is by default Propagate which tells the stage down stream to preserve the partitioning. This property conflicts with the Sequential File stage since it is a sequential operator and runs on a single node and cannot preserve partitioning.