getting warnings in Lookup and Transformer stages

[Sridhar Sivakoti]

Hi,

I have a job designed with Seq file and Data sets are inputs to the Lookup stage and output of this lookup stage into Transformer stage.

I am getting below warnings

lkupProdCogs: Input dataset 1 has a partitioning method other than entire specified; disabling memory sharing.


trnsBusDt: Input dataset 0 has a partitioning method other than entire specified; disabling memory sharing.

lkupProdCogs: When checking operator: Operator of type "APT_LUTCreateOp": will partition despite the
preserve-partitioning flag on the data set on input port 0.

lkupProdCogs,0: Ignoring duplicate entry at table record 6357043; no further warnings will be issued for this table

Can any one suggest me what might be the reason for above warnings?

Thanks
Sridhar

[nick.bond]

lkupProdCogs: Input dataset 1 has a partitioning method other than entire specified; disabling memory sharing.

This is because the job you have created that creates the dataset is configured to partition the data, probably by your lookup key. Check the setup of this. The partitioning need to match in both the job that you create the dataset and the job where you perform the lookup. To keep it simple you can create the dataset with Entire partitioning, until you hit performance issues.

lkupProdCogs: When checking operator: Operator of type "APT_LUTCreateOp": will partition despite the
preserve-partitioning flag on the data set on input port 0.

This is just telling you that the job will repartition the data even though the code is telling the job to preserve the partitioning from upstream. Where this is happening open up the stage and set the input link properties to 'Clear partitioning'.

lkupProdCogs,0: Ignoring duplicate entry at table record 6357043; no further warnings will be issued for this table

This is because the dataset contains 2 records with the same key value you have specified and the reference link to that lookup dataset has not been flagged to return multiple records. If you want multiple records to be return change the setting in the lookup stage, otherwise change the data in the lookup dataset.

Maybe just change one thing at a time because often you get a warning because of a previous warning and all you need to fix is the first.