Warning : Unbalanced input

[Havoc]

I have a Job Design which looks like this

Code: Select all

INCOMING->Merge->Merge-(hash,unique sort)->Lookup->rest->DataSet

All Merges and Lookups have a DB2 API stage as the master link and reference link respectively. Lookup stage and DB2 API stages are constrained to 1 node.

I'm getting the following warnings which werent issued before:

Code: Select all

APT_ParallelSortMergeOperator,0: Unbalanced input from partition 0: 10000 records buffered
APT_ParallelSortMergeOperator,0: (no further unbalance warnings will be issued for this partition)
APT_ParallelSortMergeOperator,0: Unbalanced input from partition 2: 10000 records buffered
APT_ParallelSortMergeOperator,0: (no further unbalance warnings will be issued for this partition)

Three other jobs append to the dataset created in this job(in Overwrite) mode. The weird thing is, if the dataset is deleted and this job is run , the warning is NOT issued. But if this job is re-run after the other three jobs have run(without dataset delete) then this warning is being issued.

Any idea why this would be happening? I tried other search results but couldnt see how I could apply the provided solutions to this problem.

Thanks in advance

[kumar_s]

The type of partition that you have chosed might cause the data to be unequally partitioned. And is the other job which uses the dataset, as different nood pool configuration?

[Ragunathan Gunasekaran]

Hi ,
Just check the partition type that you have chosen for the master link as DB2 and check whether the partition type for the reference link is entire or Auto ( By default reference stages would have a Entire partitioning).

The dataset also stores the partition information .Hence while you write the data to the dataset with imbalanced partitions then datastage may issue an warning due to that imbalanced partition write .