Partitioner Stage outputs zeros

[ian_bennett]

When using the partitioner stage with either Modulus or Hash option it outputs "0" for every column. Works fine for Round Robin.

I am using a single Decimal column for the hash key and was going to use Modulus to ensure that records with the same value are sent down the same output link.

Am I missing something obvious here?

[roy]

Hi,
Sounds like a bug here.
There is no way for me to get 6 version to test so I'd say contact your support.
Good Luck,

[rleishman]

Make sure that the key columns in all of the Output links of the Link Partitioner match those of the Input columns. Do the same for the Link Collector.

This has caused me grief in the past by producing unpredictable results.

Tell us if it works.

[ian_bennett]

Yep, all the columns match. Works fine for Round-Robin so shouldnt be anything like that.

I dont use a link collector. The partitioner is to allow me to do a parallel sort and then each stream will insert to database independently.

Make sure that the key columns in all of the Output links of the Link Partitioner match those of the Input columns. Do the same for the Link Collector.

This has caused me grief in the past by producing unpredictable results.

Tell us if it works.

[rleishman]

Just to make sure, you did check the KEY attribute on each output link matched the input link, didn't you (sorry - just me being pedantic). Just to be doubly sure, use right-click / propogate columns / {input link} / All links (if avail. in your version).

Also, make sure that the Derivation is populated in the output links (if it works for Round-Robin, then I'm sure it is OK)

The help file says that the Modulus function requires an Integer data type column. Try casting your decimal to Integer with a Transformer in a prior stage. Doesn't explain why Hash is not working though.

If you have no joy, you could always just do it with a Transformer and a different constraint on each output link.

[ian_bennett]

Thanks

Propogating did the trick!

Not sure what the problem was though since I had originally defined the columns by using a Transformer stage to copy columns and then moving the link to the Partitioner stage (old habits die hard).

Just to make sure, you did check the KEY attribute on each output link matched the input link, didn't you (sorry - just me being pedantic). Just to be doubly sure, use right-click / propogate columns / {input link} / All links (if avail. in your version).

Also, make sure that the Derivation is populated in the output links (if it works for Round-Robin, then I'm sure it is OK)

The help file says that the Modulus function requires an Integer data type column. Try casting your decimal to Integer with a Transformer in a prior stage. Doesn't explain why Hash is not working though.

If you have no joy, you could always just do it with a Transformer and a different constraint on each output link.

[ShaneMuir]

Just an update to this problem, for everyones info. I don't think it was caused by the propogating of the columns or lack thereof.

I was having the same issue in ver6, where I was using the sequence

Code: Select all

                                  /----> Seq1
                                 /
OraOCI8 ---> Trf ---> Partition <
                                 \
                                  \----> Seq2

No matter which partition algorithm I used the result was always the same: all columns in the output field would be populated with 0's. The record counts would be right (ie the partition actually worked and sent the correct number of rows down each stream) but all the values would be 0.

Anyway I found that if I added a seq file between the Trf and the partition that it would work as expected. Further digging revealed that if I activate Inter-process row buffering on the performance tab in the job properties that it would work without the intermediate sequential file.

Just thought I would share that with everyone.

[I_Server_Whale]

You bet. Thanks for Sharing!

Whale.

[ray.wurlod]

The documentation for Link Partitioner specifically notes that inter-process row buffering must be enabled when this stage type is used, because it creates a process for itself. See top of page 13-2 in Server Job Developer's Guide for version 7.5