DSXchange

Hi all,

I generate a sequence in the transformer, using @numpartition and @PARTITIONNUM.The problem i faced is - there are no duplicate sequence numbers, but the count value is missing.I mean the following is the job flow:

source dataset-> transformer stage->xml file

I am sending 100 records to the transformer and should not it give the same 100 as last records count? but it gives counts more than 100 ...(110,120,345 for eg.,) and it misses some values in the middle (35,56,78, for eg.,)

Note - logic to create the sequence is -

I have a stage vairable, countvar whose intial value is @PARTITIONNUM - @NUMPARTITIONS + 1 and in the derivation of the vaiable, i am using this - countvar + @NUMPARTITIONS.
This variable is integer and as per my requirement i have to left pad zeros, am using an output column and used the function, str to determine the number of left padded zeros and passing it to next stage.

Pls help me finding the problem.

Thanks in advance!!

Hi,
You can refer the FAQ related to this sequence number generation.
This has been discussed earliar. Since the number been generated in different partition, you cannot expect the nubmers to be gererated in a sequence order. It depends on the number of records that flows in each partition. If your partition forces the records to flow majorly in one partition you will yield different result.

You need to ensure that you are doing balanced partitioning while using this. Meaning, if you have 100 record and 2 partitions ensure you are partitioning them in equal halves i.e. 50 in one and 50 in other.