DSXchange

Hi

I need create a batch id(001,002,003) for each transaction type (1234,459,3657).I have impelmented this in transformer by incrementing the batch id for change in transaction type


Since my job in running on 2 nodes iam getting duplicate batch id's,when runed in Sequential mode it is fine

Please let me know any other way to implement instead of running the transformer in sequentail mode

Thanks
Vidya

Partition your data properly using a key-based algorithm on transaction type.

Hi Vidya,

you could use the different partitions for your batch id calculation.

You could do it by having two stage variables
- a simple counter (counter = counter + 1)
- batch_id = @PARTITIONNUM + counter*@NUMPARTITIONS

Additionally you have to follow Rays advise in order to have all identical transaction types in a single partition.

best regards
Michael

Hi

Though i have partioned (HASH) with transaction type ,iam geting duplicate batch id 's in parallel mode

ex: Partition1 batchid partiotion2 batchid
100 1 200 1
100 1 300 2
400 2 500 3


please sugeeat Some other solution ???

hi

One more requirement to this sam tansaction type should have sam batch id ..


Even the second by MT did nor work ..

Hi,

ksv2584 wrote:hi

One more requirement to this sam tansaction type should have sam batch id ..


Even the second by MT did nor work ..

well you should not expect a fully programmed syntax and solution - you have to add some more thoughts by your own.
The input should be sorted by transaction type.
You have to detect changes in your transaction type (via Sort stage or with the right logic in the transformer) and the you chnage the counter only when the transaction type changed.

Please test my solution again an print the
@PARTITIONNUM
counter
@NUMPARTITIONS
separately in the output to check what is going wrong.


regards
Michael

Or use the below one.

(((@INROWNUM -1) * @NUMPARTITIONS) + @PARTITIONNUM + 1)

Thnaks all .. I applied my logic and its working fn ..