Logic Implementation in Transfomer

ksv2584 · Post by **ksv2584** » Wed Mar 02, 2011 2:17 am

Hi

I need create a batch id(001,002,003) for each transaction type (1234,459,3657).I have impelmented this in transformer by incrementing the batch id for change in transaction type

Since my job in running on 2 nodes iam getting duplicate batch id's,when runed in Sequential mode it is fine

Please let me know any other way to implement instead of running the transformer in sequentail mode

Thanks
Vidya

ray.wurlod · Post by **ray.wurlod** » Wed Mar 02, 2011 3:00 am

Partition your data properly using a key-based algorithm on transaction type.

MT · Post by MT » Wed Mar 02, 2011 3:17 am

Hi Vidya,

you could use the different partitions for your batch id calculation.

You could do it by having two stage variables
- a simple counter (counter = counter + 1)
- batch_id = @PARTITIONNUM + counter*@NUMPARTITIONS

Additionally you have to follow Rays advise in order to have all identical transaction types in a single partition.

best regards
Michael

ksv2584 · Post by **ksv2584** » Wed Mar 02, 2011 10:43 pm

Hi

Though i have partioned (HASH) with transaction type ,iam geting duplicate batch id 's in parallel mode

ex: Partition1 batchid partiotion2 batchid
100 1 200 1
100 1 300 2
400 2 500 3

please sugeeat Some other solution ???

ksv2584 · Post by **ksv2584** » Thu Mar 03, 2011 12:44 am

hi

One more requirement to this sam tansaction type should have sam batch id ..

Even the second by MT did nor work ..

MT · Post by MT » Thu Mar 03, 2011 12:58 am

Hi,

ksv2584 wrote:hi

One more requirement to this sam tansaction type should have sam batch id ..

Even the second by MT did nor work ..

well you should not expect a fully programmed syntax and solution - you have to add some more thoughts by your own.
The input should be sorted by transaction type.
You have to detect changes in your transaction type (via Sort stage or with the right logic in the transformer) and the you chnage the counter only when the transaction type changed.

Please test my solution again an print the
@PARTITIONNUM
counter
@NUMPARTITIONS
separately in the output to check what is going wrong.

regards
Michael

Ravi.K · Post by **Ravi.K** » Thu Mar 03, 2011 1:10 am

Or use the below one.

(((@INROWNUM -1) * @NUMPARTITIONS) + @PARTITIONNUM + 1)

ksv2584 · Post by **ksv2584** » Thu Mar 03, 2011 3:12 am

Thnaks all .. I applied my logic and its working fn ..

DSXchange