CRC32 in PX

srinath0253 · Post by **srinath0253** » Wed Jan 20, 2016 5:39 am

Hi, Sorry to replying in 10 year old thread.

i have shared server container, we are using CRC32 function(for all the columns concatenated). Im using this in parallel job. it is taking too much time. we are in thought to replace this container with some parallel functions.

Could someone suggest, how to do it?. im unable to understand the logic beyond with change capture stage. CC stage needs to input links. but for CRC32 is a fucntion. how to acheive this in parallel job?

chulett · Post by **chulett** » Wed Jan 20, 2016 8:27 am

Don't apologize, simply don't do it. Start your own thread and link back to the other topic if you really think it would help. There you go, all done. Now you are the master of your own destiny.

How about more details about your job and what it is meant to accomplish. Typically all one would be using a hash value like CRC32 or MD5 would be to determine if something changed, old record to new record. In a PX job (as you noted) the Change Capture stage can provide that service, no function required. Are you having an issue setting it up, getting it to work properly?

Details, please.

srinath0253 · Post by **srinath0253** » Mon Jan 25, 2016 5:27 am

Thanks Craig..

Yes, you are correct. we are getting crc hash value to identify new records. Change capture stage is good idea.

In existing job, we calculate crc value for all columns (all columns concatenated in one string to calculate crc32). Then this value is used in a lookup with target table. I have 30 columns in that table. Then I need to lookup with all columns. It will take time.

I'm looking for any routine or script to do same as crc32. Is there anything like that in parallel jobs?

ArndW · Post by **ArndW** » Mon Jan 25, 2016 7:18 am

I'm not quite sure where you are headed on this. Are you using the "Checksum" parallel processing stage?