DSXchange

Hi All,.....Mike,
Is there a upper limit for number of columns that can be compared using CRC32 function and how will be the performance be affected by having around 80 columns to be compared using crc32.Did a quick search on the discussed pls direct me to the link.
Thanks,
Ash.

The crc32 algorithm returns 4 bytes. The longer your input data string the more likely it will be that you will get an encoding collision. So there is no hard limit to adhere to, just a realization that you might be getting a duplicate shown in crc32 that represents different data.

Hi Logic,

Here is an answer to your question from one of the previous post by Michael Hester

"It's actually 2^32 or 1 in 4,294,967,296 and that is for every row. It does not mean that an incorrect CRC will be generated if you process 4294967296 rows of data, rather each row has a 1 in 4294967296 chance. Not likely that this will fail for you.

Starbucks has been using this for 3+ years and to the best of my knowledge it has not yet failed."
_________________
Michael Hester

The crc32 algorithm returns 4 bytes. The longer your input data string the more likely it will be that you will get an encoding collision.

Of course it returns 4 bytes or 32 bits.

I don't claim to fully understand the mathematics of CRC, but I do believe that string length does not really affect the result. CRC algorithms (all variants) are generally used to produce CRC's for files that can be very large. Each bit is evaluated and then shifted (either right or left) and then the next bit is evaluated until all bits have been evaluated.

Again, if anyone is interested in the mathematics or statistics behind CRC then drop me a line and I will point you in the right direction.

Craig - maybe it's time I write that FAQ?????

Thanks Mike,Naveen and Arnd.

mhester wrote:Craig - maybe it's time I write that FAQ?????

Perhaps... perhaps...