CRC32

ranga1970 · Post by **ranga1970** » Tue Jul 19, 2005 10:47 pm

I was told CRC#@ has some limitations, But I used it and I could myself feel any of these limitations, Could some one please tell me, what kind of limitations are there and any problems faced by them

ray.wurlod · Post by **ray.wurlod** » Wed Jul 20, 2005 12:47 am

What did you feel that the limitations were? You claim to have been able to feel them?

There is, with any such model, a very tiny probability that two different values will generate the same result using algorithms like CRC32. The probability is low enough for us to feel comfortable even processing thousands of millions of rows.

ranga1970 · Post by **ranga1970** » Wed Jul 20, 2005 2:31 am

I said

Code: Select all

 was told CRC#@ has some limitations, But I used it and I could myself feel any of these limitations, Could some one please tell me, what kind of limitations are there and any problems faced by them

But actually I meant

Code: Select all

 was told CRC#@ has some limitations, But I used it and I could myself [b] did not [/b] feel any of these limitations, Could some one please tell me, what kind of limitations are there and any problems faced by them

It was typo I miised did not and it changed the meeaning all together ; I am very soory for the confusion....

thanks

ds_developer · Post by **ds_developer** » Wed Jul 20, 2005 7:40 am

One limitation is that it does not handle null values. You will need to test for nulls using the IsNull function or convert nulls in the parameter list using the NullToEmpty and NullToZero functions.

John

I_Server_Whale · Post by **I_Server_Whale** » Wed Jul 20, 2005 7:52 am

Yes! What John said is absolutely correct. You got to make sure that there are no NULL values in any of the fields of the record that you are applying CRC32 on.

Also, I have heard that there is 1 in 4 million chance of CRC32 generating the same unique number (32-bit) for two different records. But I have never experienced it and CRC32 always worked fine for me.

Hope that adds some more knowledge to all of us.

Naveen.

logic · Post by **logic** » Wed Jul 20, 2005 4:42 pm

Hi Ranga,
Null Value handling is rather a precaution to be takeb while using CRC32. It does not limit using the function.So make sure you are handling the null values properly as john has mentioned by using nulltoempty() or nulltozero()
Ash.

mhester · Post by **mhester** » Wed Jul 20, 2005 5:06 pm

It's actually 2^32 or 1 in 4,294,967,296 and that is for every row. It does not mean that an incorrect CRC will be generated if you process 4294967296 rows of data, rather each row has a 1 in 4294967296 chance. Not likely that this will fail for you.

Starbucks has been using this for 3+ years and to the best of my knowledge it has not yet failed.

chulett · Post by **chulett** » Wed Jul 20, 2005 6:01 pm

Michael, you should seriously consider putting together a FAQ on this subject for the FAQs Forum. Save you repeating yourself whenever the topic comes up.

mhester · Post by **mhester** » Thu Jul 21, 2005 3:41 pm

Craig,

I might just do that - thanks