CRC32
Moderators: chulett, rschirm, roy
CRC32
I was told CRC#@ has some limitations, But I used it and I could myself feel any of these limitations, Could some one please tell me, what kind of limitations are there and any problems faced by them
RRCHINTALA
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
What did you feel that the limitations were? You claim to have been able to feel them?
There is, with any such model, a very tiny probability that two different values will generate the same result using algorithms like CRC32. The probability is low enough for us to feel comfortable even processing thousands of millions of rows.
There is, with any such model, a very tiny probability that two different values will generate the same result using algorithms like CRC32. The probability is low enough for us to feel comfortable even processing thousands of millions of rows.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
I said
But actually I meant
It was typo I miised did not and it changed the meeaning all together ; I am very soory for the confusion....
thanks
Code: Select all
was told CRC#@ has some limitations, But I used it and I could myself feel any of these limitations, Could some one please tell me, what kind of limitations are there and any problems faced by them
Code: Select all
was told CRC#@ has some limitations, But I used it and I could myself [b] did not [/b] feel any of these limitations, Could some one please tell me, what kind of limitations are there and any problems faced by them
thanks
RRCHINTALA
-
- Premium Member
- Posts: 224
- Joined: Tue Sep 24, 2002 7:32 am
- Location: Denver, CO USA
-
- Premium Member
- Posts: 1255
- Joined: Wed Feb 02, 2005 11:54 am
- Location: United States of America
Yes! What John said is absolutely correct. You got to make sure that there are no NULL values in any of the fields of the record that you are applying CRC32 on.
Also, I have heard that there is 1 in 4 million chance of CRC32 generating the same unique number (32-bit) for two different records. But I have never experienced it and CRC32 always worked fine for me.
Hope that adds some more knowledge to all of us.
Naveen.
Also, I have heard that there is 1 in 4 million chance of CRC32 generating the same unique number (32-bit) for two different records. But I have never experienced it and CRC32 always worked fine for me.
Hope that adds some more knowledge to all of us.
Naveen.
It's actually 2^32 or 1 in 4,294,967,296 and that is for every row. It does not mean that an incorrect CRC will be generated if you process 4294967296 rows of data, rather each row has a 1 in 4294967296 chance. Not likely that this will fail for you.
Starbucks has been using this for 3+ years and to the best of my knowledge it has not yet failed.
Starbucks has been using this for 3+ years and to the best of my knowledge it has not yet failed.
Mike Hester
mhester@petra-ps.com
mhester@petra-ps.com