Page 1 of 1

Can the state file be read parallely in different jobs?

Posted: Wed Mar 30, 2011 1:41 am
by srinivas.nettalam
Hi All,
I recently shifted my company and here I have been told that they are running few jobs that copy data from source to a landing area.All these jobs run parallely and read same state file.Also they mentioned if the surrogate key generator generates a number eg 100 in the first job it doesn't generate 100 in other job.I always believed that surrogate key generator works similarly to the formula
(@PARTITIONNUM + (@INROWNUM-1) * @NUMPARTITIONS).I am not convinced that it can be read parallel and I assume there would be lot of discrepencies in generating the sequence numbers in this scenario.Am I right? Please let me know on this

Posted: Wed Mar 30, 2011 2:37 am
by ray.wurlod
A state file must generate unique numbers, no matter how many processes are reading it. So if 100 occurs in one job you should not expect 100 to appear in any other job reading the same state file.
Why would you believe otherwise?

Re: Can the state file be read parallely in different jobs?

Posted: Wed Mar 30, 2011 7:07 am
by chulett
srinivas.nettalam wrote:I am not convinced that it can be read parallel and I assume there would be lot of discrepencies in generating the sequence numbers in this scenario. Am I right?
Short answer? Nope.

Re: Can the state file be read parallely in different jobs?

Posted: Fri Apr 01, 2011 5:15 am
by srinivas.nettalam
chulett wrote:
srinivas.nettalam wrote:I am not convinced that it can be read parallel and I assume there would be lot of discrepencies in generating the sequence numbers in this scenario. Am I right?
Short answer? Nope.
Thanks