Getting non-proper sequence result by surrogate stage

[deesh]

Hi,

I am implementing a surrogate key stage in my job, but the stage is not giving proper sequence and i have given below properties

block size - 1
next highest value - yes

but still giving un sequence number the result is coming like

EMPNO,ENAME,SKEY
1,JA,45
2,GA,64
3,KL,92
4,JK,94 Please can any one help in this case

[sunsuddhu]

Are you running your job with multiple node setting? Could you try running with single node setting and see?

[ray.wurlod]

It appears that some other job is using the same state file to generate key values.

[kandyshandy]

Sort your output by SKEY and see if they appear good.

[deesh]

Hi ray,

No other job using the same state file and if i run the on sigle node it is providing the correct result why not on 4 node configuration.......

[ray.wurlod]

What's the block size?

Is your job discarding / dropping any records?

[kandyshandy]

EMPNO,ENAME,SKEY
1,JA,45
2,GA,64
3,KL,92
4,JK,94

Is this sample data or real data that you are using? Can i assume that you have only 4 records in input?

Or if you are just showing us 4 records as a sample, then sort the output by SKEY and check whether SKEY is generated in order.

E.g. if you have 100 records in input, first record may have SKEY as 10 and 20th record may have SKEY as 1. It all depends on your partition.