Surrogate Key Generator Stage

JPalatianos · Post by **JPalatianos** » Mon Jun 15, 2009 6:22 am

Hi,

We have a job that looks like this....When we execute we error out on the link from the Surrogate Key generator to the ODBC stage with the errors below.

ODBC stage
|
|
Sequential File---->> Look Up Stage-->transformer stage-->> ODBC stg
|
|
Surrogate Key Gen stg ----> ODBC stage

Warning # 1:
main_program: Operator "parallel APT_KeyGenerator in skNewPlan" is not wave aware; the operator will be reset and rerun on each wave if multiple waves present. This may lead to incorrect results and memory issues. Update the operator to make it wave aware and calls setWaveAware() in describeOperator() to inform the framework that the operator knows how to handle waves. [sc\sc.C:4252]

Fatal Error # 1:

node_node0: Player 7 terminated unexpectedly. [processmgr\player.C:157]

Fatal Error # 2:

main_program: Step execution finished with status = FAILED. [sc\sc_api.C:242]

Any suggestions or ideas would be appreciated.

Thanks - - John

JPalatianos · Post by **JPalatianos** » Mon Jun 15, 2009 6:25 am

The post should say Release 8X
Job Type: Parallel

ArndW · Post by **ArndW** » Mon Jun 15, 2009 6:26 am

Since you aren't doing wave processing you can ignore the warning; but the failure message(s) in the logs do contain additional information on the actual cause, could you post the complete and unabridged error messages please?

JPalatianos · Post by **JPalatianos** » Mon Jun 15, 2009 6:30 am

Sorry once again...my diagram didn't show up how I planned it....
The ODBC stage on top should be going into the Lookup stage and the Surrogate Key Gen stage should be coming out of the look up stage as well.

JPalatianos · Post by **JPalatianos** » Mon Jun 15, 2009 7:11 am

Hi ArndW,
Is this what you were looking for?

Server:NJROS1BBLA0602
Project:CLR_NQRPT
Job No:2
Job name:jb_ld_dim_metrix_plan
Invocation:
Event Number:753
Event type:Fatal
User:NJROS1BBLA0602\DSADM
Timestamp:6/15/2009 8:11:32 AM
Message Id:IIS-DSEE-TFPM-00192
Message:
node_node0: Player 7 terminated unexpectedly. [processmgr\player.C:157]

Server:NJROS1BBLA0602
Project:CLR_NQRPT
Job No:2
Job name:jb_ld_dim_metrix_plan
Invocation:
Event Number:754
Event type:Fatal
User:NJROS1BBLA0602\DSADM
Timestamp:6/15/2009 8:11:32 AM
Message Id:IIS-DSEE-TFSC-00011
Message:
main_program: Step execution finished with status = FAILED. [sc\sc_api.C:242]

If not please let me know how to get teh more detailed messages. These are copied from director.

Thanks - - John

JPalatianos · Post by **JPalatianos** » Fri Jun 19, 2009 6:30 am

Hi,
I opened a PMR with IBM and just recieved the following:

Hello John,
Engineering has provided the following:
This is known issue with surrogate key in 8.0.1
We are working on a fix for 8.5. The usual workaround for this is to use the key generator in the database (Oracle, DB2 etc).

Please try the workaround and let me know the results.

Thank you.

chulett · Post by **chulett** » Fri Jun 19, 2009 7:13 am

Eight dot five? So, sometime after the Apocalypse in 2012.

bobyon · Post by **bobyon** » Fri Jun 19, 2009 7:16 pm

chulett wrote:Eight dot five? So, sometime after the Apocalypse in 2012.

Aren't you being a little optimistic? That might be the announcement date, however...........

JPalatianos · Post by **JPalatianos** » Sat Jun 20, 2009 7:46 am

And they ask me to respond to....
AmI staisfied? Sure we spend all this money and there going to ask us to do the ETL in the database or withing shell scripts!!!

Hello John,
Since the job is working with the workaround and there is nothing additional I can assist with for this PMR, may I close it?
Are you satisfied with the resolution?
Please let me know.
Thank you.

Sreenivasulu · Post by **Sreenivasulu** » Sun Jun 21, 2009 4:30 am

Wave is new feature in 8.00 and hence all josb need to be wave aware

Regards
Sreeni

JPalatianos wrote:Hi,

We have a job that looks like this....When we execute we error out on the link from the Surrogate Key generator to the ODBC stage with the errors below.

ODBC stage
|
|
Sequential File---->> Look Up Stage-->transformer stage-->> ODBC stg
|
|
Surrogate Key Gen stg ----> ODBC stage

Warning # 1:
main_program: Operator "parallel APT_KeyGenerator in skNewPlan" is not wave aware; the operator will be reset and rerun on each wave if multiple waves present. This may lead to incorrect results and memory issues. Update the operator to make it wave aware and calls setWaveAware() in describeOperator() to inform the framework that the operator knows how to handle waves. [sc\sc.C:4252]

Fatal Error # 1:

node_node0: Player 7 terminated unexpectedly. [processmgr\player.C:157]

Fatal Error # 2:

main_program: Step execution finished with status = FAILED. [sc\sc_api.C:242]

Any suggestions or ideas would be appreciated.

Thanks - - John

Sreenivasulu · Post by **Sreenivasulu** » Sun Jun 21, 2009 4:31 am

Wave is new feature in 8.00 and hence all josb need to be wave aware

Regards
Sreeni

JPalatianos wrote:Hi,

We have a job that looks like this....When we execute we error out on the link from the Surrogate Key generator to the ODBC stage with the errors below.

ODBC stage
|
|
Sequential File---->> Look Up Stage-->transformer stage-->> ODBC stg
|
|
Surrogate Key Gen stg ----> ODBC stage

Warning # 1:
main_program: Operator "parallel APT_KeyGenerator in skNewPlan" is not wave aware; the operator will be reset and rerun on each wave if multiple waves present. This may lead to incorrect results and memory issues. Update the operator to make it wave aware and calls setWaveAware() in describeOperator() to inform the framework that the operator knows how to handle waves. [sc\sc.C:4252]

Fatal Error # 1:

node_node0: Player 7 terminated unexpectedly. [processmgr\player.C:157]

Fatal Error # 2:

main_program: Step execution finished with status = FAILED. [sc\sc_api.C:242]

Any suggestions or ideas would be appreciated.

Thanks - - John

chulett · Post by **chulett** » Sun Jun 21, 2009 7:58 am

Sreenivasulu wrote:Wave is new feature in 8.00 and hence all jobs need to be wave aware

You do realize how singularly unhelpful posting something like this without any explanation is, yes? The only way posted this would help would be if you also included a description of what a "wave" is and some clue how to make a job "wave aware". If there even is such a thing.

If you read the warning logged, you'll see that it is the operator that needs to be "wave aware" and properly communicate that to the parallel framework, something we as users have absolutely no control over.

ps. Forgot to add that waves are not a new feature, AFAIK they've always been in the product.

ray.wurlod · Post by **ray.wurlod** » Sun Jun 21, 2009 5:43 pm

JPalatianos wrote:And they ask me to respond to....
AmI staisfied? Sure we spend all this money and there going to ask us to do the ETL in the database or withing shell scripts!!!

Hello John,
Since the job is working with the workaround and there is nothing additional I can assist with for this PMR, may I close it?
Are you satisfied with the resolution?
Please let me know.
Thank you.

DID you let them know?

If so, did they respond your no doubt constructive criticism?

chulett · Post by **chulett** » Sun Jun 21, 2009 8:23 pm

Just wanted to pop back in and tell Sreeni I really didn't mean to single them out or anything, it's just that that is one of my pet peeves and I guess I was feeling a little... peevey... this morning.

priyadarshikunal · Post by **priyadarshikunal** » Mon Jun 22, 2009 2:37 am

I have seen such problems only when the surrogate key generator is being used with ODBC Connector. Are they talking about that problem only or some more scenarios?

DSXchange

Surrogate Key Generator Stage

Surrogate Key Generator Stage

Re: Surrogate Key Generator Stage

Re: Surrogate Key Generator Stage

Re: Surrogate Key Generator Stage