How to make any stage wave aware???????

[priyadarshikunal]

Code: Select all

Operator "parallel APT_KeyGenerator in SKG_Ins" is not wave aware; the operator will be reset and rerun on each wave if multiple waves present. This may lead to incorrect results and memory issues. Update the operator to make it wave aware and calls setWaveAware() in describeOperator() to inform the framework that the operator knows how to handle waves.

I am getting this warning while running a job in two stages
1. Surrogate key generator
2. Oracle enterprise stage.

Job Design:

I am extacting data from ODBC connector stage then performing lookup
the reject link is connected to a Surrogate key generator stage then result with surrogate key goes to a transformer then the data is inserted to the database using Oracle enterprise stage.

How can i remove this warning?

I am getting two more errors
1.

Code: Select all

write() failed: Broken pipe

2.

Code: Select all

write() failed: Bad file number

Why I am getting this error and how can i remove those errors?

[priyadarshikunal]

Now I have changed my job a lot according to other posts for last two errors
Now I am getting only one warning i.e.

Code: Select all

main_program: Operator "parallel APT_KeyGenerator in SKG_Ins" is not wave aware; the operator will be reset and rerun on each wave if multiple waves present. This may lead to incorrect results and memory issues. Update the operator to make it wave aware and calls setWaveAware() in describeOperator() to inform the framework that the operator knows how to handle waves.

I changed my design like:

odbc conn->Transformer->lookup->SurrogateKeyGen->OracleEnterprise
^
|
Oracle Enterprise

Can any one suggest how to remove that warning?

[eostic]

Hi...

I don't know how to get rid of it, but it's also unlikely to be important. Mulitple "waves" are usually only an issue with WISD or RTI jobs, or other "real time" scenarios, such as sourcing from MQSeries. Each "wave" marks a single logical request that is flowing thru an otherwise "always on" job. Without end-of-wave support, Stages like aggregator would simply wait forever and never release rows in an always running situation. In a more simplistic batch scenario, multiple waves aren't an issue.

This Stage is probably one to be careful of if you start developing WISD jobs.

Ernie

[priyadarshikunal]

Hi...

I don't know how to get rid of it, but it's also unlikely to be important. Mulitple "waves" are usually only an issue with WISD or RTI jobs, or other "real time" scenarios, such as sourcing from MQSeries. Each "wave" marks a single logical request that is flowing thru an otherwise "always on" job. Without end-of-wave support, Stages like aggregator would simply wait forever and never release rows in an always running situation. In a more simplistic batch scenario, multiple waves aren't an issue.

This Stage is probably one to be careful of if you start developing WISD jobs.

Ernie

Thanks Ernie for your post.
That really helped me to understand the waves,

here in my job, I found a work-around that "don't use ODBC connector to feed data in the job in which surrogate key generator stage is used" i.e. break the job in two parts.

But that's not the solution, its just an approach(workaround) to remove that warning since ODBC connector stage can handle waves.

Is this the right way to remove that warning???

If not please suggest me any other way!!

Thanks in advance

[bcarlson]

We are getting the same warning. We just recently migrated to v8 and the old v751a version of this same job had no warning/issues. The warning is tied to a buildop with very simple logic:

Code: Select all

//---------------------------------------------------------------------- 
// Create the next pseudo customer id: 
//---------------------------------------------------------------------- 
APT_Decimal tmpDecimalID(11,2); 
APT_DFloat tmpFloatID; 
                   
tmpFloatID = in.MIN_ID.asDFloat(); 
tmpFloatID = tmpFloatID + in.seq_nr; 
out.AL_CUSTOMER_ID.assignFromDFloat(tmpFloatID); 
                   
out.OPR_SOURCE_SYS_NR     = 80; 
out.AIMEDW_LOAN_NUMBER    = in.AIMEDW_LOAN_NUMBER; 
out.AIM0020_CLIENT_NO     = in.AIM0020_CLIENT_NO; 

No special functions being called, only standard DS built-in functions.

Did you ever find a true resolution? I am loath to simply ignore a warning, especially when I don't understand its source. Since we didn't get this in v7, I am assuming this is directly related to v8.

Anyone?

Brad.

[ArndW]

Actual, there are switches you can set to make the build-op look like it is wave aware. I am not at my system now to check how I did it, but I do think that this switch is documented.

As Ernie has already stated, unless you are doing wave processing with MQ-Series (as we are in this case), you can demote this warning message to informational or suppress it altogether and not have to worry about it.

[bcarlson]

That's good to know, I appreciate the input. Quick question - any idea why this buildop and/or job would have issues, but other jobs with buildops have had no similar issues? Have you seen any patterns in terms of what could trigger the warning?

We'll look into the buildop option you mention. Thanks again!

Brad.

[eostic]

...hard to say (identify a pattern) without seeing the job itself....right now my only guess is that EE had a lot of anomalies in wave mangagement....maybe a correction to one of them has led to it analyzing things a bit differently......

....and as a guess to patterns, do you have a single stream, split into more than one (like via Copy) and then back together via Join or similar? ....that might be a scenario that I think would trigger wave "thinking" at run time...

Ernie