Difference stage issues lots of warnings.

vmcburney · Post by **vmcburney** » Thu Mar 31, 2005 12:56 am

When I use the Difference stage I get three warning messages in my log for each column being compared:

Difference_1:When checking operator: Dropping component "fielda" because of prior component with same name
Difference_1:When checking operator: Dropping component "fieldb" because of prior component with same name
Difference_1:When checking operator: Dropping component "fielda" because of prior component with same name
Difference_1:When checking operator: Dropping component "fieldb" because of prior component with same name
Difference_1:When checking operator: Dropping component "fielda" because of prior component with same name
Difference_1:When checking operator: Dropping component "fieldb" because of prior component with same name

Any way to avoid these warnings? I like using the Difference stage, it is more flexible then the Data Capture stage as it supports non matching columns, but all these warnings is breaking my 50 warning limit. This is version 7.1 so I do not have the message handler.

jasper · Post by **jasper** » Thu Mar 31, 2005 2:10 am

We had the same issue. I think the number of warnings per field is dependant on the number of parallel processes you define.

So set this stage to sequential and hope you need less then 50 fields. The only real solution we found is actually upgrading and use the messagehandler.

roy · Post by **roy** » Thu Mar 31, 2005 4:47 am

Hi,
this sounds like a lookup stage warnings not a difference stage

never seen such warnings from diff stage

T42 · Post by **T42** » Thu Mar 31, 2005 8:17 am

Hmmmmm....

I haven't used the difference stage. If you still have this problem on Monday, private message me, and I'll investigate.

Edit: You do know that you can change your warning limit defaults? I would bet that you do know that. Hope you don't have one of those clients that demands 50 warnings "just because Ascential set it that way."

kollurianu · Post by **kollurianu** » Thu Mar 31, 2005 9:28 am

Is there something called difference stage on DS7.5 , i didnot see it in the
document . if so, what does that stage do.

Thank you all

roy · Post by **roy** » Thu Mar 31, 2005 10:09 am

Hi,
In EE there is a difference stage.
taking 2, sorted hash partitioned and with same meta data, data setes resulting in a set with the diff column 0-3 indicating no-change,insert,update,delete options (EE developers guide has exact number=meaning info)

kollurianu · Post by **kollurianu** » Thu Mar 31, 2005 10:31 am

does EE mean enterprise edition . or Does that exist on DS7.5 server editon which iam working on.

Thank you.

ray.wurlod · Post by **ray.wurlod** » Thu Mar 31, 2005 4:08 pm

Enterprise Edition (EE) is a superset of Server Edition. With Server Edition you can create server jobs and job sequences. With Enterprise Edition you can create server jobs, parallel jobs and job sequences.

Enterprise Edition MVS is a superset of Enterprise Edition. With Enterprise Edition MVS you can create mainframe jobs as well.

kollurianu · Post by **kollurianu** » Thu Mar 31, 2005 4:10 pm

Thank you Ray very much for ur answers.

vmcburney · Post by **vmcburney** » Thu Mar 31, 2005 6:10 pm

I have sent a simple difference stage example through to Ascential support to get the final word. I am calling the job from a Sequence job so I cannot get around the 50 warning limit.

vmcburney · Post by **vmcburney** » Thu Mar 31, 2005 11:17 pm

I found a message in the Orchestrate Operators Reference that did not make it into the DataStage documentation that suggests we should not use this stage:

The diff operator has been superseded by the changecapture operator. While the diff operator has been retained for backwards compatibility, Ascential recommends that you use the changecapture operator for new development.

I think Difference has a subset of the functionality of Change Data Capture. I thought it was a bit more flexible in the selection of of non value fields but closer inspection of the Orchestrate doco reveals the behaviour to be the same.

roy · Post by **roy** » Fri Apr 01, 2005 1:06 am

Hi,
Good to know Vincent Thanks(asbout the dsiff/change capture).
Iguess that warnlimit thru sequence jobs is another one for the wish list.

T42 · Post by **T42** » Sun Apr 03, 2005 7:28 pm

I could have sworn that if you call it via the sequencer, it does pull the limits you imposed on it and carry that limit through the jobs.

At least, it doesn't abort on 50 messages for me. But then I generally turn off the warning limit on my side. I will have to investigate tomorrow.

vmcburney · Post by **vmcburney** » Sun Apr 03, 2005 9:11 pm

I'd forgotten about that, the sequence job passes run time settings through to the child job, so the warning limit is passed through. The number of rows to process is also passed through for called server jobs.