compare values in transformer stage

[chulett]

You've got all the logic you need right here.

One issue could be your incoming data is not sorted properly. Another consideration could be the number of nodes this job is running on, if it is running on more than one node you'd need make sure you hash partition properly on the value(s) you are using to check for duplicates.

So... what exactly about this doesn't seem to be working for you?

[vbarca]

You need two stage variables

IsDup - set initial value to 0 and set the derivation as:
IF DSLink2.deptno = LastValue THEN 1 ELSE 0

LastValue - set initial value to 0 (or any value that will not match anything coming in from the source) and set the derivation as: DSLink2.deptno

Then in your output link you need a constraint: IsDup = 0

That should do it.

[chulett]

So... in other words, exactly what's already ready been posted as being used.

Me, personally, I would rely on the expression automatically assigning the proper values, there's no need to do it "manually". Then treat the result as the Boolean it is.

IsDup: DSLink2.deptno = LastValue

Constraint: Not(IsDup)

[ravi7487]

Thanks for the reply.

I got the output correct with the below formula.

Intial value = 0 for IsDup
Initial value =0 for currentvalue ( which is different from firstvalue in source)

If DSLink2.deptno = currentvalue then 1 else 0 = IsDup
DSLink2.deptno= currentvalue.

but, i do not want to initialize currentvalue as manually. any alternative mthod? pelase let me know

[Vidyut]

The logic seems fine....jst make sure ur data is sorted

[vikasjawa]

Ravi,
The logic provided above is fine. It seems the issue is with the way you are partitioning the data. Try doing a hash partition on deptno.

[jwiles]

I got the output correct ... but, i do not want to initialize currentvalue as manually. any alternative mthod? pelase let me know

Is there any particular reason WHY you don't want to initialize currentvalue "manually"? It's both perfectly acceptable and is the correct method per standard programming practices in almost any programming language. There is no need or reason to use any alternative methods to accomplish this job.

[chulett]

As noted, you are not "manually" initializing them, you are properly initializing them using the mechanism provided for exactly that purpose. Make sure you understand, however, that it doesn't need to be a hard-coded value there, you can use an expression or function or job parameter there for example.

ps. There's no need to initialize IsDup since it is set before ever being tested. Doesn't hurt, of course, but serves no purpose.

[ravi7487]

Thank you all for replying,
so, when comparing the values in transoformer,
for the first incoming row to transformer, the initial value for the "previousvalue" should be assigned something that will not match with the first incoming row.

Am i correct? Please let me know. Thanks.

[chulett]

Thank you for clarifying, But what initial value should I put in for the previous value?

Whatever non-null value will not match with anything that might come in on the first record.

[ravi7487]

Thanks Chulett,

how will i know what the first record is, so i can have a different value for previousvalue? is there any technique? please let me know

[vbarca]

You have to know your data. Whatever valid valid values your company uses for deptno you don't want to use for your initial value. Use something else.

[ravi7487]

Thank you very much, Thanks a ton

DSXchange members helped me a lot.

Thanks again.