Compare value with previous record.

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
videsh77
Premium Member
Premium Member
Posts: 97
Joined: Thu Dec 02, 2004 10:43 am
Contact:

Compare value with previous record.

Post by videsh77 »

How can I compare 2 records at the same time?

Prev rec A,B
Current rec B,C

col1 of current record if equals to col2 of prev record, I would like to redirect that value (in this case it is B) to other file.

How can we achieve this functionality using DataStage?

I thought of using lookup file stage, considering @OUTROWNUM, but it is aborting. I felt I might be wrong in the approach, can anyone suggest the right approach?
Thanks with regards,
videsh.
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

If you are checking for Duplicate records and trying to divert to other file, use Sort stage to sort based on a column and assign "CreateKeyChageColumn". Using Transformer or Filter, divert the records whose values are 0s.
This has been discussed several times in forum.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
csrazdan
Participant
Posts: 127
Joined: Wed May 12, 2004 6:03 pm
Location: Chicago IL

Post by csrazdan »

kumar_s wrote:If you are checking for Duplicate records and trying to divert to other file, use Sort stage to sort based on a column and assign "CreateKeyChageColumn". Using Transformer or Filter, divert the records whose values are 0s.
This has been discussed several times in forum.
Kumar,
Can you please explain what is "CreateKeyChageColumn"
Assume everything I say or do is positive
videsh77
Premium Member
Premium Member
Posts: 97
Joined: Thu Dec 02, 2004 10:43 am
Contact:

Post by videsh77 »

kumar_s wrote:If you are checking for Duplicate records and trying to divert to other file, ...This has been discussed several times in forum.
No I am not checking for duplicate records. In my file anywhere I can have col2 value of previous record could be same as col1 value of current processing record.

Also can you please pass on the single link for the previously discussed topic? I tried search with the string in subject line.
Thanks with regards,
videsh.
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

csrazdan wrote: Kumar,
Can you please explain what is "CreateKeyChageColumn"
Its the option available in Sort stage. Making this to True creates the column KeyChange. This column is set to 1 for the first row in each group where the value of the sort key changes. Subsequent records in the group have the column set to 0.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

videsh77 wrote: No I am not checking for duplicate records. In my file anywhere I can have col2 value of previous record could be same as col1 value of current processing record.
Ok, you need to compare Previous value of Col2 to current value of Col1 or the vice versa.

User Transformer stage,

User stage variables

Code: Select all

vCheck = If Input.Col1 = vPreCol2 Then 1 Else 0
vPreCol2 = Input.Col2
Based on the condition of vCheck flag, you can divert the records to various links.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
videsh77
Premium Member
Premium Member
Posts: 97
Joined: Thu Dec 02, 2004 10:43 am
Contact:

Post by videsh77 »

Thanks, it worked.
Thanks with regards,
videsh.
Post Reply