capture the changes in the fields!!

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
kennyapril
Participant
Posts: 248
Joined: Fri Jul 30, 2010 9:04 am

capture the changes in the fields!!

Post by kennyapril »

There are 10 fields from the source which have a key column and updated date.

Lets say the key column is 123 which has 4 records with some changes in the fields and updatedate.

Using sort stage the records are sorted with the key column and update date.

Now used an other sort stage to generate the key column for the changes in the fields.


The next step is we have to compare the previous and next record for the same keycolumn(123) for 4 records and if any change in the record for all the fields except the date forward it or else ignore it.



Is there any function in the transformer or using a stage variable ?


Please suggest me

Thanks
Regards,
Kenny
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Use pairs of stage variables to "remember" the value from the previous row. Use the Sort stage to detect the beginning of a new group (a new sort key).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
nbalegar
Participant
Posts: 9
Joined: Fri Jul 08, 2011 2:47 am

Re: capture the changes in the fields!!

Post by nbalegar »

Use the change key column property from the sort stage which tells whether the record is duplicate or not. 0 will be the unique record and 1 will indicate the duplicate record based on this value in the stage variable u can achieve it
kennyapril
Participant
Posts: 248
Joined: Fri Jul 30, 2010 9:04 am

Post by kennyapril »

stagevariable derivation
svprevvalue svcurrvalue
svcurrvalue
svvalue If svCurrValue <> svPrevValue Then 1 Else 0

In the constraint I will give svvalue=1

As I need only the records where there is change in any one
of the fields except the date field

Only thing I am not sure is what will be the svcurrvalue derivation
as I am looking for all the fields other than the date field.


Please suggest me If any changes are required


Thanks
Regards,
Kenny
Post Reply