Capture changes at file level

[reachsam11]

The requirement is to capture changes at sequential file level (about 50 columns). Any column may have changes. Is there a way to capture the changes (insert, delete, update indicator) without specifying all 50 columns as keys in Change Capture stage?

[chowdhury99]

I don't think so. You have to specify those columns.

Thanks

[vivekgadwal]

The requirement is to capture changes at sequential file level (about 50 columns). Any column may have changes. Is there a way to capture the changes (insert, delete, update indicator) without specifying all 50 columns as keys in Change Capture stage?

You need a set of keys, which will uniquely identify a row and then the stage will compare all the values of the other fields inside a Change capture stage as far as I know. You might want to consider using the "Difference" stage.

[rhale]

We solved this problem in 7.5.2 by writing a build op that did the comparisons. Our requirement was given a set of data in the format of

Code: Select all

any number of keys|any number of data columns|row stats such as update date and update user, 

produce a row of data for each column changed based on keys. The produced row was of the format

Code: Select all

keys|old value|new value|row stats

We send it a sorted data set and it produces the rows we need.

[reachsam11]

Thanks for the repsonses..I will try them out..

[DSguru2B]

Try the unix diff command or its variations.

[ray.wurlod]

Set the Change Mode property to Explicit Keys, All Values

Use a Difference stage with All non-Key Columns are Values or a Compare stage (depending precisely on the form of output you need)

[reachsam11]

Thanks, this works!!!

Set the Change Mode property to Explicit Keys, All Values

Use a Difference stage with All non-Key Columns are Values or a Compare stage (depending precisely on the form of output you ...