DSXchange

Hi All,

I have a requirement where I need to first capture the changed records when compared with a full file & daily delta. (Both are datasets). Also, once the changed records are identified, these records need to be updated back to the Full file to keep it current & up-to-date.

Can you pls let me know the best way to accomplish this scenario. (All the files are datasets).

Thanks Much
Freddie.

Have you tried using Change Capture and Change Apply stage types?

Thanks Ray for your inputs The change capture/Apply seems to work. The new question that is arising in my mind is on the performance of these stages.

In the Full File, we have around 500 million records. The delta file contains 10 million (Daily). I am hearing that the change capture/change apply stages are not the right option since it takes more processing time.

Can you pls share your knowledge on this ? Also, it would be great if you have any other ideas of handling this.

Thanks Much
Freddie

You are hearing from whom? And it takes more processing time than what? Change Capture is a about as fast as the I/O throughput of your DataStage server allows.

Since you are reading files, costly options like IBM CDC or Oracle Golden Gate - utilising database changelogs to identify changed records in a database directly - are not an option.

Any other ideas? There are possibly a number of ways to use other stages as a workaround for this. But datastage provides a specialized solution for exactly the purpose you want: so use it!