DSXchange

by searching this forum I've found references about change data capture(CDC) on an oracle DB. Going trough the datastage documents I can only find the change capture stage which needs the before and after images.

Is there a stage that really does change data capture? ifso is it a seperate product?what are the requierments?(oracle version?)

Hello Jasper,

we are using the Change Data Capture stage in Px here with great success; it is documented quite well on pages 697ff. of the Parallel JobDeveloper's Guide.

In order to determine changed data you will ALWAYs need a before image, it doesn't require an after image. I am unsure of what you are really looking for, if the CDC stage doesn't work for you perhaps you might explain what you do require and someone here will help you.

if I look at this document it states that it compares a before image to an after image and the output are the changes you need to do to the before to get the after.

This is not what I mean. I'm talking about a table with 100mlj+ rows . so what i need is what is provided in the Oracle change data capture. This keeps a log of changes since last sync, so these changes can be read and applied(used as a kind of replication method between DB's).

Jasper,

both the DS method and the Oracle method you are talking about need to store information about a row's contents "before" a change is applied. Datastage does this by maintaining a before image outside of the database, whilst you are talking about maintaining a change flag and possibly a complete before-image within the database itself. The difference between the two methods lies mainly in where this information is stored. You can always use a DataStage fileset to emulate the Oracle method if you prefer that; it really does depend upon your specific requirements -- mainly whether or not you have a requirement to store the original data.

This is not what the oracle change capture does. It actually looks at the log of what happened to select the changes. So this will never need to compare the 100mlj rows after with the 100mlj before. So for large tables this will be much more performant