DSXchange

Has anyone designed how they do Initial loads and Incremental's in Datastage. We have a requirement to bulk load the initial load and subsequently process delta's.

What I am try to stay away from is coding two jobs. My thought is to code one job and use a parameter to control which DB2 connector gets used. One being the Bulk loader and the other being the Insert then update.

I realize that the initial will be used minimal times. I am also not sure whether the Bulk loader will use up a connection in the job for nothing when not used

Has anyone used a different methodology?

Thanks

You can do delta jobs then pick a date range from the earliest date in the source you need. A lot of things depend on design. If all your source keys are in the target so you do not post duplicates then you should not have issues.

We are working with 300+ million rows for the first initial load, subsequent deltas are manageable. It is the first load that causes problems due to slow inserts loading. We get 4 times more performance with the Bulk loader. I was just wondering if anyone has used one job with two DB2 connectors, one for initial load(bulk) and one for deltas(inserts). Processing controlled by the value of the parameter.

Raftsman wrote:I am also not sure whether the Bulk loader will use up a connection in the job for nothing when not used

It will.

FWIW, in the very small number of times where I've had to differentiate between an initial and subsequent loads, I've created two jobs.