One Thousand Jobs' Sequence

sohasaid · Post by **sohasaid** » Fri Jan 22, 2010 2:24 pm

Guys,

I've around 1100 parallel jobs which migrating data from an older version of Oracle to Oracle 10g. Each job only captures the changes between data source and destination on daily basis.

The used job's pattern is:
ODBC Stage --> Transformer Stage to capture data changes --> Oracle Enterprise Stage

I need your advice about how should I implement the job sequence which run all of these jobs. In other words, what are the aspects should I talk into consideration when developing such sequence?

For example, putting each 50 or 100 jobs together on a batch job then gatehring all in a job sequence, or depending on the size of tables that jobs transfer,...etc.

Talking into consideration that the largest table contains around 2 million records.

Your suggestions will be valuable for me.

Regards.

kandyshandy · Post by **kandyshandy** » Fri Jan 22, 2010 3:38 pm

Something strike my mind... why do they need DS to migrate data from older version of Oracle to newer version of Oracle? Oracle export & import doesn't suffice? Transformer for change capture process?

Anyways, coming to your question.. it is purely based on the resources in your environment. I would say that you need to run 5 to 10 jobs in parallel using a sequence and see how they run. When you monitor this scenario, you will get an idea.

sohasaid · Post by **sohasaid** » Sat Jan 23, 2010 12:44 pm

Kandyshandy, thanks for your reply and I'll answer your two questions. Regarding migration, the data source is Oracle RDB which is an older version of Oracle with a different structure than the current one.
For the second one, I'm using the transformer stage for change data capture process because the DataStage license doesn't include some stages as 'Change Capture'.

ray.wurlod · Post by **ray.wurlod** » Sat Jan 23, 2010 3:08 pm

Change Capture, Difference and Compare stages do not require additional licensing.

I think you're confusing these with IBM's Change Data Capture (CDC)technology. There is a stage to interact with CDC but you do need to have CDC separately licensed. After all, it's a separate product.

kduke · Post by **kduke** » Sat Jan 23, 2010 5:19 pm

IBM used to supply BASIC programs to migrate these jobs from version 8 to 9 of Oracle. I would ask and see if they have a utility to do this for you if all you are doing are changing the stage types. If you are doing more than that then you are on your own.