Rollup of data

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Bilwakunj
Participant
Posts: 59
Joined: Fri Sep 10, 2004 7:00 am

Rollup of data

Post by Bilwakunj »

Hi,
I'm supposed to create a job, which is kind of rollup operation. But
while doing so I need to lookup some data from the other table which has been there in the target . and I'm not very sure about the volume of the data to be looked up.
So my question is , what's the best way to do this? shd I write a pl/sql or can I use the datastage for the same.can anybody give me some benchmark no. when to go for what? or what to do?

thanks,
bilwakunj
vmcburney
Participant
Posts: 3593
Joined: Thu Jan 23, 2003 5:25 pm
Location: Australia, Melbourne
Contact:

Post by vmcburney »

You would do this in DataStage. Installing DataStage and not using it for a standard database load job like this is like buying a train ticket and then walking between stations.

You can do a lookup directly against the lookup table by joining it into a transformer. This will perform a database lookup for each row that is processed. You can also preload your lookup table into a hash file on the DataStage server for a much faster lookup speed.

Impossible to benchmark as it depends on the number of columns, the configuration of the database, the speed of the network, the number of CPUs on your DataStage server, the efficiency of your design and your ability to write this as multiple instances.
Post Reply