How to increase performance?

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
kalpanam
Participant
Posts: 39
Joined: Sat Apr 19, 2008 6:14 am

How to increase performance?

Post by kalpanam »

Hi,

My job design is like:

DB2---------->Modify----------->DB2

I have 5,925,155 records in source, an I am doing direct load into target using write/append option. In modify I'm using NOWARN specification. Other than this I'm not doing any kind of conversion. In source I'm converting decimal to char using char(bigint(internet_order_nbr) function. This is taking 1 hour to get complete. Is there any option to increase the performance?

Please let me know..

Thanks& Regards,
Kalpana.
Kalpana Marupudi
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

That's about 1600 rows per second. The bottleneck is almost certainly the "writing" stage. But you need to establish the timings of each - the select with and without the functions, writing to a text file versus loading into a database via bulk load versus loading into a database via SQL. What indexes and constraints exist on the target?

Or maybe just lower your expectations.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
kalpanam
Participant
Posts: 39
Joined: Sat Apr 19, 2008 6:14 am

Post by kalpanam »

ray.wurlod wrote:That's about 1600 rows per second. The bottleneck is almost certainly the "writing" stage. But you need to establish the timings of each - the select with and without the functions, writing to a tex ...
As I'm not premier member,I'm not able to view the full answer of yours.Ray could you please provide me the full details?

Thanks & Regards,
Kalpana.
Kalpana Marupudi
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

The only way to see Ray's complete post is become a premier member. But he has already given you the approach in the visible part of his response. You need to determine which step is the blocking one. Replace your output DB2 with a peek - has the speed changed?
Post Reply