Hello All,
I have a simple EXtract job from DB2 wherein we need to dump the entire table into a flat file.
Currently I am using DB2_Connector Stage (DB2 Connector --> Copy Stage with Forced Option =True --> Sequential File ) and am able to write around 23K records per second in the target Sequential File.
Is it is the maximum performance I can get in Datastage ?
I have some tables which has 65 Million records in them and they are taking 1 hour each to get extracted.
Our goal is to complete a table extract in 15- 20 minutes.
Is there any better way to do this ? Please suggest
DB2 Extract Performance
Moderators: chulett, rschirm, roy
DB2 Extract Performance
Regards,
Shank
Shank
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Try using Table Read method in the DB2 Connector.
The Sequential File stage uses STREAMS I/O, so should not be a bottleneck. Rows/sec is a meaningless metric since we don't know how wide your rows are.
The Sequential File stage uses STREAMS I/O, so should not be a bottleneck. Rows/sec is a meaningless metric since we don't know how wide your rows are.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Your job type says server but you mention the Copy stage. If it is actually a parallel job then you could run it on a different number of nodes to improve the extract performance. It still writes your text file sequentially.
Choose a job you love, and you will never have to work a day in your life. - Confucius