Extacting a very large volume of data

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
loe_ram13
Participant
Posts: 35
Joined: Thu Apr 12, 2007 1:17 am

Extacting a very large volume of data

Post by loe_ram13 »

Dear All,
I have a monthly job that extracts data from table whose volume is very huge around 300 crore records totally & 60 columns.
I have to extract 15 columns from that table and also have a group by clause in it to find the count of each person who has called more than 3 times.
My present query takes 45 hrs to do this.
Any better idea??
Suggestions welcome.
Kindly help!!!
Thanks in advance!!!
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Not alot to go on, and more of a sql problem than a DataStage one. What database? I've found that your first line of defense for things like this is usually your DBA - have you taken this to them, see what they suggest?

There are all kinds of things that could help, but knowing nothing about your environment or structures makes it hard to offer specific suggestions.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply