Handling huge volume of data

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
naren6876
Participant
Posts: 233
Joined: Mon Feb 07, 2005 7:19 pm

Handling huge volume of data

Post by naren6876 »

Hi All,

I have 60 mln records in source and want to split (based on rownum)and process them with multi instance job in the Sequnce job with loop (12 times).
For each instance, i would like to take 5 mln rows .

My database is Oracle10g.

Is it possible to split the records based on instance and rownum?.

if not,Please shed me somelight with better approach?

Thanks in advance.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

First let us know what "process" means, what do you actually have to do with this data? And what is your target?
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Just dream up appropriate WHERE clauses (probably with job parameters for the constant(s)) for extraction SQL.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply