split records to multiple output files

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
PeterPol
Premium Member
Premium Member
Posts: 73
Joined: Wed Mar 08, 2006 8:59 am

split records to multiple output files

Post by PeterPol »

hi all,

I have a requirement to split a DBF file based on the values in one column, into a number of output files.
One output file for each split value.
So, if the input column has the following values: A, B, C, D the job should create 4 files named A.DBF, B.DBF, C.DBF, D.DBF each filled with the correctly splitted records.

Can I use the switch stage for this?
How to handle the fact that I don't know the number of output files in advance.

Peter
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

One approach would be, count the number of records (can be using link row count) and pass it as a variable into Loop activity, for each loop read the corresponding line of the file and pass the value as parameter to a job to create a file using that job parameter value.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
rajiivnb
Participant
Posts: 77
Joined: Fri Sep 10, 2004 8:38 am
Location: India

In server jobs we can use a folder stage

Post by rajiivnb »

Hi,

In Server Job I used a folder stage to do this logic. In a folder stage give your input column as the filename and the other columns concatenated give it to Record field of the Folder stage. This will create multiple files.
Post Reply