Page 1 of 1

Runtime sequential file creation

Posted: Tue Jun 10, 2008 6:42 pm
by RAJEEV KATTA
I have a scenario where in the source is a file with multiple columns.
One of the column name is cost center & the values in it are dynamic.if there are 10 cost centers then target should create 10 files with cost center=1 to first file and cost center=2 to second file.Depending on no of cost centers in input being generated dynamically no of files with respective cost center data should be created in target.What stage do I need to use for the above solution.

Posted: Tue Jun 10, 2008 11:30 pm
by Minhajuddin
Well, there is no "stage" in Datastage which does this automatically. You may have create a Custom stage for this purpose. Or you can do it by creating a Parallel routine, which will do this for you.

Posted: Wed Jun 11, 2008 12:01 am
by JoshGeorge
Try a parallel routine. See if this POST helps.

Re: Runtime sequential file creation

Posted: Wed Jun 11, 2008 3:34 pm
by wahi80
RAJEEV KATTA wrote:I have a scenario where in the source is a file with multiple columns.
One of the column name is cost center & the values in it are dynamic.if there are 10 cost centers then target should create 10 files with cost center=1 to first file and cost center=2 to second file.Depending on no of cost centers in input being generated dynamically no of files with respective cost center data should be created in target.What stage do I need to use for the above solution.
Step 1: Get Distinct no: of cost centers store in a file say DistinctCost.txt. eg: Lets say a value 10 is stored
Step 2: In a sequencer do the following steps:
Use exec command stage to read the value stored in DistinctCost.txt (cat pathaname/DistinctCost.txt)
Step 3:Start Loop use the command output value of exec command stage in the max value of the loop so that the loop is executed only 10 times
Step 4: Have a job which will read your input file (the one that contains the cost center column and other data) the job should look some what like this SFile --> Transformer -->TFile

Define a parameter (#Counetrval#) in the job this parameter should use the counter value of the loop eg: in first looping 1 will be that value passed to (#Counetrval#) ..in second round 2 will be passed
In the transformer constarint give a constraint where cost center value= Counetrval
In TFile define the name of file as xx#Counetrval#, this will ensure filename as xx1 infirst looping xx2 in second looping and so on

Regards
Wah