Urgent question: How to dynamic set file names parameter

[xmw2000]

Dears,

sorry to bother! I have one urgent question, hope you can give some suggestions! Thanks in advance.

From the logic, the job is very simple. It just analysis one input file, then according to the file content, output the related results to different sequential files and these sequential file name is dynamic.

Take an example, if input file contains "American", the it will put all the American data into "American.txt" file, while if contains "flower", the output file name should be "flower.txt".

Of course, we can draw all stages in one panel of DS designer, but it's very, very tedious because we will have 600 more output files. So is there one way to dynamic update file name parameter?

[kcbland]

Welcome to our forum.

Your requirement is not one that DataStage is meant to solve. Of course, you could use job parameters for the file names, but because you don't know the file names ahead of time you can't use job parameters.

What I would suggest is that you add a column to the output file, say in the first position, that is a partitioning value. Then, after processing all data into single output file, you can use a simple ksh or perl script to break it into the indeterminate number of output files. That is really easy to do in ksh. If the data volume is large, go for a perl, java, or c program. If the data volume is small, ksh is probably fine.

[xmw2000]

It's a great idea. I will try to use it!

Appreciate for your help!

[sb_akarmarkar]

Try with folder stage and aggregator stage...

Thanks,
Anupam

[kumar_s]

Try with folder stage and aggregator stage...

Thanks,
Anupam

Do you mean writing the record id as file name for each set of group.

[kcbland]

For some reason, I was thinking the Folder stage was Windoze only. I checked the 7.5 documentation and it looks like it might work. I'm pretty sure that it's one-row to one-file, unless you pack all rows for a file together. The only way to do this is to SORT by the filename column I suggested, then using stage variables in a Transformer concatenate all rows into a single row, pass to an Aggregator on filename break, use Aggregator LAST derivation on the data to insure all rows come from Transformer, then on to the Folder stage.

[parag.s.27]

If you know the exact column from where this data is comming then write a routine to read data from file...use help..

then in a sequence use routine execution stage to extact the data and pass it through a user variable stage...and now use this variable in file names

[parag.s.27]

that is

Code: Select all

USERVARIABLE stage----->Routine Activity Stage---->then for ur final job 
                                        give file path here             for file name use
                                         and the stage name            use: -


<Routine Stage Name>.$ReturnValue

every time this value should be fed to the job as a parameter

[ray.wurlod]

Etiquette Note
We don't do "urgent" here. This is an all-volunteer site; no-one gets paid for posting. We post as and when we can. Some of us are asleep when your need is urgent, if indeed it really is. If you want urgent response, sign up with your support provider and learn the true cost of "urgent".

[sb_akarmarkar]

Do you mean writing the record id as file name for each set of group.

Yes... Kenneth has given solution ... and it works fine ...
Just in concatenation put CHAR(13) and CHAR(10) for new line...


Thanks,
Anupam