Urgent question: How to dynamic set file names parameter
Moderators: chulett, rschirm, roy
Urgent question: How to dynamic set file names parameter
Dears,
sorry to bother! I have one urgent question, hope you can give some suggestions! Thanks in advance.
From the logic, the job is very simple. It just analysis one input file, then according to the file content, output the related results to different sequential files and these sequential file name is dynamic.
Take an example, if input file contains "American", the it will put all the American data into "American.txt" file, while if contains "flower", the output file name should be "flower.txt".
Of course, we can draw all stages in one panel of DS designer, but it's very, very tedious because we will have 600 more output files. So is there one way to dynamic update file name parameter?
sorry to bother! I have one urgent question, hope you can give some suggestions! Thanks in advance.
From the logic, the job is very simple. It just analysis one input file, then according to the file content, output the related results to different sequential files and these sequential file name is dynamic.
Take an example, if input file contains "American", the it will put all the American data into "American.txt" file, while if contains "flower", the output file name should be "flower.txt".
Of course, we can draw all stages in one panel of DS designer, but it's very, very tedious because we will have 600 more output files. So is there one way to dynamic update file name parameter?
Welcome to our forum.
Your requirement is not one that DataStage is meant to solve. Of course, you could use job parameters for the file names, but because you don't know the file names ahead of time you can't use job parameters.
What I would suggest is that you add a column to the output file, say in the first position, that is a partitioning value. Then, after processing all data into single output file, you can use a simple ksh or perl script to break it into the indeterminate number of output files. That is really easy to do in ksh. If the data volume is large, go for a perl, java, or c program. If the data volume is small, ksh is probably fine.
Your requirement is not one that DataStage is meant to solve. Of course, you could use job parameters for the file names, but because you don't know the file names ahead of time you can't use job parameters.
What I would suggest is that you add a column to the output file, say in the first position, that is a partitioning value. Then, after processing all data into single output file, you can use a simple ksh or perl script to break it into the indeterminate number of output files. That is really easy to do in ksh. If the data volume is large, go for a perl, java, or c program. If the data volume is small, ksh is probably fine.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
-
- Participant
- Posts: 232
- Joined: Fri Sep 30, 2005 4:52 am
- Contact:
For some reason, I was thinking the Folder stage was Windoze only. I checked the 7.5 documentation and it looks like it might work. I'm pretty sure that it's one-row to one-file, unless you pack all rows for a file together. The only way to do this is to SORT by the filename column I suggested, then using stage variables in a Transformer concatenate all rows into a single row, pass to an Aggregator on filename break, use Aggregator LAST derivation on the data to insure all rows come from Transformer, then on to the Folder stage.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
-
- Participant
- Posts: 221
- Joined: Fri Feb 17, 2006 3:38 am
- Location: India
- Contact:
If you know the exact column from where this data is comming then write a routine to read data from file...use help..
then in a sequence use routine execution stage to extact the data and pass it through a user variable stage...and now use this variable in file names
then in a sequence use routine execution stage to extact the data and pass it through a user variable stage...and now use this variable in file names
Thanks & Regards
Parag Saundattikar
Certified for Infosphere DataStage v8.0
Parag Saundattikar
Certified for Infosphere DataStage v8.0
-
- Participant
- Posts: 221
- Joined: Fri Feb 17, 2006 3:38 am
- Location: India
- Contact:
that is
every time this value should be fed to the job as a parameter
Code: Select all
USERVARIABLE stage----->Routine Activity Stage---->then for ur final job
give file path here for file name use
and the stage name use: -
<Routine Stage Name>.$ReturnValue
Thanks & Regards
Parag Saundattikar
Certified for Infosphere DataStage v8.0
Parag Saundattikar
Certified for Infosphere DataStage v8.0
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Etiquette Note
We don't do "urgent" here. This is an all-volunteer site; no-one gets paid for posting. We post as and when we can. Some of us are asleep when your need is urgent, if indeed it really is. If you want urgent response, sign up with your support provider and learn the true cost of "urgent".
We don't do "urgent" here. This is an all-volunteer site; no-one gets paid for posting. We post as and when we can. Some of us are asleep when your need is urgent, if indeed it really is. If you want urgent response, sign up with your support provider and learn the true cost of "urgent".
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 232
- Joined: Fri Sep 30, 2005 4:52 am
- Contact: