Page 1 of 2
Dynamically Creating Files
Posted: Mon Apr 20, 2009 2:24 am
by poornima.chandramouli
Hi,
I've a requirement like of below,
I'm reading a file where i've 2 columns (Column1 & Column2).
If this file has 10 records,then i've to create 10 files where the name of files should be based on the values from the coulmn1 & the content of the file be the values from Column2.
Example:
My Input file might be like
Column1 Column2
Seq1 Test1
Seq2 Test2
Seq3 Test3
Seq4 Test4
Seq5 Test5
Seq6 Test6
Seq7 Test7
Seq8 Test8
Seq9 Test9
Seq10 Test10
My Output files should be like
FileName Content
Seq1.txt Test1
Seq2.txt Test2
Seq3.txt Test3
Seq4.txt Test4
Seq5.txt Test5
Seq6.txt Test6
Seq7.txt Test7
Seq8.txt Test8
Seq9.txt Test9
Seq10.txt Test10
I was allowed only to use sequncers & parallel jobs.
Thanks in Advance.
Posted: Mon Apr 20, 2009 2:32 am
by priyadarshikunal
This can be best done in routines.
I was allowed only to use sequncers & parallel jobs.
What you mean by this line.
Sequences & parallel jobs
or
Sequencers and parallel jobs only in sequences.
Posted: Mon Apr 20, 2009 2:50 am
by Sainath.Srinivasan
Is it an interview Q ?
Re: Dynamically Creating Files
Posted: Mon Apr 20, 2009 3:11 am
by sharantheboss
Hi,
If you are sure about the no. of rows in the source then it can be done easily otherwise its almost impossible!
Regards
Boss
poornima.chandramouli wrote:Hi,
I've a requirement like of below,
I'm reading a file where i've 2 columns (Column1 & Column2).
If this file has 10 records,then i've to create 10 files where the name of files should be based on the values from the coulmn1 & the content of the file be the values from Column2.
Example:
My Input file might be like
Column1 Column2
Seq1 Test1
Seq2 Test2
Seq3 Test3
Seq4 Test4
Seq5 Test5
Seq6 Test6
Seq7 Test7
Seq8 Test8
Seq9 Test9
Seq10 Test10
My Output files should be like
FileName Content
Seq1.txt Test1
Seq2.txt Test2
Seq3.txt Test3
Seq4.txt Test4
Seq5.txt Test5
Seq6.txt Test6
Seq7.txt Test7
Seq8.txt Test8
Seq9.txt Test9
Seq10.txt Test10
I was allowed only to use sequncers & parallel jobs.
Thanks in Advance.
Posted: Mon Apr 20, 2009 3:29 am
by priyadarshikunal
No its not impossible unless it encounters any limitation of the product or operating system or mind
, or the file definition is bad. One more thing if it doesn't worth that effort.
In this case read the file using cat command and get the output.
count the number of delimiters in this case
field marks and run the routine in loop to get one line at a time and with first values create a file and write the second value in that file.
Re: Dynamically Creating Files
Posted: Mon Apr 20, 2009 3:29 am
by poornima.chandramouli
This is not an interview question.
It is my requirement.
I was only allowed to use Parallel jobs & Sequencers to achieve this requirement.
Re: Dynamically Creating Files
Posted: Mon Apr 20, 2009 3:31 am
by priyadarshikunal
poornima.chandramouli wrote:This is not an interview question.
It is my requirement.
I was only allowed to use Parallel jobs & Sequencers to achieve this requirement.
Once again
Parallel jobs & Sequencers . Its confusing me. can you please answer my questions.
Re: Dynamically Creating Files
Posted: Mon Apr 20, 2009 3:40 am
by poornima.chandramouli
priyadarshikunal wrote:poornima.chandramouli wrote:This is not an interview question.
It is my requirement.
I was only allowed to use Parallel jobs & Sequencers to achieve this requirement.
Once again
Parallel jobs & Sequencers . Its confusing me. can you please answer my questions.
We can use only parallel jobs & Sequencer Jobs(not server jobs)
Re: Dynamically Creating Files
Posted: Mon Apr 20, 2009 3:42 am
by poornima.chandramouli
poornima.chandramouli wrote:priyadarshikunal wrote:poornima.chandramouli wrote:This is not an interview question.
It is my requirement.
I was only allowed to use Parallel jobs & Sequencers to achieve this requirement.
Once again
Parallel jobs & Sequencers . Its confusing me. can you please answer my questions.
We can use only parallel jobs & Sequencer Jobs(not server jobs)
Or
Only Parallel Jobs alone
Or
combination of both Parallel jobs & Sequencers
Posted: Mon Apr 20, 2009 4:01 am
by ShaneMuir
Use an external file stage and in the output use an awk command to append each row to a file name called whatever value is in the first column.
Posted: Mon Apr 20, 2009 4:05 am
by poornima.chandramouli
ShaneMuir wrote:Use an external file stage and in the output use an awk command to append each row to a file name called whatever value is in the first column.
Hi Shane,
Thanks for your response,Could you please explain a bit further.
Since i'm new for using External Filter stage & also how the values can be dynamically mapped to file names,because we are not sure of the number of records.
Thanks
Posted: Mon Apr 20, 2009 4:14 am
by ShaneMuir
I'll try to expand but my unix skills aren't that great.
Basically in your external file stage, pass it 2 rows, one being the file name (in your case the first column plus whatever extension you wish to give it), and then pass the rest of your columns as one column (with whatever delimiter your target file needs).
It in the destination program of the external file stage you would need something like
Code: Select all
awk '{nPosField1=index($0,",");print substr($0,nPosField1+1)>substr($0,1,nPosField1-1)}'
Set your Target method to Specific program.
Posted: Mon Apr 20, 2009 5:49 am
by nirav.parikh
In server jobs we could have done it using output as Folder Stage. wonder what's the equivalent for Parellel jobs.
Posted: Mon Apr 20, 2009 3:42 pm
by ray.wurlod
Shane means External Target, not External Filter. The command in the external target is a relatively straightforward awk command. Read the man pages on awk, or a good UNIX for beginners manual, for more information.
Posted: Wed Apr 22, 2009 2:21 am
by ShaneMuir
ray.wurlod wrote:Shane means External Target, not External Filter.
Whoops! My bad.