Sequential processing of Files
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
Sequential processing of Files
Hi ,
I have a requirement where i need to process the files in the job sequentially. By sequentially I mean that there will be an output file which in turn will be the input to another file.
File-----> Transformer-----> File------->Transformer------> File
I am unable to attach the third file after the transformer . The second file is giving a error like the file (3rd) cannot be attached as it is not allowed due to the current setting or lack of the reject mode property. Some settings need to be changed. Please provide some solution.
Thanx in advance.
I have a requirement where i need to process the files in the job sequentially. By sequentially I mean that there will be an output file which in turn will be the input to another file.
File-----> Transformer-----> File------->Transformer------> File
I am unable to attach the third file after the transformer . The second file is giving a error like the file (3rd) cannot be attached as it is not allowed due to the current setting or lack of the reject mode property. Some settings need to be changed. Please provide some solution.
Thanx in advance.
The second file can only support a reject mode and not an input stream to your second transformer. What exactly are you trying to achieve. From your design and explanation it is clear that you are reading from a sequential file and writing to another which inturn is writing to another. Why do you need another link from your second file?
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
The reuirement is such that the 3rd file needs to be generated from the 2nd file in the same job or otherwise we have to design another job to generate the 3rd file which will increase the number of jobs tremendously.DSguru2B wrote:The second file can only support a reject mode and not an input stream to your second transformer. What exactly are you trying to achieve. From your design and explanation it is clear that you are reading from a sequential file and writing to another which inturn is writing to another. Why do you need another link from your second file?
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
Both the transformations are doing different thing, and the second transformation stage needs to work on the file produced after the first transformation. Can you suggest some other way to achieve it. This is a great bottleneck in parallel job as the same can be done very easily in server job . Please suggest some other alternative.DSguru2B wrote:It wont let you do that. The sequential file stage supports only a reject link as the second link. What transformations are involved? Cant you do all in just one transformer stage?
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Parallel jobs do not permit "passive" stages to have both input and output link (reject link is a special case).
You need two jobs, possibly run under control of a job sequence. It's the only way. Ranga's solution does not achieve the dependency of the third file upon the output from the second.
You need two jobs, possibly run under control of a job sequence. It's the only way. Ranga's solution does not achieve the dependency of the third file upon the output from the second.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
Can I do it like this :ray.wurlod wrote:Parallel jobs do not permit "passive" stages to have both input and output link (reject link is a special case).
You need two jobs, possibly run under control of a job sequence. It's the only way. Ranga's solution does not achieve the dependency of the third file upon the output from the second.
File(1)------> Transformer--------> File(2)
File(2))------> Transformer--------> File(3)
At least it reduces the number of jobs . Please suggest whether it is feasible.
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
Thats true , the second file is reading 0 rows . Can something be done with the link ordering so that the second link is triggered after the completion of the first link. If this fails then it would increase the number of jobs tremendously.DSguru2B wrote:It wont work. Your job will either fail, or process zero records from file2 if the file is present. Both the streams will fire off simultaneously.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: