can link collector stage be used with out Link partioner.

[dhiraj]

Hi,

actually i want to write the out put of 3 diffenrent transformer stages to the same sequential file.For this can i use the link collector stage to collect the output from these tranformers and write it to the sequential file.The output layout of the three transformers is the same.
I tried doing this using the link collector but it gave me the following error.

"Link collector does not support in process active to active inputs or outputs."

pls clarify why is this error shown also let me know if there are any other solutions to my program.

[morneh]

Just ignore the collector stage and link directly from your Transformers to the Sequenctial File. Just make sure that the settings inside the sequential file stage are the same for all three input links. And of course, change the 'Overwrite' setting to 'Append'

[dhiraj]

Just ignore the collector stage and link directly from your Transformers to the Sequenctial File. Just make sure that the settings inside the sequential file stage are the same for all three input links. And of course, change the 'Overwrite' setting to 'Append'

I tried doing this but it gives me an runtime error saying "unable to create file".i've changed the overwrite settings to append. are there any other settings to be chenged?

[kcbland]

Just ignore the collector stage and link directly from your Transformers to the Sequenctial File. Just make sure that the settings inside the sequential file stage are the same for all three input links. And of course, change the 'Overwrite' setting to 'Append'

DO NOT TAKE THIS ADVICE. This does not work. As data spools to files in bytes, not rows, you will not interleave rows of data but bytes of data. This will cause rows to be merged.

Your solution is either write to 3 separate files and concatenate them with a command stage or after-job/transformer command, or fix your original problem. The link collector does not have to have a prior partitioner. You simply draw an output link from three other transformers to the collector, then one output link from the collector to a transformer and then to a seqential stage.

[chulett]

DO NOT TAKE THIS ADVICE. This does not work. As data spools to files in bytes, not rows, you will not interleave rows of data but bytes of data. This will cause rows to be merged.

And let's whip out the Standard Disclaimer here, just in case anyone in the audience thinks this is a DataStage limitation... it's not. This falls back on the O/S and the fundamental way in which sequential files work. Multiple sources cannot be writing to the same flat file at the same time. Doesn't matter what you are using to do the writing.

If you are lucky the rows get merged, your data gets jumbled and your output file becomes totally useless. Typically, your job will just abort with errors of the nature that the OP mentioned.

So... sounds like, from Ken's post, if you want to fix your original problem all you need to do is insert a Transfomer stage inbetween your Collector and the Sequential file stage. You'll find this to be pretty typical - DataStage generally doesn't like you to link two Passive stages directly together. A good general Rule of Thumb is to use an Active stage (like a Transform) to link the two, even if all it does is pass the data through unchanged. Helps to avoid surprises.

[ds_developer]

Back to the original question...

I am on version 6, so this may not apply. I have found that you pretty much need a flat file on both sides of a link collector or partitioner. So the answer is:
1. add a flat file after each of your 3 transformers
2. add a link from each flat file to the link collector

then your link collector can write to the one flat file you originally wanted.

Due to this requirement, I personally try to avoid the link collector and partitioners.

John