The order in which links are used in a datastage transformer

HSBCdev · Post by **HSBCdev** » Thu Jul 22, 2004 7:27 am

I think this is probably a basics question for you (so apologies in advance).

Could you confirm the order of processing in a transformer stage.

If I send records into a transformer with two output links, does the first record go down output link 1 and then the first record go down output link 2 (and then the same for all the subsequent records) , or do all the records go down output link 1 and only when that has finished all the records go down output link 2?

The reason I'm asking is because I have a transformer which has one output going to a shared container which will write records to a hash file and a second output which goes to another transformer which expects to be able to find the record in the hash file. Is there a possibility that I could get to the second transformer before the shared container has written the record to the hash file?

Thanks for your help

tonystark622 · Post by **tonystark622** » Thu Jul 22, 2004 8:39 am

One input row is written to all output links before the next row is read.

Tony

HSBCdev · Post by **HSBCdev** » Thu Jul 22, 2004 9:02 am

Thanks Tony.

The situation I have is ..

Transformer 1 has two output links
Output link 1 goes to a shared container were the record goes through several transformers before being written to a hash file.
Output link 2 goes to a transformer which looks up the record on the hash file.

When the record gets sent down Output link 1, does the transformer wait until all the processing downstream form Output link 1 has completed before it sends the record to Output link 2? If this is the case then I think I'm ok - otherwise is it possible that the transformer after Output link 2 could be trying to look up on the hash file before the transformer a bit downstream from Output link1 has written the record to the hash file?

tonystark622 · Post by **tonystark622** » Thu Jul 22, 2004 9:22 am

When the record gets sent down Output link 1, does the transformer wait until all the processing downstream form Output link 1 has completed before it sends the record to Output link 2? If this is the case then I think I'm ok - otherwise is it possible that the transformer after Output link 2 could be trying to look up on the hash file before the transformer a bit downstream from Output link1 has written the record to the hash file?

The transformer sends a row down each of the output links in the order that the links appear in the transformer. As far as I know, it sends data down one link, then the next, then the next, etc. Because we can specify the order in which the links are processed, I would assume that it completes sending the data down one link before it sends the data down the next output link.

Tony

chulett · Post by **chulett** » Thu Jul 22, 2004 9:39 am

And if there's concerns about exactly what is going on, step through it with the Debugger a link at a time.

HSBCdev · Post by **HSBCdev** » Thu Jul 22, 2004 9:50 am

I should be okay then.

Thanks for your help.

sumitgulati · Post by **sumitgulati** » Thu Jul 22, 2004 11:05 am

If I understood it right in Output Link 2 you are making a lookup to a hash file which is being populated in Output Link 1. Now in this kind of a scenario the transformer that makes a lookup to this hash file will always wait for the hash file to get populated completely before processing the records further.

Regards,
Sumit

HSBCdev · Post by **HSBCdev** » Fri Jul 23, 2004 1:36 am

Thanks all, you've set my mind at ease.

ray.wurlod · Post by **ray.wurlod** » Fri Jul 23, 2004 4:25 am

Make sure you set the "lock for update" property in the hashed file stage.

mleroux · Post by **mleroux** » Mon Jul 26, 2004 5:04 am

Also disable write caching for the hashed file stage that writes to the hash file.

HSBCdev · Post by **HSBCdev** » Fri Jul 30, 2004 8:48 am

Thanks all