Page 1 of 1

Reading one Dataset multiple times in parallel job

Posted: Sun Oct 24, 2010 10:33 pm
by sourabhverma
Hi,

My requirment is to replicate the logic of oracle user defined function into datastage and my database is netezza which does not support these types of user defined functions. So for that i have prepared one parallel job for achieving the same thing with the help of DS job but because of the business logic i had to use one particular dataset 7 times (As lookup) in that single job.

So i just wanted to know whether referring one dataset in this way will make any performance impact ?

Posted: Sun Oct 24, 2010 11:45 pm
by sourabhverma
I forgot to mention the number of records, the job is processing:

The input stream contains around 3-4 millions of data where as the lookup dataset contains around 0.1 million of data.

Posted: Mon Oct 25, 2010 12:50 am
by nitkuar
use copy stage between reference source and lookup stage and have 7 output links from copy stage for lookup purpose.

Posted: Tue Oct 26, 2010 3:50 am
by sourabhverma
Thanks for reminding me the COPY stage.