Error with Dataset stage while using instances as target

kishore2456 · Post by **kishore2456** » Tue Nov 13, 2007 6:38 am

Hi all, i have developed a job in which i have taken lookup on dataset and i'm using the same dataset with two instances in two streams. When i tried to run the job its getting aborted by giving the following error
Here dssource/1 is my dataset

Operator initialization: The data "/dssource/1" is used as output multiple times
Operator initialization: The data "/dssource/1" may not have more than one file/ds override

Can you please tell why this came and how to resolve this

Maveric · Post by **Maveric** » Tue Nov 13, 2007 7:25 am

Don't think you can have two instances of the same DataSet. Use a copy stage instead with two output links.

ray.wurlod · Post by **ray.wurlod** » Tue Nov 13, 2007 4:28 pm

As well, you should name your Data Set control files with a ".ds" suffix.

Best practice is to create a directory in which to store Data Set and File Set control files.

kishore2456 · Post by **kishore2456** » Tue Nov 13, 2007 11:22 pm

In my job I'm implementing SCD2 in which i need to divide the flow into tow pipelines one for inserting and other for updating. So definitely i need two instances?? is there any alternate approach?

i didnt get the meaning of "Use a copy stage instead with two output links"
Though i placed a copy stage after the lookup for both i'm getting the same error??? Can u explain more

Maveric · Post by **Maveric** » Tue Nov 13, 2007 11:30 pm

Got your requirement wrong. I thought you are using same DataSet twice as source. Since you say it is used as target, the same rule applies. Multi instance of same DataSet not allowed in the same job. If the meta data for both the links is same why not use a filter stage, and a flag field to differentiate between inserts and updates?

ray.wurlod · Post by **ray.wurlod** » Wed Nov 14, 2007 5:48 am

There's no such concept as insert/update for Data Sets. Add a flag column that contains "I" or "U" in the Data Set, and use that to indicate whether the record is to be inserted and updated. Use that in the job that reads from the Data Set subsequently.