DSXchange

We seem t obe expreiencing intemittent data loss using the SQL bulk loader. One factor the might be contributing is that we have multiple streams of data feeding into the bulk loader stage directly. Has anyone seen this problem before. I am open to changing jobs to load multiple strams into a link collector and then a single strema to bulk but I don't want to do all of that work if this is not a problem situation.

Any comments would be appreciated.

Also, the data lost appeas to be distinct commit failures as they are in multiples of our commit interval (10,000 rows). It's 4 streams going to the bulk loader and only one stream appears to be loading all data consistantly.

Thanks.

Hi,
IMHO, the correct method for loading bulks is:
if in codependant or to same table:
1. prepare data and ctl files.
2. build a load script containing all data files and respective ctl files (if different schemes)
3. run 1 bulk load.

if to different tables or non dependant ones you can run multiple bulk loads which differ in the tables they load (and all other related stuff)

I'm not sure a bulk stage is ment to have multiple input links - writing simultaniously simply from the sheer reason it creates a sequential data file and you do know you can't do it with sequential stage as well


IHTH,