Data Set for Link collector links need to be the same size?
Posted: Tue Sep 09, 2008 5:22 pm
Hey All,
Hopefully an easy question. We have 2 sequential files (same structure) loading to one table. We were using link collector stage but our job has failed a few times after running for a period of time. The files have different amounts of data (1st has 50 million rows and 2nd has 70 million). Would this cause a timeout error using the link collector stage, round-robin algorithm? Is the simplest solution to break it up into 2 jobs and process each separately? Thanks in advance!
Hopefully an easy question. We have 2 sequential files (same structure) loading to one table. We were using link collector stage but our job has failed a few times after running for a period of time. The files have different amounts of data (1st has 50 million rows and 2nd has 70 million). Would this cause a timeout error using the link collector stage, round-robin algorithm? Is the simplest solution to break it up into 2 jobs and process each separately? Thanks in advance!