Page 1 of 1

Merging with Link Collector

Posted: Tue Jan 17, 2006 2:37 pm
by gateleys
Hi, I need to merge the content of 3 seq files into one hashed file. I used a Collector stage to stream the data into the hashed file. The job runs without any errors and the input link to the hashed file also displays the correct # of rows. But when I try to view the data (in hashed file), it says 'Data source is empty'.

However, if the output of my Collector stage is fed to a sequential file and streamed to the hashed file via a Transformer, the View Data works fine and displays the records.

I really can't understand this behavior. Am I missing something here?

Posted: Wed Jan 18, 2006 12:19 am
by roy
Hi,
If your writing to a hash file there is no need to syncronize the streams just write to it with 3 direct links it will take parrallel writing and smile.
Bare in mind duplicate key recoeds will destructively overwrite previous entries of the same key so you'll end up with the last row written for a givven key.

IHTH,