Improving Performance
Posted: Fri Apr 04, 2008 9:26 pm
Hi,
I need to join a DataSet (left link) with 66 millons of records with a Sequential File (right link) with 28 millons of records.
I tried reading the Sequencial File with Sequential Stage 1, 2 and 4 readers per node but the importing process to the virtual DataSet is taking a lot of time.
1) Is there any tip to improve the importing process of sequential files?
2) If I need to join / merge two big sequential files (>20M records), is it posible to join / merge them without importing them to a virtual dataset in DataStage EE?. If no, what is the best way to do this?
Thx
I need to join a DataSet (left link) with 66 millons of records with a Sequential File (right link) with 28 millons of records.
I tried reading the Sequencial File with Sequential Stage 1, 2 and 4 readers per node but the importing process to the virtual DataSet is taking a lot of time.
1) Is there any tip to improve the importing process of sequential files?
2) If I need to join / merge two big sequential files (>20M records), is it posible to join / merge them without importing them to a virtual dataset in DataStage EE?. If no, what is the best way to do this?
Thx