Posted: Tue Jul 16, 2002 7:53 pm
If your Unix box has multiple processors you need to get get your jobs running in parallel. Normally a single job utilizes just one processor and can leave much of the server resources idle. There are a few techniques for getting getting maximum utilization of the server:
- run independant jobs in parallel. Try to get one job running for each processor.
- break a large job into multiple instances. When using flat files this may mean outputting to multiple files and merging them on completion.
- Since you are working with a lot of sequence files on Unix have a look at the CoSort plugin for DataStage.
- The Ascential Parallel Extender product can give you dramatic performance improvements on Unix.
- run independant jobs in parallel. Try to get one job running for each processor.
- break a large job into multiple instances. When using flat files this may mean outputting to multiple files and merging them on completion.
- Since you are working with a lot of sequence files on Unix have a look at the CoSort plugin for DataStage.
- The Ascential Parallel Extender product can give you dramatic performance improvements on Unix.