Reduce time taken to read data/rows from sql table

dsuser7 · Post by **dsuser7** » Thu Jun 11, 2009 4:17 am

Hi,

I'm trying to read data from 1 SQL table in the DataStage parallel job and write it to a Sequential File.

However it is taking around 2hrs of time for this process with near 760,000 records data (there is no transformer stage-just an ODBC stage, copy stage and the output Sequential stage). The output file generated is of 900MB size.
There are 700 columns per row.

Please suggest how to improve performance and fine tune.

Thank you.

miwinter · Post by **miwinter** » Thu Jun 11, 2009 4:27 am

1. Refine your SQL to ensure it's as performant as it can be
2. Which RDBMS platform are you running against?
3. Do you need to be writing to a sequential output? Can a dataset be employed instead?
4. The filesystem you are writing to for output could be suffering contention with other processes.

sajidkp · Post by **sajidkp** » Thu Jun 11, 2009 7:22 am

Try to utilise parellism in the source ODBC connector optimised with your resources