Dataset reading problem when envrionment changed.
Posted: Wed Sep 07, 2011 1:03 am
Hi,
I have designed one job with
Dataset ---> copy----------->copy
In One environment it takes 4 mins to read dataset with 20 million records, In other environment it takes 30+ minutes to finish.
I am not sure whats going wrong.
In both the environment OS is same DS version is same.
1. the DS file generated in same way
2. iostat same.
3. changed/increased buffer settings but no improvement.
4. resource availability same
5. config file same except node names.
6. row/sec different in both environments.(100000 vs 10000)
7. No schema differences.
8. Dataset sizes are same in both environments.
9. Job is multi instance but running single instance in both envs.
10. when added PM PLAYER TIMING ds operator taking long time.
In same job reading dataset taking much more time in one environment than other but other jobs also using other datasets are not showing that much difference.
Please let me know how to fix this issue.
Thanks in advacne.
I have designed one job with
Dataset ---> copy----------->copy
In One environment it takes 4 mins to read dataset with 20 million records, In other environment it takes 30+ minutes to finish.
I am not sure whats going wrong.
In both the environment OS is same DS version is same.
1. the DS file generated in same way
2. iostat same.
3. changed/increased buffer settings but no improvement.
4. resource availability same
5. config file same except node names.
6. row/sec different in both environments.(100000 vs 10000)
7. No schema differences.
8. Dataset sizes are same in both environments.
9. Job is multi instance but running single instance in both envs.
10. when added PM PLAYER TIMING ds operator taking long time.
In same job reading dataset taking much more time in one environment than other but other jobs also using other datasets are not showing that much difference.
Please let me know how to fix this issue.
Thanks in advacne.