I am having a performance issue with a job I am working running currently. The source is a fileset. the job read around 3 million records. when I ran the job it completes in 2 minutes. when I run the same job 6 instances at a time then each instance takes around 25 minutes, out of which 23 minutes is spent on I/O throughput. the job takes 23 minutes just to read the source file.
Can someone please explain me where do I look to find out why the single instance takes 2 minutes while running 6 instances, takes 23 minutes for I/O throughput.
Thanks
Mark
I/O thoughput
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 263
- Joined: Fri Sep 23, 2005 6:49 am
I/O thoughput
when I ran single instance 20,306 records are processed per second. And 1685 rows/sec were processed when 6 instances of the same job were running concurrently. the jobs were running with a 2x1 configuration file.
Thanks
Mark
Thanks
Mark
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Premium Member
- Posts: 263
- Joined: Fri Sep 23, 2005 6:49 am
I/O throughput
Thank You. Where do I find this word contention
-
- Premium Member
- Posts: 263
- Joined: Fri Sep 23, 2005 6:49 am
I/O throughput
I could not find any word resource or contention in the datastage log
You won't. You are likely facing I/O contention issues, caused by many processes trying to read the same files at the same time. You will need to work with your system administrators to look at system reports (iostat, vmstat, sar, etc.) to see where there may be a bottleneck.
Regards,
Regards,
- james wiles
All generalizations are false, including this one - Mark Twain.
All generalizations are false, including this one - Mark Twain.
-
- Participant
- Posts: 31
- Joined: Mon Jun 28, 2010 5:15 am
- Location: pune
Re: I/O throughput
you were suppose to search word 'contention' in a dictionary...Just a joke sorry if you find this comment rude
But as jwiles said., you are facing i/o contention...multiple processes reading the same file..there is going to be a resource sharing between these processes..which is causing the slow down
But as jwiles said., you are facing i/o contention...multiple processes reading the same file..there is going to be a resource sharing between these processes..which is causing the slow down
-
- Participant
- Posts: 47
- Joined: Thu Dec 09, 2010 4:37 am