I/O Wait for Datasets Sequentail File

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
jwiles
Premium Member
Premium Member
Posts: 1274
Joined: Sun Nov 14, 2004 8:50 pm
Contact:

Re: I/O Wait for Datasets Sequentail File

Post by jwiles »

When you run the "ls -l" command in the datasets directory, how many files are listed when the command returns?

Does the command hang if you run it in another directory?

High I/O waits are usually caused by one or more of:

1) Hardware contention (I/O channels, disk drives, network for NAS)
2) High number of files in a directory (thousands)
3) Too small of a block size/buffer size with high data volumes

Your description points me first towards the second option as you mention that ls -l also seems to hang. The number of files in a directory has a direct impact on how long it takes to access within the directory...the o/s must step through the directory listing in order to find the requested file information.

Also, have any new processes (DataStage or non-DataStage) been implemented on your servers? How about new source files or a sudden increase in data quantities?

Regards,
- james wiles


All generalizations are false, including this one - Mark Twain.
mandyli
Premium Member
Premium Member
Posts: 898
Joined: Wed May 26, 2004 10:45 pm
Location: Chicago

Post by mandyli »

Thanks .

I Hope this is more I/0 issue.
Post Reply