DSXchange

people,

When using a sequential stage with the read method as 'File Pattern', i cannot use "Number of Readers Per Node" option as they are mutually exclusive..

can someone explain why they are mutually exculusive ??

thanks

They are mutually exclusive because DataStage will read Sequential file in sequential mode.. Row by Row.. Only one node will be used to read the entire file.

Regards
Amitava

The number of Readers Per Node has a complex start up involving counting the size of the files, dividing the file across the specified number of readers to partition your data. I'm guessing that they don't allow this option for File Patterns due to complexity it adds to the stage code. It may be an option that is much easier to apply programmatically when you have a fully defined set of files rather then a fuzzy file pattern.

You can get around the problem by retrieving your file list at the sequence job level and running the job multiple times with different file names.

As well as everything Vincent said, note that Readers Per Node is intended only for fixed-width format files. It's fully described in the Parallel Job Developer's Guide

One final note - the goal of multiple readers is to take advantage of multiple nodes - instead of one single stream of data, you have as many as there are readers. On the other hand, if you are using either filesets or the file pattern, you will already be taking advantage of multiple nodes - one per file

Now, if you could specify multiple readers to be use for each file (fixed length, of course), that would be great - but I believe that would be a discussion for another forum ( PX enhancements?)