DSXchange

Hi all,

I'm having troubles with my Filter Stage. Here is the situation

Sequence:
StartLoop -----> Job ---->
^ |
| |
| V
--------------------EndLoop

Job:
Sequential File ---> Filter --->

The file contains more records than the target can handle at a time. That's why I need to split up the sequential file in different pieces.

In an early job of the sequence I make up the number of records of the sequential file and divide it by the number what the target can handle at a time (10 in this example).
I loop the job the number of times it needs and within the filter I do this:
(ROWID > ((#loopcounter#*10)-10)) and (ROWID < ((#loopcounter#*10)+1))

The #loopcounter# is StartLoop.$counter.

The error messages I get are:
- Filter: Error occurred during initializeFromArgs().
- Filter: Parse error: Expected field or literal after comparison operator,got: "("
Parse error: Expected boolean operator (and, or) got: "("
Parse error: Expected 'is' statement ('is true', 'is false') or boolean operator (and, or) got: "-10".

Would anyone mind helping me? Thanks in advance!

Peter Veenis

Welcome aboard.

I'm surprised to hear of any target that can only handle a limited number of rows.

The filter expression you use in a Filter stage is actually quite limited (check the Parallel Job Developer's Guide for details). In the meantime replace your Filter stage with a Transformer stage, which can handle far more complex expressions.

ROWID is not a native part of DataStage, I assume it's in your data or generated by the Sequential File stage.

I have seen it while interacting with SAP systems with limited resources.

Check if the split command works there in command prompt. It does work for me with MKS toolkit installed. here -l is the option to define the number of records. This you can use after the file is created. but for 11.3 you should check as they have tried to reduce the dependency on mks toolkit.

Since you are on 11.3, did you check the properties of sequential file stage which should be capable to generating multiple files?

The () characters are what is throwing the error. Something a complex as you are doing belong using a transformer or a build op stage.