Filter stage does not work
Moderators: chulett, rschirm, roy
Filter stage does not work
I have a simple job:
Dataset--> Filter --> Dataset
The Where clause I have in the Filter stage is:
StringToDecimal(InputLink.InputCol) > 5
It gives the following error:
Filter_12: Parse error: Expected comparison, 'between', or 'like' operator, got: "("Parse error: Expected boolean operator (and, or) got: "A"
I did a search on this site for this error but it doesn't seem like there is a solution.
Dataset--> Filter --> Dataset
The Where clause I have in the Filter stage is:
StringToDecimal(InputLink.InputCol) > 5
It gives the following error:
Filter_12: Parse error: Expected comparison, 'between', or 'like' operator, got: "("Parse error: Expected boolean operator (and, or) got: "A"
I did a search on this site for this error but it doesn't seem like there is a solution.
-
- Participant
- Posts: 48
- Joined: Wed Jun 01, 2005 7:10 am
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
One task, one stage. (Transformer excepted but, if you examine the code it generates, there's a heap of Orchestrate operators that might appear there.)
The Transformer stage can, of course, do what the Filter stage does in addition to the transformation. So, if you lose the Filter stage and set up multiple outputs from the Transformer stage, your functional requirement is still met. If you're on 7.5.1A or later, you probably have a more efficient solution as well.
The Transformer stage can, of course, do what the Filter stage does in addition to the transformation. So, if you lose the Filter stage and set up multiple outputs from the Transformer stage, your functional requirement is still met. If you're on 7.5.1A or later, you probably have a more efficient solution as well.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Some other options:
1) You could throw in a modify before the filter to handle the string to decimal conversion. Then in the filter you could use 'InputCol > 5'.
2) If there are only a few possible values, you could even leave it as a string. For example, if you just needed to find between 5 and 8 and the original type is a char(3), you could use the following:
#2 is not the most glamorous way to do it, but will get the job done. Personally, if you are trying to avoid a Transformer, I think #1 might be the easiest.
Brad.
1) You could throw in a modify before the filter to handle the string to decimal conversion. Then in the filter you could use 'InputCol > 5'.
2) If there are only a few possible values, you could even leave it as a string. For example, if you just needed to find between 5 and 8 and the original type is a char(3), you could use the following:
Code: Select all
InputCol = ' 5'
or
InputCol = ' 6'
or
InputCol = ' 7'
or
InputCol = ' 8'
Brad.
Yes I would second that. We had few Filters in our application because we felt Filter fits in the role, but the IBM experts suggested otherwise. [to use a Parallel Transformer instead of a Filter, which can improve the performance and at the same time can do almost all the tasks needed]koolnitz wrote:If you are on v7.5.1a or above, I'd personally go for single Transformer stage rather than using Modify plus Filter.
Helpful article: Is DS Parallel Transformer Evil?
Vincent quotes "Over my years of using parallel jobs I haven't noticed that much of an overhead and when you use the transformer for several tasks (transform, filter and copy) it can be faster."
Vincent quotes "Over my years of using parallel jobs I haven't noticed that much of an overhead and when you use the transformer for several tasks (transform, filter and copy) it can be faster."
Nitin Jain | India
If everything seems to be going well, you have obviously overlooked something.
If everything seems to be going well, you have obviously overlooked something.
Exactly...Until the Parallel Transformer, yes the BASIC Transformer was a drag. but the Parallel Transformer does all-in-one, if that's what is needed.koolnitz wrote:Helpful article: Is DS Parallel Transformer Evil?
Vincent quotes "Over my years of using parallel jobs I haven't noticed that much of an overhead and when you use the transformer for several tasks (transform, filter and copy) it can be faster."
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
IBM made several performance-related changes to the parallel Transformer stage in version 7.5.1A. If you have this or a later version prefer the Transformer stage to Filter or Switch. Modify and Copy, however, are still faster at doing the things they do.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.