filter stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
deepthi
Participant
Posts: 56
Joined: Thu Apr 28, 2005 9:52 am

filter stage

Post by deepthi »

hi

i need to reject some rows in a file, where the column "FRESH" is null value .
i used the command: FRESH is not null in the filter stage but it did not work . please help me with this problem
thank you
deepthi
dsedi
Participant
Posts: 220
Joined: Wed Jun 02, 2004 12:38 am

Post by dsedi »

Try to use a Transformer and include your condition as a " constraint" on it.
It should work.

Edi
bcarlson
Premium Member
Premium Member
Posts: 772
Joined: Fri Oct 01, 2004 3:06 pm
Location: Minnesota

Re: filter stage

Post by bcarlson »

deepthi wrote:hi

i need to reject some rows in a file, where the column "FRESH" is null value .
i used the command: FRESH is not null in the filter stage but it did not work . please help me with this problem
thank you
deepthi
How did it not work? Did it result in an error?

Were you trying to trap the rejected records? If so, did you have a reject link? Did you make sure that the 'Output Reject Records' was set to TRUE (Stage/Properties)?

You don't necessarily need to jump to a transformer. The filter should work. If all you have to do is filter some input, then you are over complicating things by using a transformer. 99% of the time a specific stage will do it faster than a transformer - simply because that is what it was designed to do. A transformer is more dynamic and can do more than one thing, but it seems to me that simple tasks should be relegated to simple stages. I can use a 4x4 truck to drive in a road race (it can drive fast and do many other things), but the guy in the race car is going to beat me on a track - that's what it was built for.

I have nothing against transformers. We use buildops instead, but those are really just another type of transformer and all the same arguments apply. I could filter data with a buildop, but it would be simpler, more straight forward, and probably faster to use the Filter Stage. It also makes your design more self-documenting - in other words, if someone else were to look at your program they should be able to get the general idea of how it works.

Well, that's my 2 cents... HTH.

Brad.
deepthi
Participant
Posts: 56
Joined: Thu Apr 28, 2005 9:52 am

Re: filter stage

Post by deepthi »

initially, when i use filter stage, no rows are rejected which have nulls in a specified column "FRESH", all the rows are passing to the destination file.

i checked the option 'Output Reject Records' to true but it is asking for reject link, i could not include this reject link to the filter stage.
thank you

bcarlson wrote:
deepthi wrote:hi

i need to reject some rows in a file, where the column "FRESH" is null value .
i used the command: FRESH is not null in the filter stage but it did not work . please help me with this problem
thank you
deepthi
How did it not work? Did it result in an error?

Were you trying to trap the rejected records? If so, did you have a reject link? Did you make sure that the 'Output Reject Records' was set to TRUE (Stage/Properties)?

You don't necessarily need to jump to a transformer. The filter should work. If all you have to do is filter some input, then you are over complicating things by using a transformer. 99% of the time a specific stage will do it faster than a transformer - simply because that is what it was designed to do. A transformer is more dynamic and can do more than one thing, but it seems to me that simple tasks should be relegated to simple stages. I can use a 4x4 truck to drive in a road race (it can drive fast and do many other things), but the guy in the race car is going to beat me on a track - that's what it was built for.

I have nothing against transformers. We use buildops instead, but those are really just another type of transformer and all the same arguments apply. I could filter data with a buildop, but it would be simpler, more straight forward, and probably faster to use the Filter Stage. It also makes your design more self-documenting - in other words, if someone else were to look at your program they should be able to get the general idea of how it works.

Well, that's my 2 cents... HTH.

Brad.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

You need to draw a second output link, then right click on that link and choose Convert to Reject in order to make it a rejects link.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
lshort
Premium Member
Premium Member
Posts: 139
Joined: Tue Oct 29, 2002 11:40 am
Location: Toronto

Post by lshort »

Are you sure that the column contain SQL NULL and not an empty string ("")?
Lance Short
"infinite diversity in infinite combinations"
***
"The absence of evidence is not evidence of absence."
deepthi
Participant
Posts: 56
Joined: Thu Apr 28, 2005 9:52 am

Post by deepthi »

thank you very much this worked for me , the rows are getting rejected.

thank you
deepthi


ray.wurlod wrote:You need to draw a second output link, then right click on that link and choose Convert to Reject in order to make it a rejects link.
Post Reply