Filter Stage is Slow

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
opdas
Participant
Posts: 115
Joined: Wed Feb 01, 2006 7:25 am

Filter Stage is Slow

Post by opdas »

Hi,
I found that using a filter stage slows the process when compared to the same condition being used in transformer stage.
We are more flexible to use condition in constarint in transformer stage than in Filter.
Then where should we use filter stage???
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

What filter condition you use?
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
opdas
Participant
Posts: 115
Joined: Wed Feb 01, 2006 7:25 am

Post by opdas »

A character comparision !
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

This might be not true for all cases.
May I know, how you compare the performance? The load consistency of the server?
But still there may be some case where transformer might have eaten up large resource, but have performed well.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
opdas
Participant
Posts: 115
Joined: Wed Feb 01, 2006 7:25 am

Post by opdas »

My comparing criteria was only rows/secs from transformer and filter stage.
Post Reply