Lookup Stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
logic
Participant
Posts: 115
Joined: Thu Feb 24, 2005 10:48 am

Lookup Stage

Post by logic »

Hi All,
In a process flow we were using a server job for doing a lookup to a hash file to lookup the value of a field and in the output we were writing the derivation for a column as:
If Field1 is 0 thenlookup.field1+@outrownum else mainstream.field1 .
The output was going to oracle.
The main stream file input was a file created in a previous parallel job(say job "X") in the flow.
Since the size of file created is huge we decided to modify the flow.
This is what we are tryng to do.
Instead of writing the output of job x to a file we are trying to pass it to a lookup stage and the output of lookup stage will go to oracle......the key factor however is to define the derivation of the output column which we are unable to do in the output stage. Is there any way to do it in lookup stage without passing the output to a transform?
What are we missing :? ?
Thanks.
Ash.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

If you're on 7.5 you can derive output columns in a Lookup stage. However I don't think it's quite flexible enough to do what you require. I think you'll need to follow it with a Transformer or Switch stage. A Transformer stage offers the advantage of having stage variables available.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
logic
Participant
Posts: 115
Joined: Thu Feb 24, 2005 10:48 am

Post by logic »

Thanks Ray,
The version is 7.5 .still unable to edit derivation in the output of the lookupstage. :?: .anyway I was also rethinking that approach..will try with the transformer and switch as suggested by you....
Ash.
vmcburney
Participant
Posts: 3593
Joined: Thu Jan 23, 2003 5:25 pm
Location: Australia, Melbourne
Contact:

Post by vmcburney »

Remeber that @outrownum works different in parallel jobs, when the transformer is partitioned each instance will start at 1 giving you duplicate counter values. When I need a unique counter in a transformer I use a Stage Variable, set it to a starting value of @PARTITIONNUM-@NUMPARTITIONS+1 and increment it by @NUMPARTITIONS for each row. This makes it unique across partitions.
logic
Participant
Posts: 115
Joined: Thu Feb 24, 2005 10:48 am

Post by logic »

Thanks Vincent..
Post Reply