Link Collector Issue

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
raviyn
Participant
Posts: 57
Joined: Mon Dec 16, 2002 6:03 am

Link Collector Issue

Post by raviyn »

Is there any issues that need too be considered while using the Collector Stage in Datastage Server Jobs?????

I need to do a following thing in Server Job

Problem : If a value for a Particular Column is A.
then i need to do a lookup with a hash file. If it is not found or if the say COLUMN <> A,
then i need to do a lookup once again with another Hash file B , then Combine the Flows together and do further Processing.

For the Above Problem
I have used following Logic to Build

..........................HASH 1

SplitTrans -------> Trans1--------->Found
|...................................................|
|<-----------------NotFound.............Collector2----->FurtherTrans
Collector1.......................................|
|-------------------->Trans2------->Found
............................HASH2

Solution Used :
1) SPLIT TRANS for COLUMN =A and COLUMN <> A.
2) Then those records which failed lookup with Trans1 and Column<> a from Split trans (Used Collector1 stage to collect) and then go for a lookup with HASH2.....
3) Used Collector2 for succesful lookups of TRans1 and Trans2 Succesful lookups to again collect in Collector to send to FurtherTrans for further Transformations....

Phew!!!!!!!!! .
I guess ppl shld have understood the Logic.
So I am doing the Above stuff with Link Collectors.
Does it lead to any Performance Issues ????????

Or is there any Better way of Doing the Above Problem...

:? :? :? :? :? :?
rasi
Participant
Posts: 464
Joined: Fri Oct 25, 2002 1:33 am
Location: Australia, Sydney

Post by rasi »

You can do like the following.

---------------------Hashlook for A
|
Source -----------Constraint Col=A--------------Output_A

---------------------Hashlook for <> A
|
Source -----------Constraint Col<>A--------------Output_Others

Then join those file together using your OS command (cat or copy)

Thanks
Siva
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

These things look so much better if you wrap them in the Code tags.

Then fiddle with Preview till it looks perfect. 8)

Then submit.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
raviyn
Participant
Posts: 57
Joined: Mon Dec 16, 2002 6:03 am

Post by raviyn »

Oops Right Ray.
But I hope u got my problem.

Concerns for your soln. Rasi is that :

1) Putting 2 queries on Source side will loads the Source because of which I don't have the Liberty i.e why I have a SplitTrans (Transformer) to do Splitting.
2) Also in the Solution HASH1 and HASH2 are two different hash files. those which fail lookup with HASH1 needs to be done a lookup again with hash2 i.e why I had a collector to do that.

So Is it not a good Practice to use a Collector to first Collect then Do a lookup ?????

Or using the hash file twice put two Trans (once for a lookup colum<>A)
(second for Column = A andd HASH1 lookup Failure)
and do the lookups.

Coming back my question still remains is using Link Collectors a Problem???
Post Reply