Selective lookups

Archive of postings to DataStageUsers@Oliver.com. This forum intended only as a reference and cannot be posted to.

Moderators: chulett, rschirm

Locked
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Selective lookups

Post by admin »

Hello to all :)
Well ive got a question, that may bee of interest, im doing a lookup using
an oracle 8i field, but only if that field takes the value A or B or C the
lookup is made if the field takes the value F the lookup is not needed, one
possible solution is to create a constraint in a transformer stage and split
the data in two, but that is making me to duplicate the hole extraction
process from this point forward, my question is if is it possible in a
transformer stage to look at the content of the field and according to his
value proceed or not whit a lookup?

Thankss
Pedro

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Hello

In your transform, in the derivation, you can do a condition like
IF F Then F Else lookup

regards
Rui Soares


Rui Soares
NOVABASE - Data Quality - Mi
Tlm : +351 . 96 347 0840
Mail : RUI.SOARES@NOVABASE.PT

-----Original Message-----
From: pedro santos [mailto:pedro_f_santos@hotmail.com]
Sent: Segunda-feira, 5 de Novembro de 2001 15:27
To: datastage-users@oliver.com
Subject: Selective lookups


Hello to all :)
Well ive got a question, that may bee of interest, im doing a lookup using an oracle 8i field, but only if that field takes the value A or B or C the lookup is made if the field takes the value F the lookup is not needed, one possible solution is to create a constraint in a transformer stage and split the data in two, but that is making me to duplicate the hole extraction process from this point forward, my question is if is it possible in a transformer stage to look at the content of the field and according to his value proceed or not whit a lookup?

Thankss
Pedro

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

What, no response from Phil Walker? Maybe he slept in this morning.

This is one of Phils classic complaints or enhancement requests.

Ascential .. when are we going to have constraints on reference links to determine whether or not the reference link is used? This would be very useful for performance reasons.

As for your problem, Pedro. I realise there is a potential performance issue of extra lookups that you dont need, but you could:

* pass the field you are using to decide if you need the lookup to the reference link,
* use a user defined query (assuming this is also to Oracle)
* add a clause to your query that tests for A or B or C.

While this involves a potentially unnecessary reference to Oracle, Oracle should be able to handle it fairly quickly as it will have a clause that effectively tests for something like 1=2 and should be able to return no rows without even referring to any tables.

-----Original Message-----
From: pedro santos [mailto:pedro_f_santos@hotmail.com]
Sent: Tuesday, 6 November 2001 1:27 AM
To: datastage-users@oliver.com
Subject: Selective lookups

Hello to all :)
Well ive got a question, that may bee of interest, im doing a lookup using an oracle 8i field, but only if that field takes the value A or B or C the lookup is made if the field takes the value F the lookup is not needed, one possible solution is to create a constraint in a transformer stage and split the data in two, but that is making me to duplicate the hole extraction process from this point forward, my question is if is it possible in a transformer stage to look at the content of the field and according to his value proceed or not whit a lookup?

Thankss
Pedro

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

No, I have long given up hope that this feature will be added - even though in some performance tuning notes I have say that it is more performant to have one transformer stage in a stream rather then many linked togther - and you do have to duplicate logic and processing - and there is a bug in that even though you can have a reference to a reference it does not show the link between the linked columns and if you manually fill it in it come up red and.....

Sorry for the long sentence but do not get me started....

8^)

Phil

-----Original Message-----
From: David Barham [mailto:david@barham.hm]
Sent: Tuesday, November 06, 2001 11:06 AM
To: datastage-users@oliver.com
Subject: RE: Selective lookups


What, no response from Phil Walker? Maybe he slept in this morning.

This is one of Phils classic complaints or enhancement requests.

Ascential .. when are we going to have constraints on reference links to determine whether or not the reference link is used? This would be very useful for performance reasons.

As for your problem, Pedro. I realise there is a potential performance issue of extra lookups that you dont need, but you could:

* pass the field you are using to decide if you need the lookup to the reference link,
* use a user defined query (assuming this is also to Oracle)
* add a clause to your query that tests for A or B or C.

While this involves a potentially unnecessary reference to Oracle, Oracle should be able to handle it fairly quickly as it will have a clause that effectively tests for something like 1=2 and should be able to return no rows without even referring to any tables.

-----Original Message-----
From: pedro santos [mailto:pedro_f_santos@hotmail.com]
Sent: Tuesday, 6 November 2001 1:27 AM
To: datastage-users@oliver.com
Subject: Selective lookups

Hello to all :)
Well ive got a question, that may bee of interest, im doing a lookup using an oracle 8i field, but only if that field takes the value A or B or C the lookup is made if the field takes the value F the lookup is not needed, one possible solution is to create a constraint in a transformer stage and split the data in two, but that is making me to duplicate the hole extraction process from this point forward, my question is if is it possible in a transformer stage to look at the content of the field and according to his value proceed or not whit a lookup?

Thankss
Pedro

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
Locked