LOOKUP with partial string

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
BuddingDev
Premium Member
Premium Member
Posts: 43
Joined: Wed Feb 08, 2012 8:12 pm
Location: United States

LOOKUP with partial string

Post by BuddingDev »

Hi,

Is their way to do lookup with partial string match?

Like I am looking for a presence of one word in a primary link rather than exact field match and consider it to be matched.

Thanks
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Not easy. You could try a sparse lookup with custom SQL using LIKE operator.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

I can confirm sparse lookup using LIKE condition does work with the DB2 Connector stage. Oracle also supports sparse lookups. It would be nice if you could do a wildcard match in the Lookup stage no matter what stage was on the reference link. As far as I know you can do exact matches, caseless matches, or range lookups, but not exactly wildcards...
Choose a job you love, and you will never have to work a day in your life. - Confucius
kandyshandy
Participant
Posts: 597
Joined: Fri Apr 29, 2005 6:19 am
Location: Singapore

Re: LOOKUP with partial string

Post by kandyshandy »

BuddingDev wrote:Like I am looking for a presence of one word in a primary link rather than exact field match and consider it to be matched.
You can, by this method. Suppose your one word is 'dummy'. Search both input and look up key column for this word & if found, populate a new column with value 'dummy'. For other records, populate this new column with original key value. Now do a look up using this new column as key and you can get that you need.

It's friday, so take it easy ;)
Kandy
_________________
Try and Try again…You will succeed atlast!!
kandyshandy
Participant
Posts: 597
Joined: Fri Apr 29, 2005 6:19 am
Location: Singapore

Post by kandyshandy »

In fact, you can parameterize your word as well so that you don't have to edit your job when that word changes.
Kandy
_________________
Try and Try again…You will succeed atlast!!
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

What makes you think there is only "one word" involved here?
-craig

"You can never have too many knives" -- Logan Nine Fingers
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

Kandy, are you suggesting using Index() and perhaps making 2 passes through the data? Sounds like that may work to match a partial string and it would simply add extra processing time. I'll have to try it "in my free time." I may have to start my own topic on that later, because I need to do wildcard lookups in real-time web service jobs with NO database I/O (i.e. no sparse lookups).
Choose a job you love, and you will never have to work a day in your life. - Confucius
kwwilliams
Participant
Posts: 437
Joined: Fri Oct 21, 2005 10:00 pm

Post by kwwilliams »

Your best case is to write a custom operator to do this work. It may be possible to do wildcard lookups through a series of passes through the data -- but this will never be as effecient as a custom op to get the work done in real time or batch for that matter.
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

Thanks for the tip, Keith. Goal will be to reach millisecond response time! Will have to test that!!
Choose a job you love, and you will never have to work a day in your life. - Confucius
BuddingDev
Premium Member
Premium Member
Posts: 43
Joined: Wed Feb 08, 2012 8:12 pm
Location: United States

Post by BuddingDev »

Thank you all for your understandable or not quite understandable responses. :D

I appreciate your inputs but this issue's requirement has changed so I am good for now.
Post Reply