Capturing non-matched records from reference file

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
vskr72
Premium Member
Premium Member
Posts: 128
Joined: Wed Apr 28, 2004 9:36 pm

Capturing non-matched records from reference file

Post by vskr72 »

I need to capture non-matched recrods from the reference file. Its a Hashed file. I tried using the constraint RefLink.NOTFOUND, I see all blanks in the file. Any option other than swapping the Ref to Stream and Stream to Ref.

Kumar
I_Server_Whale
Premium Member
Premium Member
Posts: 1255
Joined: Wed Feb 02, 2005 11:54 am
Location: United States of America

Re: Capturing non-matched records from reference file

Post by I_Server_Whale »

vskr72 wrote: I see all blanks in the file.
Kumar
What exactly do you mean by blanks?

spaces, empty file, etc, etc...?

What columns are you passing through the link? Do they include any keys?

The more info you provide, the better is the chance of getting a good answer.

Thanks,
Whale.
Anything that won't sell, I don't want to invent. Its sale is proof of utility, and utility is success.
Author: Thomas A. Edison 1847-1931, American Inventor, Entrepreneur, Founder of GE
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

You must be derive the rejected columns from the source and not the hashed file. I bet you are deriving it from the hashed file. The lookup failed and hence the values will be NULL. Makes sense?
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
vskr72
Premium Member
Premium Member
Posts: 128
Joined: Wed Apr 28, 2004 9:36 pm

Post by vskr72 »

Thanks. I guess it makes sense.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

It's impossible to capture the records that were not lookup up when performing a lookup, unless you immediately delete the successes from the reference table (hashed file) and inspect the survivors afterwards.

In SQL you might use a NOT IN comparison to a subquery that included all your matches. Doing such SQL against text files (for example via ODBC driver for text files) would be horribly slow.

But a lookup - especially a hashed file lookup - only allows an "=" condition on the key value. So it's out of the question that you could determine which keys were not looked for with this architecture.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
aakashahuja
Premium Member
Premium Member
Posts: 210
Joined: Wed Feb 16, 2005 7:17 am

Post by aakashahuja »

Assuming that you want to filter out the rows from the hash file whose corresponding rows couldnt be found in the stream link then it could be done the other was round.

That is to interchange the positions of your source and lookup stages. That is take hash file as the source and file (assuming this was your source) as lookup (if its PX else move the data in to a hash file first).

Then you can easily pass out the records which do not match....
Post Reply