How to perform a hashed file lookup
Posted: Thu Jan 19, 2006 9:33 am
Apologies for such a beginners question, no DS experience:
How do I perform a hashed file lookup. Specifically how do I set up my constraints..
I've created a transformer stage with two inputs. The format of both inputs is the same, one is my hashed file for lookup (lkReferenceLookup), the other comes from a previous stage with my data (lkValidRecords)
I've got two decimal columns, both columns are keys.
when the lookup doesn't succeed I want to send it out to my oracle link lkNewRecords, and went it does to a sequential file of duplicates, lkDuplicates.
in the transformer stage, I dragged both the columns from the lkValidRecords to the lkReferenceLookup, connecting the two.
(I read something about multi-column lookup, but I couldn't get any checkbox option to appear)
Then I dragged the columns from my input src (lkValidRecords) to both of the outputs.
This was sending all records down both branches. I read a post that I need to set up some constraints to prevent
I'm not sure whether the lookup is actually taking place... is there a way that I can check this.
so I have created two stage variables DuplicateRecord, NewRecord. I wrote expressions that looked like this
(lkValidRecords.column_1 = lkReferenceLookup.column_1) and (lkValidRecords.column_2 = lkReferenceLookup.column_2)
replacing = with NE for the reverse.
I used these as constraints on the outputs
This meant that nothing was sent down either branch.
Finally I tried. lkReferenceLookup.NOTFOUND on one of my stage variables and this made the hang..
thanks
How do I perform a hashed file lookup. Specifically how do I set up my constraints..
I've created a transformer stage with two inputs. The format of both inputs is the same, one is my hashed file for lookup (lkReferenceLookup), the other comes from a previous stage with my data (lkValidRecords)
I've got two decimal columns, both columns are keys.
when the lookup doesn't succeed I want to send it out to my oracle link lkNewRecords, and went it does to a sequential file of duplicates, lkDuplicates.
in the transformer stage, I dragged both the columns from the lkValidRecords to the lkReferenceLookup, connecting the two.
(I read something about multi-column lookup, but I couldn't get any checkbox option to appear)
Then I dragged the columns from my input src (lkValidRecords) to both of the outputs.
This was sending all records down both branches. I read a post that I need to set up some constraints to prevent
I'm not sure whether the lookup is actually taking place... is there a way that I can check this.
so I have created two stage variables DuplicateRecord, NewRecord. I wrote expressions that looked like this
(lkValidRecords.column_1 = lkReferenceLookup.column_1) and (lkValidRecords.column_2 = lkReferenceLookup.column_2)
replacing = with NE for the reverse.
I used these as constraints on the outputs
This meant that nothing was sent down either branch.
Finally I tried. lkReferenceLookup.NOTFOUND on one of my stage variables and this made the hang..
thanks