Unexpected output from MATCH Stage

Infosphere's Quality Product

Moderators: chulett, rschirm

Post Reply
kripakarprasad
Participant
Posts: 15
Joined: Mon Jan 31, 2005 6:19 pm

Unexpected output from MATCH Stage

Post by kripakarprasad »

I have a match stage where i block on VariableA and match based on VarB, VarC and VarD. I give a Agreement weight of 50 for each of the matches (b,c &D). The cut off for match is 149 / clerical 149 / Dups 150.

I had my test data manufactured to pass this match and it came out as residual. When i change the cutoffs to 1/1/2, then I get the data out of the match with a weight 150.

I tweaked the cutoffs to 99/99/100 and the expected record matches as expected, of course with the weight 150.

Should I not have cutoffs more than 100 ?? (I know this doesn't sound right).. I am not sure what I am missing here..

Why would a record with agg match weight 150, not come out as a match when i give 149/149/150, but come out as a match when I give 99/99/100.
JRodriguez
Premium Member
Premium Member
Posts: 425
Joined: Sat Nov 19, 2005 9:26 am
Location: New York City
Contact:

Post by JRodriguez »

Well you could have cutoffs with values greater than 100. Agree ..If the composite value for the record is 150 and the match cut off is 149 then the record will be a match ..

How do you pick your cut off values? Basically you want to run the match passes with sample data in the Match Designer -set your cut offs to zero - inspect your data and the histogram and visually find out when the records become matches, at that point you want to set your cut off values

It will be possible that your have the output link from the match stage in reverse order?
Julio Rodriguez
ETL Developer by choice

"Sure we have lots of reasons for being rude - But no excuses
kripakarprasad
Participant
Posts: 15
Joined: Mon Jan 31, 2005 6:19 pm

Post by kripakarprasad »

Julio,

These cutoffs were defined based on the weights given for every criteria.

I am not sure why when records aren't coming out as MATCH when the consolidated weight is greater than the cutoff.

Is there anyway to turn on some kind of tracing on to determine the issue. I have about 40 other matches and all of them work fine without any issues.

Thanks,
JRodriguez
Premium Member
Premium Member
Posts: 425
Joined: Sat Nov 19, 2005 9:26 am
Location: New York City
Contact:

Post by JRodriguez »

Kripakarprasad,

I would check if:

- In the match command you are using "Reverse Matching" (This is where the agreement weight is asigned whenever the column disagree and viceversa). This property is just a check field in your match command windows

- If any of your match commands use "Vector" and the composite weight of all columns in the vector is less than the cut off value

- If any of your match commands is using weight overrides and the overrides is not working as you expect to work. Pay more attention to "Scale" this is the most tricky one


Regards
Julio Rodriguez
ETL Developer by choice

"Sure we have lots of reasons for being rude - But no excuses
JRodriguez
Premium Member
Premium Member
Posts: 425
Joined: Sat Nov 19, 2005 9:26 am
Location: New York City
Contact:

Post by JRodriguez »

I guess you still have this issue as you explained in your private message, but I rather prefer to answer here because it will help others in future search

I believe that the cause of all your issues reside in this statement: "I give a Agreement weight of 50 for each of the matches (b,c &D)"
We don't provide the agreement or disagreement weight for each match command or column, this is calculated base on the m-prob/u-prob, and the tool do the calculation for you. In some cases you want to overrides the calculated weights conditionally... but normally you rely on the ones calculated by the tool

How are you setting the agreement weight for each match command? Are you applying an Agreement Weight(WA) overrides, specifically with the Replace options? If yes then as I noted in my previous post your data will become residual ... How? The composite weight(150), the results from your match commands will be replace by the value set in the WA parameter(50) when the columns agreed, so the final value to compare will be 50 and 99 instead of the regular values 150 and 99, because you are instructing the tool to do so .... and your record become residual

Remove the agreement weight overrides, if you are doing this, and your results should be back to normal
Julio Rodriguez
ETL Developer by choice

"Sure we have lots of reasons for being rude - But no excuses
Post Reply