Quality Stage Performace Issue
Posted: Thu Nov 08, 2007 11:02 pm
Hi all,
I am using Quality stage (8.0 hawk) for formatting address data. I ran through the various stages (investigate, standardize, match frequency, unduplicate and survival)
Data is about 500000 records investigate standardize took 40 mins each
match freequency took about 45 mins and unduplicate do the same. But while i am doing the survial it is taking almost more than four hours to run only 50000 records and i killed the job. Can any one suggest me how to improve the performance as well as minimise the duration.
Nature of data (investigation)
rec1 rec2 rec3 rec4
jnm gg 1 number, street
jnm gg 2 city, state.
Standardization ( USPREP, USNAME, USADDR,USAREA)
match frequency
unduplication match (created match specification and pass) selected match, duplicate and m probability .9 and u probability .1.
standardize (qsMatchType="MP" (Matched patten) selected.
Thanks for the suggestions.
I am using Quality stage (8.0 hawk) for formatting address data. I ran through the various stages (investigate, standardize, match frequency, unduplicate and survival)
Data is about 500000 records investigate standardize took 40 mins each
match freequency took about 45 mins and unduplicate do the same. But while i am doing the survial it is taking almost more than four hours to run only 50000 records and i killed the job. Can any one suggest me how to improve the performance as well as minimise the duration.
Nature of data (investigation)
rec1 rec2 rec3 rec4
jnm gg 1 number, street
jnm gg 2 city, state.
Standardization ( USPREP, USNAME, USADDR,USAREA)
match frequency
unduplication match (created match specification and pass) selected match, duplicate and m probability .9 and u probability .1.
standardize (qsMatchType="MP" (Matched patten) selected.
Thanks for the suggestions.