Matching error (not responding)

Infosphere's Quality Product

Moderators: chulett, rschirm

Post Reply
vairus
Participant
Posts: 52
Joined: Thu Feb 07, 2008 8:02 am
Location: Johannesburg

Matching error (not responding)

Post by vairus »

Hi guys,
i got error while running matching case. i had 3 passes.total rows 2.3 million. standardization file size is 6gb and frequency file is 2mb. system had 3 gb ram. paging size starting and ending is 2 gb. processor 3.5 ghz .

i monitor the datasets .it finished first pass and it create empty .DS file for second pass And in scratch it created 10.7gb temp files after that the program freezed it showing "NOT RESPONDING".then there is no activity in scratch folder

i done with 2 passes and i got the result... i din't get result from 3 passes.

Any body have solution for this.... did i have to change any variable setting... or is it a memory problem..any clues

thanks in advance
vairamuthu
vmcburney
Participant
Posts: 3593
Joined: Thu Jan 23, 2003 5:25 pm
Location: Australia, Melbourne
Contact:

Post by vmcburney »

How is your blocking/partitioning on that second pass? If it's running with very large blocks it could take a very long time to finish. Those temp files suggest it's building some really big lookup files.
vairus
Participant
Posts: 52
Joined: Thu Feb 07, 2008 8:02 am
Location: Johannesburg

thanks dude

Post by vairus »

am doing match(second pass) on registration number, year and type and i choose only a single blocking columns..i think thats tthe problem.

i ll create some more column for blocking in ruleset..

Thanks for the help vmcburney
vairamuthu
BIuser
Premium Member
Premium Member
Posts: 238
Joined: Thu Feb 02, 2006 4:03 am
Location: South Africa

Re: Matching error (not responding)

Post by BIuser »

vairus wrote:Hi guys,
i got error while running matching case. i had 3 passes.total rows 2.3 million. standardization file size is 6gb and frequency file is 2mb. system had 3 gb ram. paging size starting and ending is 2 gb. processor 3.5 ghz .

i monitor the datasets .it finished first pass and it create empty .DS file for second pass And in scratch it created 10.7gb temp files after that the program freezed it showing "NOT RESPONDING".then there is no activity in scratch folder

i done with 2 passes and i got the result... i din't get result from 3 passes.

Any body have solution for this.... did i have to change any variable setting... or is it a memory problem..any clues

thanks in advance
Hi,
if you using DB2 then you can create partitioning on data and create blocks of data as well use some kind of indexing on thses data block, hopefully it might work bcoz it seems it performance issue.
-------------------------
https://www.ssa.co.za
vairus
Participant
Posts: 52
Joined: Thu Feb 07, 2008 8:02 am
Location: Johannesburg

Re: Matching error (not responding)

Post by vairus »

hi

i added extra blocking column .Now matching specification runs. problem is single blocking colum.

Could you please tell me where we could find the total running time of maching specification in qualitystage?

thanks
vmcburney
vairamuthu
Post Reply