Hi guys,
i got error while running matching case. i had 3 passes.total rows 2.3 million. standardization file size is 6gb and frequency file is 2mb. system had 3 gb ram. paging size starting and ending is 2 gb. processor 3.5 ghz .
i monitor the datasets .it finished first pass and it create empty .DS file for second pass And in scratch it created 10.7gb temp files after that the program freezed it showing "NOT RESPONDING".then there is no activity in scratch folder
i done with 2 passes and i got the result... i din't get result from 3 passes.
Any body have solution for this.... did i have to change any variable setting... or is it a memory problem..any clues
thanks in advance
Matching error (not responding)
Matching error (not responding)
vairamuthu
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
How is your blocking/partitioning on that second pass? If it's running with very large blocks it could take a very long time to finish. Those temp files suggest it's building some really big lookup files.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
thanks dude
am doing match(second pass) on registration number, year and type and i choose only a single blocking columns..i think thats tthe problem.
i ll create some more column for blocking in ruleset..
Thanks for the help vmcburney
i ll create some more column for blocking in ruleset..
Thanks for the help vmcburney
vairamuthu
Re: Matching error (not responding)
Hi,vairus wrote:Hi guys,
i got error while running matching case. i had 3 passes.total rows 2.3 million. standardization file size is 6gb and frequency file is 2mb. system had 3 gb ram. paging size starting and ending is 2 gb. processor 3.5 ghz .
i monitor the datasets .it finished first pass and it create empty .DS file for second pass And in scratch it created 10.7gb temp files after that the program freezed it showing "NOT RESPONDING".then there is no activity in scratch folder
i done with 2 passes and i got the result... i din't get result from 3 passes.
Any body have solution for this.... did i have to change any variable setting... or is it a memory problem..any clues
thanks in advance
if you using DB2 then you can create partitioning on data and create blocks of data as well use some kind of indexing on thses data block, hopefully it might work bcoz it seems it performance issue.
-------------------------
https://www.ssa.co.za
https://www.ssa.co.za
Re: Matching error (not responding)
hi
i added extra blocking column .Now matching specification runs. problem is single blocking colum.
Could you please tell me where we could find the total running time of maching specification in qualitystage?
thanks
vmcburney
i added extra blocking column .Now matching specification runs. problem is single blocking colum.
Could you please tell me where we could find the total running time of maching specification in qualitystage?
thanks
vmcburney
vairamuthu