Hi all--
Can we Run the Quality Stage job which uses the Match stage to be run in Parallel mode. Would there be any abnormality in the operation and in Match results as i am afraid the partitioning obstructs data from getting matched.
Thanks.
Can QS MATCH be Run in Parallel Mode
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
You can use DataStage to provide both the primary input and the reference input into a QualityStage plugin that is doing matching. The safest design would be to sort and partition the primary input and set the partitioning of the reference data as entire. I doubt there is any partitioning method other then entire that suits the matching algorithm.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Thanks for the Reply
thanks for the reply Mr.Vmcburney. But its my fault that i did not provide all the info thats required. I am running the QS Jobs using from the Unix command prompt from the DS Batch job. I am using theJobname.par file for running. so in that case, where i am not using the QS for DS plug-in and so i cannot provide any partitioning method. In this case can i run the QS Job in the Parallel mode or do i need to run the Job in File Mode only?
Thanks once again.
Thanks once again.
g.kiran
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Get in touch with your support provider. There is a recent patch that addresses being able to run STAN in parallel. Nothing I've heard of for MATCH, but some kinds of match wouldn't have a problem - you might just have to be careful with partitioning so that you could be sure that all potential matches occur on the same processing node.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.