link collector

prerana dixit · Post by **prerana dixit** » Mon Sep 21, 2009 3:03 am

Can link collector be used to perform union between two queries used in ODBC stage?

ArndW · Post by **ArndW** » Mon Sep 21, 2009 3:22 am

Yes, while a link collector stage performs no SQL "join" functions, it does perform, in effect, a "UNION" of n-input links into one output link regardless of values as long as the columns are identical.

prerana dixit · Post by **prerana dixit** » Mon Sep 21, 2009 5:17 am

Do we have to use link collector and link partitioner in pair or single works fine i.e to use link collector do we need to use link partitioner as well?

ArndW · Post by **ArndW** » Mon Sep 21, 2009 5:19 am

The two are independant of each other and do not need to be used as a pair.

Fission_Attackk · Post by **Fission_Attackk** » Mon Sep 21, 2009 5:46 am

ArndW wrote:The two are independant of each other and do not need to be used as a pair. ...

hi can some one tell me what is meta data.

chulett · Post by **chulett** » Mon Sep 21, 2009 6:52 am

Sure, all kinds of people can.

http://bit.ly/D7RaW

prerana dixit · Post by **prerana dixit** » Mon Sep 21, 2009 8:24 am

ArndW wrote:The two are independant of each other and do not need to be used as a pair. ...

Actually i have a job that has a source as DRS stage with source query as union between two tables.This this stg writes to a table and then to a hash file.The source query takes longer for execution and hence the job.Are there any ways(other than optimising source query and enabling row buffering on) to optimize the performance of job?Say by using IPCs or Link partitioner or link collector.HOw to use these stages?

ArndW · Post by **ArndW** » Mon Sep 21, 2009 8:43 am

Those are a lot of questions for one post. Since a server job link collector cannot be set to take input from one link, then the other you cannot optimize that way. Do you have PX available?

chulett · Post by **chulett** » Mon Sep 21, 2009 9:10 am

Your best bet is to optimize the source query and presize the hashed file (Minimum Modulus) properly.

sweety123 · Post by **sweety123** » Mon Aug 16, 2010 8:09 am

Hi,

Does all the links to a link collector stage necessarily should have the same schema??

Or is it that the schema's can be different but only the matching columns can be passed to the output link?

ray.wurlod · Post by **ray.wurlod** » Mon Aug 16, 2010 4:43 pm

Link Collector requires identical record schema on all links (inputs and output).

HariK · Post by **HariK** » Tue Aug 17, 2010 1:44 am

The Link collector Stage can be used independently if you have set the row buffer inter process property active in job properties.

from what I recall what Link collector does is an SQL 'UNION ALL' functionality not the SQL 'UNION' functionality. So you might have to design your job to remove duplicates such as using an intermediate hash file