Link Collector
Moderators: chulett, rschirm, roy
Link Collector
Hello:
I am doing a job on Server 7.5 from Windows. The job consists of shared containers, hash lookups, link collector/partitoner and sequential files as source and target files.
When I compile this job I get an error "Link Collector stage does not support in-process active-to-active inputs or outputs and the job name."
Have transformed the transformer into a shared container in this job and using this shared container 4 times in this job. Need to use the shared container instead of transformer.
What else can I use cause need the shared container and the collector to do the job efficiently.
Would appreciate any advice at your earliest.
Thanks in advance.
DS_MJ
I am doing a job on Server 7.5 from Windows. The job consists of shared containers, hash lookups, link collector/partitoner and sequential files as source and target files.
When I compile this job I get an error "Link Collector stage does not support in-process active-to-active inputs or outputs and the job name."
Have transformed the transformer into a shared container in this job and using this shared container 4 times in this job. Need to use the shared container instead of transformer.
What else can I use cause need the shared container and the collector to do the job efficiently.
Would appreciate any advice at your earliest.
Thanks in advance.
DS_MJ
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
Re: Link Collector
I Prefer Inter Process stages before the collection,For improved performaces.DS_MJ wrote:Hello:
I am doing a job on Server 7.5 from Windows. The job consists of shared containers, hash lookups, link collector/partitoner and sequential files as source and target files.
When I compile this job I get an error "Link Collector stage does not support in-process active-to-active inputs or outputs and the job name."
Have transformed the transformer into a shared container in this job and using this shared container 4 times in this job. Need to use the shared container instead of transformer.
What else can I use cause need the shared container and the collector to do the job efficiently.
Would appreciate any advice at your earliest.
Thanks in advance.
DS_MJ
Link Collector Error while compiling
Hello:
Thank you for your replies.
However, my link partitioner is directly connected to my source sequential file and my Link Collector is directly connected to my target Sequential files. My hash file lookup's go into the shared container and from shared container it goes into Link collector and from link collector to my target sequential files. So:
INPUT Sequence:
Source Sequential file--> Link Partitioner(Round Robin) --> Shared container's <-- Hash lookup's going into Shared container.
OUTPUT sequence:
Hash lookup's --->Shared container's --> Link collector(Round Robin) --> Target Sequential file.
This has worked for my other jobs however there was only one hash lookup going into my shared container --> link collector -->Target Sequential file.
Thanks,
DS_MJ
Thank you for your replies.
However, my link partitioner is directly connected to my source sequential file and my Link Collector is directly connected to my target Sequential files. My hash file lookup's go into the shared container and from shared container it goes into Link collector and from link collector to my target sequential files. So:
INPUT Sequence:
Source Sequential file--> Link Partitioner(Round Robin) --> Shared container's <-- Hash lookup's going into Shared container.
OUTPUT sequence:
Hash lookup's --->Shared container's --> Link collector(Round Robin) --> Target Sequential file.
This has worked for my other jobs however there was only one hash lookup going into my shared container --> link collector -->Target Sequential file.
Thanks,
DS_MJ
Place 4 IPC's at 4 Shared Container links, Then map 4 o/p links to the LCDS_MJ wrote:Hi Manteena:
Thanks for your response.
However, there are 4 Shared container's hence 4 links go into the link collector hence cannot use IPC stage between the shared container's and Link collector. IPC stage accepts one input.
Thanks
DS_MJ
SC1-IPC1-LC1 Like wise
Thanks Manteena:
The compilation completed successfully when I put SC1-->IPC1-->LC1-->Target Seq File
SC2-->IPC2-->LC2-->Target Seq File...and so on as you suggested....!
But still I do not understand , why do I have to break LC into 4 pieces with IPC stage in the middle, why wouldnt one LC work...?
Thanks,
Mamta
The compilation completed successfully when I put SC1-->IPC1-->LC1-->Target Seq File
SC2-->IPC2-->LC2-->Target Seq File...and so on as you suggested....!
But still I do not understand , why do I have to break LC into 4 pieces with IPC stage in the middle, why wouldnt one LC work...?
Thanks,
Mamta
You can try selecting Inter or in Process at job properties - performance box, so that you can eleminate these intermediate files or IPCs.DS_MJ wrote:Thanks Manteena:
The compilation completed successfully when I put SC1-->IPC1-->LC1-->Target Seq File
SC2-->IPC2-->LC2-->Target Seq File...and so on as you suggested....!
But still I do not understand , why do I have to break LC into 4 pieces with IPC stage in the middle, why wouldnt one LC work...?
Thanks,
Mamta
By selecting any of these options DS opens temperory Passive stage between the active stages.I think you have active-active-active process between your source and target
-
- Charter Member
- Posts: 199
- Joined: Tue Jan 18, 2005 2:50 am
- Location: India
Simple solution, without adding any extra stage:
Goto Job Properties -->Performance Tab --> you will see Use project defaults box ticked, uncheck that and tick "Enable Row Buffer" ---> Tick Inter process radio button. set the buffer size in accordance with your data or you can also keep it default.
By using Interprocess-it acts as a passive stage- one active stage will write the rows/records in the buffer and the second active stage will then read the records from the buffer.
Goto Job Properties -->Performance Tab --> you will see Use project defaults box ticked, uncheck that and tick "Enable Row Buffer" ---> Tick Inter process radio button. set the buffer size in accordance with your data or you can also keep it default.
By using Interprocess-it acts as a passive stage- one active stage will write the rows/records in the buffer and the second active stage will then read the records from the buffer.
Shantanu Choudhary