How to avoid using flat files & connect transformers out

[vinaymanchinila]

Should it be In Process or Inter Process and how does it behave. How much should I select for buffer size.

But still I am thinking about the IPC stage not working and the reason for it.

Thanks,

[ravikumarreddy]

Hai Vinay,

I have a solution for your query. let me say how i have solved. i have taken three inputs each connected to tranformer and now i have 3 transformers and i have linked them to Link Collector. in the job properties-->Performance enable row buffering to inter process. there is no need of using IPC stage after transformer. let the size be default and have a look below how inprocess and inter process behaves

•In process. You can improve the performance of most DataStage jobs by turning in-process row buffering on and recompiling the job. This allows connected active stages to pass data via buffers rather than row by row.
•Inter process. Use this if you are running server jobs on an SMP parallel system. This enables the job to run using a separate process for each active stage, which will run simultaneously on a separate processor.

i think now u have got it



RAVI

[Sainath.Srinivasan]

But still when you merge the rows, you will have to provide a constant layout. I think you will have to make necessary changes about the 'NOT NULL' column mentioned before.

[vinaymanchinila]

Ravi,
Did you compile the job, if you drag output from 3 transformers to the input of link collector, when you compile it says"LINKCOLLECTOR does not support active stages" !

[yaminids]

I have used 3 IPC stages and then directly connected them to an Link collector and then to the target.
The Issue is all the 3 transformers run fine and when the link comes to the IPC stage it just shows running. I stopped the job and increased the IPC connection time out to 999999. Still the job does not execute!.
Any help would be appreciated.

I used the above logic and it worked for me.

-Yamini

[ravikumarreddy]

Hai Vinay,

i have run the job it works for me and just you have to enable row buffer to inter process by default it will be project defaults in the performance of job properties.


Ravi

[arin_am]

[quote="vinaymanchinila"]Hi,
I have 3 transformers generating data with same metadata. Instead of staging it into 3 flat files and merge them and load into Target, is there a way I can avoid the flat files and merge the output of the transformers . I did use the link collector but it doesnt take active inputs!

Thanks[/quote]

How about going to "Job Properties" -> "Performance"
i) uncheck Project Details
ii) check Enable Row Buffer
iii) select Inter Process

Recompile and Run.

[vinaymanchinila]

Hye I got it with the IPC stage.
Thanks,