Performance Improve - CFF Stage

chillisrp · Post by **chillisrp** » Thu Oct 18, 2012 11:32 am

Hello All,

Please help me to Improve the Performance writing the data using with CFF stage . I have designed my job as below.

Source DataSet--->Transformer--->Target - CFF (Complex Flat File)

This Job processing on Node 4 ,and there are 150 Fields in the Target . It is processing 4400 records per second .

Here i need your suggestions to improve the performance .

Thanks
Chilli

ray.wurlod · Post by **ray.wurlod** » Thu Oct 18, 2012 2:02 pm

How many records? Why is 4400 rows/second unacceptable? The CFF stage potentially has to do a lot of work. Increasing the number of nodes (given that the underlying server has sufficient capacity) would be one approach. Limiting other activity on the server while this job is running would be another.

vivekgadwal · Post by **vivekgadwal** » Mon Oct 22, 2012 1:50 pm

Also, you can check and see if "Multiple readers per node" is beneficial for you or not.

anu123 · Post by **anu123** » Mon Feb 25, 2013 12:24 pm

Hi Ray,

I am facing the same issue .

Can we expect better performance if i create the same format file using Sequential File stage?

Sequential File stage is better than CFF stage? Or CFF stage is the best?

Please suggest any other alternatives we have...Thanks

ray.wurlod · Post by **ray.wurlod** » Mon Feb 25, 2013 2:51 pm

vivekgadwal wrote:Also, you can check and see if "Multiple readers per node" is beneficial for you or not.

Probably not. The problem occurs when writing a file.