DSXchange

hi,

In my parallel job one stage (transformer) is sequential. Does it have any negative impact on performance? will it be bottleneck for the complete prallel flow?

Thanks,
Vishal

Hi

Have you developed and tested with any data ?

without knowing anything how we will say about performance.

Is this is simple and low number of rows or huge?

Thanks
Man

Yes i did but with very small data. However, my job will process around 1 million records.

Is the transformer sequential for a certain reason? Does it HAVE to be sequential in order to meet business logic requirements?

Any sequentially operating stage in a parallel job has the potential to be a bottleneck depending upon the job design and data being processed, but that doesn't mean they will be.

Regards,

I have the following logic to compare the name of two records. Where the file is sorted on name and rownumber, because there can be two groups same name but sandwitched with another group of name. THe below does not work correctly if the stage is parallel

StageVar1 = Input.name
StageVar2 = (if StageVar1 = StageVar3 then StageVar4 else StageVar4 + 1)
StageVar3 = StageVar1
StageVar4 = StageVar2

Get the partitioning right and it WILL work properly in parallel. A sequential stage in a parallel job will always be a bottleneck.