DSXchange

Hi
my design is something like
sequential file1-->transformer-->aggregator-->funnel--->sequential file2.

My requirement is that the oreder of rows in sequential file1 should be same as that of sequential file2.

the order is coming fine till the o/p of transformer.after that the order is getting lost.

I tried keeping the execution mode "parallel" and partitioning as "same" in both aggregator and funnel stages but the order is not preserved.

I also tried keeping the execution mode "sequential" and partitioning as "auto" in both aggregator and funnel stages but the order is not preserved.
can any one provide me some pointers to get rid of this problem

This is a ridiculous requirement. You are grouping data (in the Aggregator) so the rows can only come out of that group by group.

A Funnel stage with only one input is a waste of space.

ray.wurlod wrote:This is a ridiculous requirement. You are grouping data (in the Aggregator) so the rows can only come out of that group by group.

A Funnel stage with only one input is a waste of space. ...

actually there is another input to funnel stage which comes from an 'external command' stage :D

A couple of things spring to mind.

1. Add the @INROWNUM value to each record as it passes through the transformer and then sort on this in the sequential file. I suspect you'll need to execute the stage sequentially though.

2. Is the data in the input file sorted in anyway. If so you could just re-apply the sort in the output sequential file again running sequentially, otherwise you might find the sort happens within partitions.

suneyes wrote:actually there is another input to funnel stage which comes from an 'external command' stage :D

Then your question is not the one originally posed. But my answer is unaffected. And there's another possible cause of rows being out of order - what mode of operation is being used in the Funnel stage?

shaimil wrote:A couple of things spring to mind.

1. Add the @INROWNUM value to each record as it passes through the transformer and then sort on this in the sequential file. I suspect you'll need to execute the stage sequentially though.

2. Is the data in the input file sorted in anyway. If so you could just re-apply the sort in the output sequential file again running sequentially, otherwise you might find the sort happens within partitions.

The output from transformer is coming the same way it is in the input(i checked it by giving the transformer output to a sequential file).
whatever is happening,its happening in the aggregator stage.

You are grouping data (in the Aggregator) so the rows can only come out of that group by group.