DSXchange

Hello everybody,

In my job I am combining 10 files using a funnel. these 10 files are sequential files and 3 fields on these files are common, the first 3 fields. I use a "continuous funnel" and combine them then use a sort stage to sort by the fields (3rd field - 1st key, 1st field-2nd sort key, 2nd field - 3rd sort key). then I send the output to the final file.

for some reason, it is sorting data correctly for 9 files and then for the 10th file it is not putting it in the right sort order.
example:
file 1
rec_type seq account
100 1 10
100 2 10
100 3 10
100 1 20
100 2 20
100 3 20
file 2
rec_type seq account
200 1 10
200 2 10
200 3 10
200 1 20
200 2 20
200 3 20
file 3
rec_type seq account
300 1 10
300 2 10
300 3 10
300 1 20
300 2 20
300 3 20

Out put file should be like this:
100 1 10
100 2 10
100 3 10
200 1 10
200 2 10
200 3 10
300 1 10
300 2 10
300 3 10
100 1 20
100 2 20
100 3 20
200 1 20
200 2 20
200 3 20
300 1 20
300 2 20
300 3 20

Instead the file looks like this.
100 1 10
100 2 10
100 3 10
200 1 10
200 2 10
200 3 10
300 2 20
300 1 20
300 3 20
100 1 20
100 2 20
100 3 20
200 1 20
200 2 20
200 3 20
300 1 10
300 3 10
300 2 10

Can anyone please explain why? What am I doing wrong here?

Help is greatly appreciated.

Change Sort stage execution mode to sequential mode and try it.

vidya_6_2000 wrote:Hello everybody,

In my job I am combining 10 files using a funnel. these 10 files are sequential files and 3 fields on these files are common, the first 3 fields. I use a "continuous funnel" and combine them then use a sort stage to sort by the fields (3rd field - 1st key, 1st field-2nd sort key, 2nd field - 3rd sort key). then I send the output to the final file.

for some reason, it is sorting data correctly for 9 files and then for the 10th file it is not putting it in the right sort order.
example:
file 1
rec_type seq account
100 1 10
100 2 10
100 3 10
100 1 20
100 2 20
100 3 20
file 2
rec_type seq account
200 1 10
200 2 10
200 3 10
200 1 20
200 2 20
200 3 20
file 3
rec_type seq account
300 1 10
300 2 10
300 3 10
300 1 20
300 2 20
300 3 20

Out put file should be like this:
100 1 10
100 2 10
100 3 10
200 1 10
200 2 10
200 3 10
300 1 10
300 2 10
300 3 10
100 1 20
100 2 20
100 3 20
200 1 20
200 2 20
200 3 20
300 1 20
300 2 20
300 3 20

Instead the file looks like this.
100 1 10
100 2 10
100 3 10
200 1 10
200 2 10
200 3 10
300 2 20
300 1 20
300 3 20
100 1 20
100 2 20
100 3 20
200 1 20
200 2 20
200 3 20
300 1 10
300 3 10
300 2 10

Can anyone please explain why? What am I doing wrong here?

Help is greatly appreciated.