Hello everybody,
In my job I am combining 10 files using a funnel. these 10 files are sequential files and 3 fields on these files are common, the first 3 fields. I use a "continuous funnel" and combine them then use a sort stage to sort by the fields (3rd field - 1st key, 1st field-2nd sort key, 2nd field - 3rd sort key). then I send the output to the final file.
for some reason, it is sorting data correctly for 9 files and then for the 10th file it is not putting it in the right sort order.
example:
file 1
rec_type seq account
100 1 10
100 2 10
100 3 10
100 1 20
100 2 20
100 3 20
file 2
rec_type seq account
200 1 10
200 2 10
200 3 10
200 1 20
200 2 20
200 3 20
file 3
rec_type seq account
300 1 10
300 2 10
300 3 10
300 1 20
300 2 20
300 3 20
Out put file should be like this:
100 1 10
100 2 10
100 3 10
200 1 10
200 2 10
200 3 10
300 1 10
300 2 10
300 3 10
100 1 20
100 2 20
100 3 20
200 1 20
200 2 20
200 3 20
300 1 20
300 2 20
300 3 20
Instead the file looks like this.
100 1 10
100 2 10
100 3 10
200 1 10
200 2 10
200 3 10
300 2 20
300 1 20
300 3 20
100 1 20
100 2 20
100 3 20
200 1 20
200 2 20
200 3 20
300 1 10
300 3 10
300 2 10
Can anyone please explain why? What am I doing wrong here?
Help is greatly appreciated.
Using a funnel and Sort
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 10
- Joined: Wed Apr 16, 2008 7:39 am
- Location: USA
Re: Using a funnel and Sort
Change Sort stage execution mode to sequential mode and try it.
vidya_6_2000 wrote:Hello everybody,
In my job I am combining 10 files using a funnel. these 10 files are sequential files and 3 fields on these files are common, the first 3 fields. I use a "continuous funnel" and combine them then use a sort stage to sort by the fields (3rd field - 1st key, 1st field-2nd sort key, 2nd field - 3rd sort key). then I send the output to the final file.
for some reason, it is sorting data correctly for 9 files and then for the 10th file it is not putting it in the right sort order.
example:
file 1
rec_type seq account
100 1 10
100 2 10
100 3 10
100 1 20
100 2 20
100 3 20
file 2
rec_type seq account
200 1 10
200 2 10
200 3 10
200 1 20
200 2 20
200 3 20
file 3
rec_type seq account
300 1 10
300 2 10
300 3 10
300 1 20
300 2 20
300 3 20
Out put file should be like this:
100 1 10
100 2 10
100 3 10
200 1 10
200 2 10
200 3 10
300 1 10
300 2 10
300 3 10
100 1 20
100 2 20
100 3 20
200 1 20
200 2 20
200 3 20
300 1 20
300 2 20
300 3 20
Instead the file looks like this.
100 1 10
100 2 10
100 3 10
200 1 10
200 2 10
200 3 10
300 2 20
300 1 20
300 3 20
100 1 20
100 2 20
100 3 20
200 1 20
200 2 20
200 3 20
300 1 10
300 3 10
300 2 10
Can anyone please explain why? What am I doing wrong here?
Help is greatly appreciated.