Tsort insertion

[Gokul]

In our job we have source, join on 2 inputs and target.
This job ran for 1 million records, but wen we tried for 5 million, it failed giving 'tsort --unable to write to the file....'.

As a solution we inserted explicit sort stages before the input links to the join and re-executed the job. The job ran fine.

Is there any other solution for the same?
One I can think is of setting environment variable APT_NO_PART and APT_NO_TSORT variables. But will these hamper performance and will the data be same?

Thanks,
Gokul

[chulett]

You inserted explicit sort stages set how, to 'Don't sort already sorted'? If so, then you could also use the $APT_NO_TSORT variable but the end result would be identical and the sort stage is better 'documentation' IMHO.

[ray.wurlod]

You probably need to increase available memory or, if that's not possible, scratch space. The Sort stage allows you to allocate memory for that stage alone, while a global change may be wrought via the APT_TSORT_STRESS_BLOCKSIZE environment variable.