Page 1 of 1

Need information about the sort option in merge stage..

Posted: Tue Aug 11, 2009 4:02 am
by datisaq
In merge stage there is an option as "sort order" available, i given that as "ascending" for my input data.I selected the partition mechanism as "Hash" and i didn't enable any link sort option not even used an external sort stage since the sort option avaiable in merge stage.

But when i compiled the code and saw the generated OSH there is no where the "sort" operator is present.

Did the merge operator itself will do the sorting? Or else
We need to explicilty specify some link sort or sort stage for sorting the data?

Kindly clarify this..

Posted: Tue Aug 11, 2009 5:35 am
by chulett
I believe you are telling it the data is already sorted, not asking it to sort the data.

Posted: Tue Aug 11, 2009 6:36 am
by datisaq
No my data is not sorted i thought the option mention over there in merge stage will sort my data.

Posted: Tue Aug 11, 2009 4:36 pm
by ray.wurlod
You won't see it in the generated OSH. You will see it in the score.

Posted: Tue Aug 11, 2009 4:53 pm
by dganeshm
ray.wurlod wrote:You won't see it in the generated OSH. You will see it in the score. ...
how do we see the score

Posted: Tue Aug 11, 2009 6:18 pm
by ray.wurlod
Dump the score into the job log by including the environment variable APT_DUMP_SCORE into your job as a job parameter and setting it to "True".

Posted: Wed Aug 12, 2009 12:55 am
by datisaq
Thanks Ray for the help..
Yes it's inserting the "tsort" operator.