this is an excerpt from the AIX sort MAN pages:
Could you explain how this does not get rid of duplicate records? If you don't believe it, try it.-u
Suppresses all but one line in each set of lines that sort equally according to the sort keys and options.
For the original poster, do a "cat -v" of the result to see if there might not be nonprintable characters in the string.
</a>