Sort Stage

[greggknight]

(jb_PARSE_ITEM..ss_ITEM_SORT: Arithmetic overflow) Arithmetic overflow is the error I am recieving from the job when trying to sort 219 rows

[ray.wurlod]

How are you trying to sort? What kind of data are the sort keys?

[greggknight]

Integer -MDM_COMPANY asc,
Integer -MDM_ITEM_NUMBER asc,
Decimal 10,6 LineTransactionId dsc

It works fine with fewer transactions.

I am parsing a folder of xml files I then send them to a sort stage to group them so that I can remove duplicate LineTransactionId keeping the larger LineTransactionId and sending the remaining LineTransactionId to a duplicate bucket.

Max rows in vertual Mem = 10000
stable sort = yes, tried it as no makes no difference.
max open files = 20. was 10 changed it to 20 makes no difference
2 Ints
5 chars
and 1 decimal is the record set


there's a transformer on the input link which removes unwanted records
and sends them to the rejected link.
with this constraints.
Substrings(lk_XML_ITEM_Data_Out.BE_Item_BusinessUnit,4,3) = 999
and Substrings(lk_XML_ITEM_Data_Out.BE_Item_BusinessUnit,1,3) = Company
REJECTED

There is a transformer stage on the output link which removes the duplicates with this constraint
RowProcCompareWithPreviousValue(lk_ITEM_SORT_OUT.MDM_ITEM_NUMBER) = 0 goes to unique file
RowProcCompareWithPreviousValue(lk_ITEM_SORT_OUT.MDM_ITEM_NUMBER) = 1 goes to duplicate file.

[ray.wurlod]

Ah, more information. How large are these XML files?