Page 1 of 1
Sort Stage
Posted: Wed Oct 01, 2008 6:17 pm
by greggknight
(jb_PARSE_ITEM..ss_ITEM_SORT: Arithmetic overflow) Arithmetic overflow is the error I am recieving from the job when trying to sort 219 rows
Posted: Wed Oct 01, 2008 7:15 pm
by ray.wurlod
How are you trying to sort? What kind of data are the sort keys?
Posted: Thu Oct 02, 2008 8:56 am
by greggknight
Integer -MDM_COMPANY asc,
Integer -MDM_ITEM_NUMBER asc,
Decimal 10,6 LineTransactionId dsc
It works fine with fewer transactions.
I am parsing a folder of xml files I then send them to a sort stage to group them so that I can remove duplicate LineTransactionId keeping the larger LineTransactionId and sending the remaining LineTransactionId to a duplicate bucket.
Max rows in vertual Mem = 10000
stable sort = yes, tried it as no makes no difference.
max open files = 20. was 10 changed it to 20 makes no difference
2 Ints
5 chars
and 1 decimal is the record set
there's a transformer on the input link which removes unwanted records
and sends them to the rejected link.
with this constraints.
Substrings(lk_XML_ITEM_Data_Out.BE_Item_BusinessUnit,4,3) = 999
and Substrings(lk_XML_ITEM_Data_Out.BE_Item_BusinessUnit,1,3) = Company
REJECTED
There is a transformer stage on the output link which removes the duplicates with this constraint
RowProcCompareWithPreviousValue(lk_ITEM_SORT_OUT.MDM_ITEM_NUMBER) = 0 goes to unique file
RowProcCompareWithPreviousValue(lk_ITEM_SORT_OUT.MDM_ITEM_NUMBER) = 1 goes to duplicate file.
Posted: Thu Oct 02, 2008 3:06 pm
by ray.wurlod
Ah, more information. How large are these XML files?