Sort Stage
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 120
- Joined: Thu Oct 28, 2004 4:24 pm
Sort Stage
(jb_PARSE_ITEM..ss_ITEM_SORT: Arithmetic overflow) Arithmetic overflow is the error I am recieving from the job when trying to sort 219 rows
"Don't let the bull between you and the fence"
Thanks
Gregg J Knight
"Never Never Never Quit"
Winston Churchill
Thanks
Gregg J Knight
"Never Never Never Quit"
Winston Churchill
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Premium Member
- Posts: 120
- Joined: Thu Oct 28, 2004 4:24 pm
Integer -MDM_COMPANY asc,
Integer -MDM_ITEM_NUMBER asc,
Decimal 10,6 LineTransactionId dsc
It works fine with fewer transactions.
I am parsing a folder of xml files I then send them to a sort stage to group them so that I can remove duplicate LineTransactionId keeping the larger LineTransactionId and sending the remaining LineTransactionId to a duplicate bucket.
Max rows in vertual Mem = 10000
stable sort = yes, tried it as no makes no difference.
max open files = 20. was 10 changed it to 20 makes no difference
2 Ints
5 chars
and 1 decimal is the record set
there's a transformer on the input link which removes unwanted records
and sends them to the rejected link.
with this constraints.
Substrings(lk_XML_ITEM_Data_Out.BE_Item_BusinessUnit,4,3) = 999
and Substrings(lk_XML_ITEM_Data_Out.BE_Item_BusinessUnit,1,3) = Company
REJECTED
There is a transformer stage on the output link which removes the duplicates with this constraint
RowProcCompareWithPreviousValue(lk_ITEM_SORT_OUT.MDM_ITEM_NUMBER) = 0 goes to unique file
RowProcCompareWithPreviousValue(lk_ITEM_SORT_OUT.MDM_ITEM_NUMBER) = 1 goes to duplicate file.
Integer -MDM_ITEM_NUMBER asc,
Decimal 10,6 LineTransactionId dsc
It works fine with fewer transactions.
I am parsing a folder of xml files I then send them to a sort stage to group them so that I can remove duplicate LineTransactionId keeping the larger LineTransactionId and sending the remaining LineTransactionId to a duplicate bucket.
Max rows in vertual Mem = 10000
stable sort = yes, tried it as no makes no difference.
max open files = 20. was 10 changed it to 20 makes no difference
2 Ints
5 chars
and 1 decimal is the record set
there's a transformer on the input link which removes unwanted records
and sends them to the rejected link.
with this constraints.
Substrings(lk_XML_ITEM_Data_Out.BE_Item_BusinessUnit,4,3) = 999
and Substrings(lk_XML_ITEM_Data_Out.BE_Item_BusinessUnit,1,3) = Company
REJECTED
There is a transformer stage on the output link which removes the duplicates with this constraint
RowProcCompareWithPreviousValue(lk_ITEM_SORT_OUT.MDM_ITEM_NUMBER) = 0 goes to unique file
RowProcCompareWithPreviousValue(lk_ITEM_SORT_OUT.MDM_ITEM_NUMBER) = 1 goes to duplicate file.
"Don't let the bull between you and the fence"
Thanks
Gregg J Knight
"Never Never Never Quit"
Winston Churchill
Thanks
Gregg J Knight
"Never Never Never Quit"
Winston Churchill
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: