Complex XML generation failure at high data volumes

[inter5566]

Hello,

My client is facing an issue with running a complex XML generation job when row volumes go above about 10,000.

The job reads data from an Oracle source which contains flattened data about a composite customer (dealers/suppliers/"individual customers"). The data is passed through a transformer to create a multitude of links that break the data into a more normalized structure. The links are of 6 primary types, and are then grouped together by type into 6 separate funnel stages. One funnel is for the primary customer information header, then the others are for repeating names, locations, contact methods, dates, and dynamic attributes. The six funnels then feed into the newer XML stage to create individual xml documents for each customer.

The job runs successfully when the row volume is below about 10,000 rows. Once the data volume goes above that level, we abort with unknown errors. We have tried various settings for heap, thread, and stack sizes to no avail.

We have a PMR opened, but I wanted to check here to see if anyone has experienced a similar issue and found a resolution.

Thanks for looking,
-Steve