Page 1 of 1
XMLReader
Posted: Wed Jan 07, 2004 7:04 am
by leifabak
Hi,
I am using the XmlReader stage to load data into an oracle 9.0.2 database. This works just fine when the files are less than 110 Mb in size. When the files get larger DataStage aborts with no proper error message. Anyone with the same experience and hopefully a solution to this problem?
regards
Leif-Arne Bakker
Memory Leak?
Posted: Wed Jan 07, 2004 12:15 pm
by 1stpoint
This sounds like a Memory Leak issue. You should open a case with support and they can walk you through the debugging process. If you're on unix you might want to do a ps -aux while the job is running and watch the process memory to see if it is causing the job to crash.
Re: XMLReader
Posted: Fri Jan 09, 2004 2:09 pm
by rwierdsm
leifabak wrote:Hi,
I am using the XmlReader stage to load data into an oracle 9.0.2 database. This works just fine when the files are less than 110 Mb in size. When the files get larger DataStage aborts with no proper error message. Anyone with the same experience and hopefully a solution to this problem?
regards
Leif-Arne Bakker
The recommendation we had from Ascential support on this problem was to split the processing into two jobs. The first job reads the XML into a sequential file, the second processes the sequential file. All our XML reads are done this way now.
For some reason (memory leak as suggested above?) if the XML reader has to pause to do lookups, write to DB, etc, memory gets swapped out ( or whatever!) and the reader gets unhappy.
Regards,
Rob Wierdsma