jzparad wrote:Thanks Andrew.
What I'm trying to do is read data from an Oracle database (9.2) and produce one XML file which will be measured in tens of gigabytes. I know that DataStage uses the Xalan XSLT processor for reading and transforming but I don't know anything about the way it creates XML documents. Have you tried to create very large files using DataStage XML pack 2. If so, any comments would be appreciated.
Jim, you're hijacking this thread away from the original poster.
![Shocked :shock:](./images/smilies/icon_eek.gif)
But on the note of your query, you're probably not using the right approach for your volume. You're probably dealing with hundreds of millions of rows of data, so choking that thru a Server job (can't tell your OS, release, etc because it's not your thread
![Crying or Very sad :cry:](./images/smilies/icon_cry.gif)
) is probably not scalable. You're going to have to high-performance spool (ie NOT A SERVER JOB) the output and then convert to XML. For low volumes on a decent machine DS will be alright, but for high volumes you're going to need to get the volume data out of Oracle and that requires multiple output streams equivalent to what your system can handle (probably, an output stream per partition and not more than 2 streams per cpu). You're going to have to investigate scripted alternatives.
Of course if you're paid by the hour, and the customer doesn't mind waiting, get all you can.
![Cool 8)](./images/smilies/icon_cool.gif)