Hi,
I am new to using XML capabilities of DataStage. I would like to know whether there is any limit in the number of rows that can be read in terms of count or Size? Say for e.g. there is a 2GB limit on DataSet?
Regards,
Sai
XM Input Stage
Moderators: chulett, rschirm, roy
The 2GB limit on the DataSet is imposed by the operating system constraints and not a DataStage constraint (mostly for the data files in the resource disk). Number of rows is never a problem, and the size is also not a constraint as long as it is not constrained by the OS.
"given enough eyeballs, all bugs are shallow" - Eric S. Raymond
You are likely to hit in-memory limits in the current version of XMLPack, long before you have issues with ultimate filesize. A "single document" cannot be more than about 300 to 500M in size, depending on the number of nodes and your env....
...however, that is "single document"...you could certainly run a job that is retrieving, parsing, and reading/processing hundreds of such documents in a single run, where such documents are a collection of XML instance documents in a single subdirectory.
Ernie
...however, that is "single document"...you could certainly run a job that is retrieving, parsing, and reading/processing hundreds of such documents in a single run, where such documents are a collection of XML instance documents in a single subdirectory.
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>