Reading Large XML from MQ and passing it to MQ Input Stage
Posted: Fri Dec 14, 2007 1:40 pm
Hi,
I have a requirement to read a very large XML file from MQ and pass it in to XML INPUT and then to flat file.
MQ------->XML INPUT ---> Transform ------> flat file.
The file I have now is 26.5 MB in size and has 26113609 characters. I could read the same XML as input from a sequntial file (Using XML file path option of XML input) and write to flat file.
I think the issue is in defining the field size of the XML file. When I define the XML as VARCHAR/LONGVARCHAR 99999999 it give me error "APT_CombinedOperatorController,0: Fatal Error: Fatal: Length of column and Display of column cannot be both 0." If I give it as VARCHAR 999999 it does not gives me eror and terminates successfully, but will not give me result since part of the XML will be truncated and hence may not be parse properly.
1. Could any one please suggect how do I handle this kind of large XMLs from MQ?. I have read that there are restriction imposed by DataStage when the XML is larger than 100 MB.
Sincerely,
Joffi
I have a requirement to read a very large XML file from MQ and pass it in to XML INPUT and then to flat file.
MQ------->XML INPUT ---> Transform ------> flat file.
The file I have now is 26.5 MB in size and has 26113609 characters. I could read the same XML as input from a sequntial file (Using XML file path option of XML input) and write to flat file.
I think the issue is in defining the field size of the XML file. When I define the XML as VARCHAR/LONGVARCHAR 99999999 it give me error "APT_CombinedOperatorController,0: Fatal Error: Fatal: Length of column and Display of column cannot be both 0." If I give it as VARCHAR 999999 it does not gives me eror and terminates successfully, but will not give me result since part of the XML will be truncated and hence may not be parse properly.
1. Could any one please suggect how do I handle this kind of large XMLs from MQ?. I have read that there are restriction imposed by DataStage when the XML is larger than 100 MB.
Sincerely,
Joffi