runlocally()

bakul · Post by **bakul** » Sun Oct 16, 2005 11:32 pm

I am using the 2 XML Output stages, 1 Merge stage and a final XML Output stage to generate an XML. The input file is around 2GB.
The job completes succesfully. However the entire XML is not generated. The job log shows the error,

XML_Output_14,0: Caught unknown exception from runLocally()
XML_Output_14,0: runLocally() did not reach EOF on its input dataset 0

Does anyone know the cause of this error? Can anything be done to avoid this ?

Regards,
Bakul

kumar_s · Post by **kumar_s** » Mon Oct 17, 2005 7:36 am

Hi,
Just give a try. Make it to read n-1 rows and write it to the output.
if its working fine then there would be some issue with the nth record.

regards
kumar

ray.wurlod · Post by **ray.wurlod** » Mon Oct 17, 2005 3:57 pm

It may be - and I'm guessing here - that the XML stage you've used is limited to 2GB and the file is actually slightly larger. The error message indicates that the process failed to reach end-of-file. If, for example, the XML stage had been written with a signed 32-bit offset pointer, it could not read past 2GB.

Ask your support provider or post the question on ADN

bakul · Post by **bakul** » Tue Oct 18, 2005 4:31 am

Thanks for your inputs!

I have posted the query on ADN.

Regards,
Bakul

chulett · Post by **chulett** » Tue Oct 18, 2005 6:54 am

ray.wurlod wrote:It may be - and I'm guessing here - that the XML stage you've used is limited to 2GB and the file is actually slightly larger.

Don't believe that that is the case as I'm using the XML Output stage to generate ~300MB XML files.

ray.wurlod · Post by **ray.wurlod** » Tue Oct 18, 2005 4:06 pm

2GB is much bigger than 300MB. 2GB is special, as it's the largest address that can be accessed using a signed 32-bit pointer.

chulett · Post by **chulett** » Tue Oct 18, 2005 4:20 pm

Duh... sorry... I thought we were dealing with a little file, for some reason I read it as 2MB not 2GB.