Loading XML from folder stage

bkumar103 · Post by **bkumar103** » Wed May 21, 2008 7:03 am

Hi all,

In my job I loaded all the XMLs into the local directory using a FOLDER stage. Then in my next job I am using a FOLDER stage to pull the XMLs .
my second job is as follows ,
FOLDER--> TRANSFORMER-->XMLINPUT-->(..)
But I don't know (may be because of large size of the XML or ___) why we got an error as below,
"NormalPWxmljob..XMLReader: Xalan fatal error (publicId: , systemId: , line: 1, column: 1): An exception occurred! Type:TranscodingException, Message:An invalid multi-byte source text sequence was encountered"

Iam using the XML files of size 16085 bytes each .
What should be done to overcome this error ?
Is there any other appraoch to do this job ?

Thanks in advance

eostic · Post by **eostic** » Wed May 21, 2008 8:39 am

...are you really using DataStage 6? That may result in a host of others issues, although it's probably ok if you are truly using XMLInput and not the much older XMLReader....

Anyway...I have my doubts that size is the issue...that's not "that" big an xml document. I usually have no problems up to around 500M.

Is this an NLS environment? is there anything interesting about the file, regarding the multi-byte message you are receiving? What is the UTF value in the header?

Ernie

throbinson · Post by **throbinson** » Wed May 21, 2008 8:47 am

Why do you have a Transformer stage between the Folder stage and the XMLInput Stage? I would think it should be Folder Stage and then XMLInput.

chulett · Post by **chulett** » Wed May 21, 2008 8:52 am

Not sure why that would be a problem. Besides, if you don't add it, DataStage will.

bkumar103 · Post by **bkumar103** » Thu May 22, 2008 2:44 am

Yes, I am using Datastage version 6 . Sorry I have wrongly mentioned the size . It is actually 212609 bytes. We are not having any header.This is the sample of our file,
<?xml version="1.0"?>

<XwilWorkfile xmlns="http://www.xxxis.com......................
.............................................
>

Previously we are extracting XMLs from MQ directly and did the parsing , the job ran fine. But when we used the folder stage to extract XMLs , we are getting this warning.

eostic · Post by **eostic** » Thu May 22, 2008 5:35 am

ok...first...help me out....if you are using release 6, is it with the XMLInput Stage? It was optionally installable on top of release 6, as I recall, but the XMLReader and XMLWriter were in the box also......

That being said, the Folder Stage should end up treating the content "as if" it was coming from a message queue --- however, if my memory serves me correctly, I recall asking engineering to make a change to the Folder Stage defaults because of some issues I had reading in XML documents that were "pretty" formatted. There is a check box in the Folder Stage called "Preserve CRLF".....I can't remember which way we switched it, but take a look what you have, and try checking it or unchecking it (whatever you don't have now) and see what impact it has.....

Ernie

bkumar103 · Post by **bkumar103** » Fri May 23, 2008 4:07 am

Hi

Thank you very much . I used 'YES' and it worked .

throbinson · Post by **throbinson** » Fri May 23, 2008 12:29 pm

Chulett

Not sure why that would be a problem. Besides, if you don't add it, DataStage will.

I don't think that is the case. I think the XMLInput, an Active stage, will not put an "IDENT" Transformer between it and the Folder stage.

Are you sure Datastage will add it?

chulett · Post by **chulett** » Fri May 23, 2008 12:38 pm

Hmmm... never really thought of XMLInput as an active stage but if it is, you may be correct. Easy enough to check. Regardless, my main point was that adding one isn't any kind of a problem, we do that routinely here from Folder to XMLInput.