XML Format Issues while generating XML output

ray.wurlod · Post by **ray.wurlod** » Mon Oct 17, 2011 6:54 pm

XML is free format. Your expected and actual results are functionally identical. There are plenty of XML "prettifiers" out there; it is not the purpose of an ETL tool to do that. It is producing valid XML.

chulett · Post by **chulett** » Mon Oct 17, 2011 9:25 pm

Formatting is for peoples, not processes - turn off the formatting and just let it stream it out. When you need to look at it, let something like Internet Explorer automagically pretty it up.

If you really, really think you "need" it to look like you posted, find yourself a pretty print utility for UNIX and hook that into your job flow as a post-process.

marpadga18 · Post by **marpadga18** » Thu Oct 20, 2011 12:16 pm

I wrote a Shell Script to remove the Line Breaks in the XML output and passed that Shell Script in After Job Subroutine in job properties.

Without the shell script to Load 1 Million Records, time taken for my job to finish is 1 minute
With the shell script to Load 1 Million Records, time taken for my job to finish is 30 minutes.

I got 2 tables to load, with each table with 550 Million records (total 1.1 Billion records) then the time to take these 2 jobs to finish will be 2 x 225 Hours.

I need to have some solution from datastage itself to gain performance.
Any comments .....

eostic · Post by **eostic** » Thu Oct 20, 2011 12:27 pm

....the formal solution is to "uncheck" 'generate formatted output'.......it's just a check box. The point that Ray and Craig are making is that we should never check that box......

....uncheck it and the crlf's will just "go away"...

Ernie

marpadga18 · Post by **marpadga18** » Thu Oct 20, 2011 2:10 pm

Ernie, I tried my job unchecking Generate Formatted Output and ran the job, I got entire output in one single line.
This doesn't works for me. I need the output as shown in my question

eostic · Post by **eostic** » Thu Oct 20, 2011 3:08 pm

Follow Craig's note.

chulett · Post by **chulett** » Thu Oct 20, 2011 5:01 pm

Please make sure you understand that having everything "in one single line" is the standard and perfectly normal. No system should need it to be formatted and I would challenge your statement that it actually needs to be pretty printed like that.

However, I've already noted how to accomplish that.

eostic · Post by **eostic** » Thu Oct 20, 2011 6:37 pm

Exactly... the desired goal is to have 'one line'. No wasted bytes...less space on the wire, on disk, etc.

Ernie

marpadga18 · Post by **marpadga18** » Tue Oct 25, 2011 10:35 pm

This line break issue doesn't exist in 7.5 version, but exist in 8.5.
As of now, we wrote a shell script which will delete the line breaks for a XML file.

mail2hfz · Post by **mail2hfz** » Wed Apr 04, 2012 6:56 pm

Hi..This is regarding the XML formating issue..You mentioned that you used the shell script to remove the line breakers..can you share that script