XML Input stage is running slow

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Marley777
Premium Member
Premium Member
Posts: 316
Joined: Tue Jan 27, 2004 3:26 pm

XML Input stage is running slow

Post by Marley777 »

Thanks for reading. Have a job that reads a CLOB field from a DB2 table containing xml (db2 connector). The next stage uses the clob field as input and parses the data/xml using xpath in DataSatge XML Input stage. We then insert into a DB2 table (DB2 Connector). DBA's tell me all is good on the DB2 side. Unix admins are looking at the system now, but not seeing any bottle necks. I'm thinking it's the XML Input stage. The job works as expected but is very slow. Anyone ever experience performance issues with the XML Input stage and what did you do to fix the performance problem?
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Verify the bottleneck. Replace the DB2 target with a sequential file and note the speed. If no significant change then remove the XML Input stage and put the sequential file target there. After that, you'll now which part is the actual bottleneck.
-craig

"You can never have too many knives" -- Logan Nine Fingers
FranklinE
Premium Member
Premium Member
Posts: 739
Joined: Tue Nov 25, 2008 2:19 pm
Location: Malvern, PA

Re: XML Input stage is running slow

Post by FranklinE »

Marley777 wrote:Anyone ever experience performance issues with the XML Input stage and what did you do to fix the performance problem?
Oh, boy, did I.

Situation: file level tag (one instance) containing multiple detail tags. I tried to "blend" the two (repeat data from the file instance on all the details) and once my input volume hit about 300 details, it bogged down to almost a standstill. I discovered it because my final stage is FTP, and it timed out waiting for the next row.

I'm still not sure why, but using two output links from the XML Input stage solved it. One defines only the file tag, the other only the details tag.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson

Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
Post Reply