External Source Stage with Multiple XML

dspradeep · Post by **dspradeep** » Wed Nov 18, 2009 6:28 am

Hi,

Actually I am having 1000 xml in one folder and i need to process this all xml and load it into oracle db.

I have developed the job like below

RowGen -->Trans ---> XML Input ---> Oracle

in Sequencer, I am using ls -m command to take all the file with comma seperator and doing loop to load it into DB but I heared if i use External source stage then i put <code> ls *.xml </code> then datastage concate all the xml file and do the process at same time. is it TRUE?

I am suspecting because every XML have starting with some unwanted tags (ie) Namespace info as well if it concate then it may be wrong.

Please suggest me for moving to next step.

One more request from your guys,

any body having external source stage exact syntax for windows pls give me bcz i tried lot but not working.

Thanks & Regards
Pradeep

chulett · Post by **chulett** » Wed Nov 18, 2009 7:27 am

Not true. No 'concatenation' will occur but it will process all of the files one after the other without that pesky looping job sequencer.

For DOS, you want to look into dir with the /b option but you're better off sticking with UNIX commands.

dspradeep · Post by **dspradeep** » Wed Nov 18, 2009 8:26 am

ok thanks then i can go for my previous design itself.

you mentioned that <code> dir with /b </code> but i am not getting exactly. can u pls post exact syndex

thanks for your quick replay

chulett · Post by **chulett** » Wed Nov 18, 2009 8:28 am

"dir" is a DOS command similar to "ls" in UNIX and the /b means "bare format". Open a "cmd" window and type "dir /?" for the syntax. Or google.

dspradeep · Post by **dspradeep** » Wed Nov 18, 2009 8:49 am

sry for asking that silly question ...

i undertood after that ... I have used the command in ESS like below

<code> dir /folder1/file1.xml /b
</code>

but i am getting this below error
External_Source_0,0: Source subproc: dir: not found

pls help me ...

chulett · Post by **chulett** » Wed Nov 18, 2009 10:08 am

You're certain your DataStage server is on a Windows box? And you're not actually including the "<code>" tags you keep using here, are you?

eostic · Post by **eostic** » Wed Nov 18, 2009 5:49 pm

Use "ls". "pretend" it's unix.

chulett · Post by **chulett** » Wed Nov 18, 2009 6:13 pm

Right, because you've always got the MKS Toolkit to fall back on with a Windows server and was part of the reason why I said you'd be better off sticking with UNIX commands. Seems to me it is more problematical getting a fully pathed filename from a dir command which is trivial with ls.

Just surprised it doesn't seem to know what 'dir' is.

ray.wurlod · Post by **ray.wurlod** » Wed Nov 18, 2009 6:23 pm

Since DIR is an internal command, the only way it could not be found is if the operating system of the server were other than Windows.

dspradeep · Post by **dspradeep** » Wed Nov 18, 2009 11:48 pm

we are using DS7.5.1 in UNIX box and DS8.1 in Windows box both servers are using for diff project. when i try
<code> ls -t filename </code> command in DS7.5 then i can able to view the file list but the same job in DS8.1 i can't view the file.

But MKSToolkit is installed in my Windows server mechine.

chulett · Post by **chulett** » Thu Nov 19, 2009 6:30 am

You'd need to do a much better job of what "I can't view the file" means for people to help. What happens? What list is generated? Details, man, details!