DSXchange

Hello Everyone,
I have a scenario, In the Input folder we will have dynamic sub folders with XML Files. My questions is How can we read those xml files in the dymaic sub folders.

Ex: input/061212/ ---this has 15 files
input/0613/12/ -- this has 20 files

I want to read all these 35 files from Datastage.
and the sub-folder names will change dynamically may be for every 1hour or so..

Thanks

Do you want to read the files' content or names?

Do you have any preferred mechanism for identifying these "dynamic" directory names?

Since we're talking XML consumption here, I'm assuming the goal is to collect the various filenames so the file path can be passed to an XML Input stage where it will do the actual reading of the contents.

Yes, no, maybe so? If that's the case then off the top of my head I'd say perhaps a "find" command in an External Source stage?

I dont have any preferred mechanism. My job is to read those xml files and load its content to a database and after successfull completion of Loading, i have move those files to arcive directory.

I used the Find command to print all the XML files in the sub directories and redirect that output to a textfile and from that text file I am able to read those XML files too. BUT Any one can tell me the Move command to move those files that are present in the text file??

text files content
/IIS/data/Source/wra.xml
/IIS/data/Source2/from.xml
.
.

I want to move these files in the text file to go to another directory

There's no single command, but you can write a shell script or DataStage sequence to loop through the file a line at a time reading the line and effecting the mv command.

I wrote a small Script to do that (Moving files to Archive).
I am wondering is there any way to move the files while loading into the table??

You could, of course, have DataStage write to the new location and simply delete the original on successful completion.

How to do that! While i was writing to database how can i move the files from source to archive directory!!

Example: If my source file has 10 absolute paths:
iis/data/f1/1.xml
iis/data/f2/2.xml

while loading the 1.xml into the database how can we move the same file at the same time to another location!!

Another link in the job design, perhaps from a Copy stage, to write into the new location.

But How can we pass that path?
I tried to use External Target Stage. The Path location is stored in a column.