Page 1 of 1

can i delete files from load directory using datastage??

Posted: Sat Mar 20, 2010 3:26 am
by kaushal.kumar@igate.com
Hi,

Below is my requirement.

read files from load dir --> put it into staging area table-----> business validation---> put it into o/p files in target dir.

Once i will have data in target dir,i am goin to set flag in one of the control table with staus"S" or "W" for that particula file.
My require ment is if i send file name , file location(load dir path) as parameter ,status flag,can i delete files from load directory???

Please advice :(

Posted: Sat Mar 20, 2010 7:47 am
by chulett
Yes, and you have all kinds of options to accomplish that. Bottom line, you just need to run "DEL <filename>" from the O/S. This can be done in a custom routine, before/after job directly or with a script, in a Sequence job with the Execute Command stage... etc etc...

Posted: Sat Mar 20, 2010 10:01 pm
by kaushal.kumar@igate.com
chulett wrote:Yes, and you have all kinds of options to accomplish that. Bottom line, you just need to run "DEL <filename>" from the O/S. This can be done in a custom routine, before/after job directly or with a script, in a Sequence job with the Execute Command stage... etc etc...
can i write a custom routine for this ?
please advice i don't konw how to do this :(

Posted: Sat Mar 20, 2010 10:37 pm
by chulett
Sure but it really depends on when you need to do this. A custom routine in BASIC using DSExecute() would be fairly simple to build and could be called 'after job' but you'd have to let us know how that would work for you, timing and/or process flow wise.

How were you planning on implementing your delete solution? Standalone job to read / process your 'control table'?

Posted: Sat Mar 20, 2010 10:43 pm
by kaushal.kumar@igate.com
chulett wrote:Sure but it really depends on when you need to do this. A custom routine in BASIC using DSExecute() would be fairly simple to build and could be called 'after job' but you'd have to let us know how that would work for you, timing and/or process flow wise.

How were you planning on implementing your delete solution? Standalone job to read / process your 'control table'?
Yah sure.
first i will read files from load directory.I will remove header and trailer from record.I will put it into intermediate table. job will make any entry in one control table for that particular file with status as running.Then i will do some business validation on record level.one validation done i will have to put data into output files .Once this process will over in the control table status will change as 'S' sucessfull or 'W' warning.Once i will get status as S or W i will have to delete files from load directory.
Please advice

Posted: Sun Mar 21, 2010 7:22 am
by chulett
Can you use a Server job for this, that would be fairly simple to build. Or are you one of those shops where only PX jobs are allowed because Server jobs are some kind of evil? :?

Posted: Sun Mar 21, 2010 9:38 am
by kaushal.kumar@igate.com
chulett wrote:Can you use a Server job for this, that would be fairly simple to build. Or are you one of those shops where only PX jobs are allowed because Server jobs are some kind of evil? :?
lol.....for me server jobs are not evil :D ..i can go for server job routine also..i don't thik for delete process it would give any performance/time consuming problem :?
Please advice

Posted: Sun Mar 21, 2010 9:45 am
by chulett
What I would look at doing would be writing a custom BASIC routine that leveraged DSExecute() to delete whatever filename is passed in. Then your Server job could source from your control table, filter on your status code and select the filename into your job. Then in a transformer, call the routine and delete the file. If that succeeds, you can then do whatever it is you need to do to the control table to show you've processed that record.

Lather, rinse, repeat.

Posted: Sun Mar 21, 2010 9:53 am
by kaushal.kumar@igate.com
chulett wrote:What I would look at doing would be writing a custom BASIC routine that leveraged DSExecute() to delete whatever filename is passed in. Then your Server job could source from your control table, filter on your status code and select the filename into your job. Then in a transformer, call the routine and delete the file. If that succeeds, you can then do whatever it is you need to do to the control table to show you've processed that record.

Lather, rinse, repeat.
Hi can i use basic routine inside parallel job and there can i call server routine to perform this.
will it work :?:

Posted: Sun Mar 21, 2010 10:06 am
by chulett
Yes, but you'd have to use a Server Shared Container to be able to do that, hence the suggestion to just do the whole thing in Server.