can i delete files from load directory using datastage??

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
kaushal.kumar@igate.com
Participant
Posts: 77
Joined: Tue Apr 14, 2009 4:03 am

can i delete files from load directory using datastage??

Post by kaushal.kumar@igate.com »

Hi,

Below is my requirement.

read files from load dir --> put it into staging area table-----> business validation---> put it into o/p files in target dir.

Once i will have data in target dir,i am goin to set flag in one of the control table with staus"S" or "W" for that particula file.
My require ment is if i send file name , file location(load dir path) as parameter ,status flag,can i delete files from load directory???

Please advice :(
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Yes, and you have all kinds of options to accomplish that. Bottom line, you just need to run "DEL <filename>" from the O/S. This can be done in a custom routine, before/after job directly or with a script, in a Sequence job with the Execute Command stage... etc etc...
-craig

"You can never have too many knives" -- Logan Nine Fingers
kaushal.kumar@igate.com
Participant
Posts: 77
Joined: Tue Apr 14, 2009 4:03 am

Post by kaushal.kumar@igate.com »

chulett wrote:Yes, and you have all kinds of options to accomplish that. Bottom line, you just need to run "DEL <filename>" from the O/S. This can be done in a custom routine, before/after job directly or with a script, in a Sequence job with the Execute Command stage... etc etc...
can i write a custom routine for this ?
please advice i don't konw how to do this :(
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Sure but it really depends on when you need to do this. A custom routine in BASIC using DSExecute() would be fairly simple to build and could be called 'after job' but you'd have to let us know how that would work for you, timing and/or process flow wise.

How were you planning on implementing your delete solution? Standalone job to read / process your 'control table'?
-craig

"You can never have too many knives" -- Logan Nine Fingers
kaushal.kumar@igate.com
Participant
Posts: 77
Joined: Tue Apr 14, 2009 4:03 am

Post by kaushal.kumar@igate.com »

chulett wrote:Sure but it really depends on when you need to do this. A custom routine in BASIC using DSExecute() would be fairly simple to build and could be called 'after job' but you'd have to let us know how that would work for you, timing and/or process flow wise.

How were you planning on implementing your delete solution? Standalone job to read / process your 'control table'?
Yah sure.
first i will read files from load directory.I will remove header and trailer from record.I will put it into intermediate table. job will make any entry in one control table for that particular file with status as running.Then i will do some business validation on record level.one validation done i will have to put data into output files .Once this process will over in the control table status will change as 'S' sucessfull or 'W' warning.Once i will get status as S or W i will have to delete files from load directory.
Please advice
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Can you use a Server job for this, that would be fairly simple to build. Or are you one of those shops where only PX jobs are allowed because Server jobs are some kind of evil? :?
-craig

"You can never have too many knives" -- Logan Nine Fingers
kaushal.kumar@igate.com
Participant
Posts: 77
Joined: Tue Apr 14, 2009 4:03 am

Post by kaushal.kumar@igate.com »

chulett wrote:Can you use a Server job for this, that would be fairly simple to build. Or are you one of those shops where only PX jobs are allowed because Server jobs are some kind of evil? :?
lol.....for me server jobs are not evil :D ..i can go for server job routine also..i don't thik for delete process it would give any performance/time consuming problem :?
Please advice
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

What I would look at doing would be writing a custom BASIC routine that leveraged DSExecute() to delete whatever filename is passed in. Then your Server job could source from your control table, filter on your status code and select the filename into your job. Then in a transformer, call the routine and delete the file. If that succeeds, you can then do whatever it is you need to do to the control table to show you've processed that record.

Lather, rinse, repeat.
-craig

"You can never have too many knives" -- Logan Nine Fingers
kaushal.kumar@igate.com
Participant
Posts: 77
Joined: Tue Apr 14, 2009 4:03 am

Post by kaushal.kumar@igate.com »

chulett wrote:What I would look at doing would be writing a custom BASIC routine that leveraged DSExecute() to delete whatever filename is passed in. Then your Server job could source from your control table, filter on your status code and select the filename into your job. Then in a transformer, call the routine and delete the file. If that succeeds, you can then do whatever it is you need to do to the control table to show you've processed that record.

Lather, rinse, repeat.
Hi can i use basic routine inside parallel job and there can i call server routine to perform this.
will it work :?:
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Yes, but you'd have to use a Server Shared Container to be able to do that, hence the suggestion to just do the whole thing in Server.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply