Wait for file activity stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
splayer
Charter Member
Charter Member
Posts: 502
Joined: Mon Apr 12, 2004 5:01 pm

Wait for file activity stage

Post by splayer »

How do I wait till my file has completely copied to the folder before I say that Wait For File Acitvity is successful and fire off the next stage?
seanc217
Premium Member
Premium Member
Posts: 188
Joined: Thu Sep 15, 2005 9:22 am

Post by seanc217 »

From what I have heard most people send a trigger file after the file you want to monitor for transfer is complete. So, you would monitor for the trigger file and then start your job.

Hope this helps
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Exactly... an empty trigger file, typically, that is only sent after the main data file has completely transferred. Otherwise you face exactly the situation you've described - the WFF triggers the moment it detects the file, finished or not.

Also referred to as a 'semaphore' file.
-craig

"You can never have too many knives" -- Logan Nine Fingers
kris007
Charter Member
Charter Member
Posts: 1102
Joined: Tue Jan 24, 2006 5:38 pm
Location: Riverside, RI

Post by kris007 »

Here's one of the practices I have been following. Define a process such that whenever you are copying a file say A.txt it is followed by another dummy file say A.dummy. In this case, once A.txt is copied completely, A.dummy is copied. Now all you will need to do is wait for A.dummy before you start processing A.txt. Hope that was clear. :wink:
Kris

Where's the "Any" key?-Homer Simpson
splayer
Charter Member
Charter Member
Posts: 502
Joined: Mon Apr 12, 2004 5:01 pm

Post by splayer »

I know about this solution as it has been discussed in this forum before. This requires that the source process be changed to send the 2nd file. I was wondering if there was a solution using only OS(UNIX) tools or Datastage but apparently there isn't.
Post Reply