Wait for file stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
mydsworld
Participant
Posts: 321
Joined: Thu Sep 07, 2006 3:55 am

Wait for file stage

Post by mydsworld »

I have a dbout regarding the 'Wait for file' activity stage in DS.Suppose I am getting a remote file through FTP. Now, think of a situation where the file hasn't come in full and 'Wait for file' detects the presence of a file and fires a subsequent job. So, does 'Wait for file' waits for the receipt of the file in full?
If not, then what could be the better solution in this case.
keshav0307
Premium Member
Premium Member
Posts: 783
Joined: Mon Jan 16, 2006 10:17 pm
Location: Sydney, Australia

Post by keshav0307 »

if the file is coming through FTP then send a another file(may be with same name with some other extension) of zero byte just after the ftp complete as flag of FTP completion. and in the file stage detect for this file presence.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Re: Wait for file stage

Post by chulett »

mydsworld wrote:So, does 'Wait for file' waits for the receipt of the file in full?
No, it simply checks for existence.

As noted, the typical solution is the use of a zero byte 'flag' or 'semaphore' file sent after the main file. When it is received, you know it is safe to go get the main file.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply