FTP Query

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
mydsworld
Participant
Posts: 321
Joined: Thu Sep 07, 2006 3:55 am

FTP Query

Post by mydsworld »

I have a Sequence like this :

Job1 (FTP job doing GET/PULL) -> Wait for file ->....other jobs

Now the trigger condition on Job1 is 'Success'/'Warning'.

So, will there be any possibility where partially transferred file be taken up for processing by 'Wait for File' activity stage. My intension is to get the file in full and only then start next job activities.If there is no file transferred no subsequent job should start.

Please let me know whether I am correct.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

You are correct about a partially transferred file triggering the Wait For File stage. The usual solution is to get two files, the second a small or even empty file, and trigger the Wait For File stage from the second file, which will not begin to arrive until the first file is complete.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
mydsworld
Participant
Posts: 321
Joined: Thu Sep 07, 2006 3:55 am

Post by mydsworld »

I understand the concept of Trigger file. The problem would have been with that when I have something :

Wait for File -> Other jobs

Then 'Wait for file' takes up the partially received file and start calling other jobs.

But here I am invoking the 'Wait for file' activity stage only after the successfull completion of previous FTP job.

FTP Job -> Wait for File -> Other jobs

'Wait for File' will be called only after the FTP is over and I have either full file or no file. So, do you think there is any threat of processing partially received file.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

No, no threats there. That WFF step seems superfluous to me, but whatever.
-craig

"You can never have too many knives" -- Logan Nine Fingers
mydsworld
Participant
Posts: 321
Joined: Thu Sep 07, 2006 3:55 am

Post by mydsworld »

Actually I need this WFF stage as I have two links from WFF.

1. When FTPed file not found in local directory
2. When FTPed file found in local directory.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

So - under what circumstances would you not find a file there after successful completion of the FTP job?
-craig

"You can never have too many knives" -- Logan Nine Fingers
mydsworld
Participant
Posts: 321
Joined: Thu Sep 07, 2006 3:55 am

Post by mydsworld »

FTP job runs but didn't find any file.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

I wouldn't consider that as "successful completion" but that's just me. Please to ignore the intrusion, we now return you to your regularly scheduled conversation...
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply