Abort Job when no files found (Extrnal Source Stage)

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Aquilis
Participant
Posts: 204
Joined: Thu Apr 05, 2007 4:54 am
Location: Bangalore
Contact:

Abort Job when no files found (Extrnal Source Stage)

Post by Aquilis »

Hi All,

Am reading a file on Extrnal Source Stage using a command below.
Am doing this because I need to remove trailer and Header Records from file before reading.

Issue is - when there are no files to load, instead of ABORT, job is throwing an warning and getting completed successfully.
Can anyone suggest a better way to abort the job when no files found.
Please see the UNIX Command used (in External source Stage) and error warning below.

Code: Select all

cat #path#/REPO.*.2012-07-11*.txt | sed '1d;$d'

Code: Select all

Warning  Message :
Source subproc: cat: PATH/REPO.*.2012-07-11*.txt: No such file or directory
Aquilis
ShaneMuir
Premium Member
Premium Member
Posts: 508
Joined: Tue Jun 15, 2004 5:00 am
Location: London

Post by ShaneMuir »

You could simply put the job in a sequence, and set the output links accordinging, ie, OK will only be triggered if the job finishes with no warnings, and a second link for Otherwise can then go to a terminate stage to stop the sequence.

Or if you wanted to go even further in the sequence, you could test for the existence of a file first with an execute command stage and not even start your file reading job.
Aquilis
Participant
Posts: 204
Joined: Thu Apr 05, 2007 4:54 am
Location: Bangalore
Contact:

Post by Aquilis »

Thanks Shane.

But Is it possible to right some kind of unix script which I could call in External source stage or in any other DS stage and on the basis of return code , job gets aborted?Is anything like this possible (if i do not wish to create a sequence)?
Aquilis
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

What is your runtime warning limit - any chance you could set it to 1?
-craig

"You can never have too many knives" -- Logan Nine Fingers
ssreeni3
Participant
Posts: 29
Joined: Fri May 18, 2012 1:35 am

Post by ssreeni3 »

Yes. By using Execute Command stage and Terminator stages in a Sequence.
chetan.c
Participant
Posts: 112
Joined: Tue Jan 17, 2012 2:09 am
Location: Bangalore

Post by chetan.c »

Hi,
This code will abort the job if there are no files found.

Code: Select all

if ls #path#/REPO.*.2012-07-11*.txt  >/dev/null 2>&1;then cat #path#/REPO.*.2012-07-11*.txt |sed '1d:$d';else exit;fi
The better way would be as Shane suggested ,not to trigger the job itslef if file is not found.

Thanks,
Chetan.C
Post Reply