aborting jobs when data source empty

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
will
Participant
Posts: 24
Joined: Mon Apr 03, 2006 11:20 pm

aborting jobs when data source empty

Post by will »

Hello,

Although the data source is empty either because the source server is down or simply because there is no data , the job runs perfectly.If i want the job to abort what could i do? Can i make use of the activity stages in the sequencer (eg terminator) or use the triggers to detect data source empty?
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Yes, but prefer to issue a warning (UtilityWarningToLog routine) and exit (Terminator activity) rather than to abort.

Create a small job that selects COUNT(*) from source into a text file, then read this text file (Execute Command activity) then, depending on the result, either continue with regular processing or take the path described in the previous paragraph.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Iam still thinking a way to capture the file size (alone) from dos prompt.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
seanc217
Premium Member
Premium Member
Posts: 188
Joined: Thu Sep 15, 2005 9:22 am

Post by seanc217 »

How about writing a Korn shell script to capture the size?
You could do an ls and pipe that into awk and capture the file size there.
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

seanc217 wrote:How about writing a Korn shell script to capture the size?
You could do an ls and pipe that into awk and capture the file size there.
But its for windows.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
seanc217
Premium Member
Premium Member
Posts: 188
Joined: Thu Sep 15, 2005 9:22 am

Post by seanc217 »

You can write shell scripts in windows utilizing the MKS toolkit. It is installed when you install datastage. The advantage to writing shell scripts is that they will translate to Unix with few problems. If you don't want to write a shell scripts, you could write a .bat file to check filesize before you run the job and only execute if greater than 0. Use a combiniation of the dir and find command in a command window.

Hope this helps.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

kumar_s wrote:Iam still thinking a way to capture the file size (alone) from dos prompt.

Code: Select all

test -z $filename
You have MKS Toolkit installed, so you have the test command. This can also be used in an if construct in a shell script using square brackets.

Code: Select all

if [ -z $filename ] ...
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

I guess test -z $filename is again a unix command.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Yes, but if you have DataStage 7.5x2, then you have UNIX on Windows (MKS Toolkit ships and installs with the product).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
sendmk
Charter Member
Charter Member
Posts: 136
Joined: Mon Oct 03, 2005 5:02 am

Post by sendmk »

ray.wurlod wrote:Create a small job that selects COUNT(*) from source into a text file, then read this text file (Execute Command activity) then, depending on the result, either continue with regular processing or take the path described in the previous paragraph.
i wonder how you can make Execute Command activity fail/pass if source file is empty/full.

may be i did not get what ray told

pls somebody through some light.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

sendmk wrote:i wonder how you can make Execute Command activity fail/pass if source file is empty/full.
By checking the result of the command. Perhaps it counts the number of records in the file, so you check to see if the returned value is 0 or >0. Perhaps it simply tests to see if the file exists and is empty or not empty, so you check to see of the returned value is true or false. Etc Etc.

Do the checking in the triggers you define in the Execute Command stage and branch accordingly. Custom triggers that check the StageName.$ReturnValue if I remember correctly.
-craig

"You can never have too many knives" -- Logan Nine Fingers
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

kumar_s wrote:Iam still thinking a way to capture the file size (alone) from dos prompt.
May be you can try this. Did a search and found it.

Code: Select all

:: chk4zero.bat
@echo off
if %1'==' echo usage is: %0 FILENAME.EXT && goto :eof
if not exist %1 echo %1 was not found && goto :eof
if %~z1 EQU 0 goto :0
goto :eof
:: Put in any code you want to run if the file is not zero length.


::
:0
echo %1 is size zero
:: DONE
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
sendmk
Charter Member
Charter Member
Posts: 136
Joined: Mon Oct 03, 2005 5:02 am

Post by sendmk »

did not find anything as given below,
chulett wrote:StageName.$Returnvalue if I remember correctly.
my whole question is how to capture that return value in further actions.....
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Exactly as Craig stated; ActivityName.$ReturnValue where ActivityName is the name of your Execute Command activity. If you're on 7.x you can select these "activity variables" from the "Add Parameter" button's dialog. In earlier releases (5.2 and later) they had to be entered manually, but could still be used.

Where and how did you look?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply