Branching Code without Raising Director Warnings

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
palmeal
Participant
Posts: 122
Joined: Thu Oct 14, 2004 7:56 am
Location: Edinburgh, Scotland

Branching Code without Raising Director Warnings

Post by palmeal »

I have a Job Sequence that creates approx 500 data files (one at a time inside a loop).
After each file is created I run an awk command in a Routine to check the contents of the file. If the file has certain data then I write debug info out to another file - this is the case in about 1 in 5 files.
If the debug file is created then I want my Job Sequence to follow a different path. To do this I have tried using an 'ls' command inside an Execute Command Activity and have also used a Wait For File Activity - these both work fine and do what I want EXCEPT that when the debug file is not found a warning message is written to the Director log. When running a datastage job from the command line my process crashes after 50 warnings (as covered in other posts).
What can I do to branch my code on the existence of a file without a warning being written to Director when it isn't found ? If I pass anything other than 0 out of a Routine I get a warning written to Director log - that would be the simple option.
I even tried 'touch'ing the debug file to get an empty entry and then using 'sum filename' in an Execute Command Activity and writing a custom trigger to check the left most part of the return <> 0 for branching the code but this didn't work.
I feel that I am missing something really simple here and would appreciate any pointers.
There are only 10 kinds of people in the world, those that understand binary and those that don't.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Use a custom trigger rather than an unconditional trigger.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
palmeal
Participant
Posts: 122
Joined: Thu Oct 14, 2004 7:56 am
Location: Edinburgh, Scotland

Post by palmeal »

Thanks Ray but I was already using a Custom trigger rather that an unconditional one - my problem though was trying to use the ReturnStatus rather than CommandOutput.

So my solution is:

Check data file for Data (in Basic Routine)
If data file has certain info then
create a debug file with info added to it
Else
create the same debug file but empty (use 'touch')
End

Execute Command Activity - sum (debug file name)
Execute Command Activity Triggers:
(1) Left(DoesStatusFileExist.$CommandOutput,1)=0
(2) Otherwise

i.e. if the debug file has a 0 size then branch one way otherwise branch the other.

This works but seems over complicated for checking to see if a file has data in it or not.

Any thoughts or suggestions on how to make this process simpler would be appreciated. A flag that could be set between activities (and not lost) or a way to avoid writing warnings to the Director log. It's a shame that Ans has to be passed as a 0 in order to suppress warnings. Being able to pass values through Ans or another means would make this product a whole lot quicker to develop.
There are only 10 kinds of people in the world, those that understand binary and those that don't.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

I agree. Sometimes I use the job's user status area as a mechanism to pass back a value.

Since routine arguments are passed by reference, you can alter their values in the routine and retrieve the results from the arguments. It's not recommended practice, but it does work. Remember to document a warning to future maintainers that you're relying the side effect!
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply