Dear All ,
we need to run sequence job depending on reading flag from database ,if flag Y then sequence job will start else job abort without running .How we can implement this work flow in DS. and every 15 minute check again this flag .
work flow in IBM DataStage
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 50
- Joined: Sat Jan 31, 2009 3:39 am
- Location: Riyadh
- Contact:
There are many ways to do this. First of all, you need to decide on a method to read your database flag. Here are just 2 options:
a) DS job to read the DB and, if "Y", create a dummy file
b) UNIX command line call to the DB which creates output which can be parsed for the flag
Once you have a method of flagging, you can create a sequence with a loop that has a 15-minute pause in it. Check for the flag and depending upon the result execute your actual processing job.
a) DS job to read the DB and, if "Y", create a dummy file
b) UNIX command line call to the DB which creates output which can be parsed for the flag
Once you have a method of flagging, you can create a sequence with a loop that has a 15-minute pause in it. Check for the flag and depending upon the result execute your actual processing job.
-
- Premium Member
- Posts: 50
- Joined: Sat Jan 31, 2009 3:39 am
- Location: Riyadh
- Contact:
Re: work flow in IBM DataStage
if we take first option
a) DS job to read the DB and, if "Y", create a dummy file
how I can pass flag Y to anther job to start running or not ?????
what the benefit to crate dummy file ?
a) DS job to read the DB and, if "Y", create a dummy file
how I can pass flag Y to anther job to start running or not ?????
what the benefit to crate dummy file ?
The dummy file gives you something to check for, as Arnd noted. Think of it like a flag or semaphore - your Sequence job checks for the existence of the file and only if found allow the processing job to run. And there are several ways you can 'check', one way is by using a simple UNIX test function in an Execute Command stage, for example.
It might be a little tricky to conditionally create the file in a job, however. You'd need something like a routine to do that in a transformer rather than relying on the Sequential File stage, as it will always create the file regardless of the quantity of data sent to it. You could conditionally populate it however, that would change the above technique from one that simply checks for the file to one that actually needs to read the file as well. Still, not that hard to do.
Another approach, perhaps easier to understand and implement, would be to leverage the USERSTATUS area. The 'DB Checker' job writes something like "Y" or "N" to its own user status area and then anything downstream of it in the Sequence job can easily check that value and trigger the appropriate action or path based on the value found. Search the forums for USERSTATUS to see the plethora of conversations already out there on this topic.
It might be a little tricky to conditionally create the file in a job, however. You'd need something like a routine to do that in a transformer rather than relying on the Sequential File stage, as it will always create the file regardless of the quantity of data sent to it. You could conditionally populate it however, that would change the above technique from one that simply checks for the file to one that actually needs to read the file as well. Still, not that hard to do.
Another approach, perhaps easier to understand and implement, would be to leverage the USERSTATUS area. The 'DB Checker' job writes something like "Y" or "N" to its own user status area and then anything downstream of it in the Sequence job can easily check that value and trigger the appropriate action or path based on the value found. Search the forums for USERSTATUS to see the plethora of conversations already out there on this topic.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers