Need shell script to start job based on remote trigger file
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 13
- Joined: Fri Mar 14, 2008 4:30 pm
Need shell script to start job based on remote trigger file
Hi Everyone,
I have a requirement to look for a trigger file on other server and upon find will kick off a DS job. For this i need to write a shell script that looks for a trigger file, if finds then kick the DS job and if not the script has to run again untill it finds the file.. something like loop or sleep mechanism..
Any one who have implemeted this similar situation can help me in writing the script, any ideas on how to acheive this requirement will also be appreciated.
Thanks in Advance
**Note: Subject line made more descriptive - Content Editor **
I have a requirement to look for a trigger file on other server and upon find will kick off a DS job. For this i need to write a shell script that looks for a trigger file, if finds then kick the DS job and if not the script has to run again untill it finds the file.. something like loop or sleep mechanism..
Any one who have implemeted this similar situation can help me in writing the script, any ideas on how to acheive this requirement will also be appreciated.
Thanks in Advance
**Note: Subject line made more descriptive - Content Editor **
-
- Premium Member
- Posts: 425
- Joined: Sat Nov 19, 2005 9:26 am
- Location: New York City
- Contact:
Re: Shell Script to kick DS job
sryarraguntla,
Take a look at the Wait For File activity in the sequence palette. This activity address what you are looking for and more .... unless you wanted it done specifically by a shell script
Take a look at the Wait For File activity in the sequence palette. This activity address what you are looking for and more .... unless you wanted it done specifically by a shell script
Julio Rodriguez
ETL Developer by choice
"Sure we have lots of reasons for being rude - But no excuses
ETL Developer by choice
"Sure we have lots of reasons for being rude - But no excuses
If the file isn't easily accessible via the WaitForFile mechanism, your other solution is to write a shell script that wakes up periodically, and uses FTP to check for the trigger and move it to a local directory (when present).
Then you can still use the WaitForFile stage in your job to check for the local copy when it arrives.
Then you can still use the WaitForFile stage in your job to check for the local copy when it arrives.
-
- Participant
- Posts: 334
- Joined: Fri Dec 01, 2006 5:17 am
- Location: Texas
Hi,
You can use the following shell script
If you want to run the jobs from different servers you can use ssh command after 'if then'...ssh command will be :
Or as suggested by asorrell use ftp(scp) and transfer the file and then run the dsjob script.
Hope this will help you out !
You can use the following shell script
Code: Select all
#/bin/sh
PATHDIR=/opt/home
val=2
while [ $val -eq 2 ]
do
find $PATHDIR -name "triggerfilename.txt" > $PATHDIR/gotit.txt #finding the trigger file
val1=`wc -l $PATHDIR/gotit.txt|cut -f1 -d'/'`
if [ $val1 -ne 0 ]
then
#run the dsjob command script
val=1
else
echo 'Still Running'
fi
sleep 35 #Increase the sleep,if required
done
Code: Select all
ssh -t -l username IP PATHDIR/dsjobrun.sh
Hope this will help you out !
http://findingjobsindatastage.blogspot.com/
Theory is when you know all and nothing works. Practice is when all works and nobody knows why. In this case we have put together theory and practice: nothing works. and nobody knows why! (Albert Einstein)
Theory is when you know all and nothing works. Practice is when all works and nobody knows why. In this case we have put together theory and practice: nothing works. and nobody knows why! (Albert Einstein)
I wrote something similar a while back. Its here.
You can edit it to fire a job once the file is found. You can also set a time limit.
You can edit it to fire a job once the file is found. You can also set a time limit.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
-
- Participant
- Posts: 13
- Joined: Fri Mar 14, 2008 4:30 pm
Really thanks for all your replys. I appreciate your help and quick response.
Ameyjoshi14-- Can i connect to the server AS400 through ftp and execute the script which you gave to check for the trigger file if found run the dsjob if not sleep..
If means the code should looks like
#!/bin/sh
HOST='xxxxx'
USER='xxxxx'
PASSWD='xxxx'
ftp $HOST <<END_SCRIPT
user $USER
$PASSWD
------ the next command would be your script ----
Please let me know if iam correct
Once again thanks
Ameyjoshi14-- Can i connect to the server AS400 through ftp and execute the script which you gave to check for the trigger file if found run the dsjob if not sleep..
If means the code should looks like
#!/bin/sh
HOST='xxxxx'
USER='xxxxx'
PASSWD='xxxx'
ftp $HOST <<END_SCRIPT
user $USER
$PASSWD
------ the next command would be your script ----
Please let me know if iam correct
Once again thanks
-
- Participant
- Posts: 334
- Joined: Fri Dec 01, 2006 5:17 am
- Location: Texas
Hi,
I am not sure if the above commands will be fine...
What we did is develop the scrpit, which can telnet automatically and run commands on the other machine and end the telnet session but for this you need to install EXPECT and TCL packages on your unix system.
Or as suggested by asorrell you can transfer the file from one server to another server (where you require it) and then run the dsjob command.
Then the script will look as :
I was not able to test this code since right now i do not have access to unix box.
Hope this will solve your problem
I am not sure if the above commands will be fine...
What we did is develop the scrpit, which can telnet automatically and run commands on the other machine and end the telnet session but for this you need to install EXPECT and TCL packages on your unix system.
Or as suggested by asorrell you can transfer the file from one server to another server (where you require it) and then run the dsjob command.
Then the script will look as :
Code: Select all
#/bin/sh
PATHDIR=/opt/home
val=2
while [ $val -eq 2 ]
do
find $PATHDIR -name "dum12.txt" > $PATHDIR/gotit.txt #finding the trigger file
val1=`wc -l $PATHDIR/gotit.txt|cut -f1 -d'/'`
if [ $val1 -ne 0 ]
then
val=1
#you can create another script and use this as wrapper.
ftp -v -n "YOUR.IP.ADD.RESS" << cmd
user "user" "passwd"
cd /distant/directory
lcd /local/directory
put your files
#All you have to do after is to create the cron if you are "putting" a file,replace the put with ls and see if your file is in there.
quit
cmd
else
echo 'Still Running'
fi
sleep 35 #Increase the sleep,if required
done
Hope this will solve your problem
http://findingjobsindatastage.blogspot.com/
Theory is when you know all and nothing works. Practice is when all works and nobody knows why. In this case we have put together theory and practice: nothing works. and nobody knows why! (Albert Einstein)
Theory is when you know all and nothing works. Practice is when all works and nobody knows why. In this case we have put together theory and practice: nothing works. and nobody knows why! (Albert Einstein)
-
- Participant
- Posts: 334
- Joined: Fri Dec 01, 2006 5:17 am
- Location: Texas
Hey! That's really nice scipt... :D if i would have found it first....by little modification(as per the requirement) lot of work in our project would have been savedDSguru2B wrote:I wrote something similar a while back. Its here.
You can edit it to fire a job once the file is found. You can also set a time ...
http://findingjobsindatastage.blogspot.com/
Theory is when you know all and nothing works. Practice is when all works and nobody knows why. In this case we have put together theory and practice: nothing works. and nobody knows why! (Albert Einstein)
Theory is when you know all and nothing works. Practice is when all works and nobody knows why. In this case we have put together theory and practice: nothing works. and nobody knows why! (Albert Einstein)