How do I get dsjob -logdetail to list the log from the last
Moderators: chulett, rschirm, roy
How do I get dsjob -logdetail to list the log from the last
Hi,
I did search on "dsjob -logdetail", plus a lot of hacking with dsjob, before posting...
How do I get dsjob -logdetail to list the log from the last job run? IOW, how can I derive the <first event id>? The only way I could see is issue dsjob -logsum <project> <jobname>, capture the first line, and cut the event id. And I think this would only work if I'd first cleared the log before the job run.
Even better would be if I could limit the output to Warnings. But I think grep would take care of that.
Thanks,
Scott
I did search on "dsjob -logdetail", plus a lot of hacking with dsjob, before posting...
How do I get dsjob -logdetail to list the log from the last job run? IOW, how can I derive the <first event id>? The only way I could see is issue dsjob -logsum <project> <jobname>, capture the first line, and cut the event id. And I think this would only work if I'd first cleared the log before the job run.
Even better would be if I could limit the output to Warnings. But I think grep would take care of that.
Thanks,
Scott
Some people are the Michael Jordan's of Datastage. I'm more like Muggsy Bogues
I need the log details of the last job run. I'm getting a lot of data truncation warnings writing to the database, and need to pass this information to the provider of the source data so they can clean it.nagarjuna wrote:Are you trying to capture the lateset log ? May be you can try using dsjob -lognewest.
Some people are the Michael Jordan's of Datastage. I'm more like Muggsy Bogues
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
So?
The -lognewest allows you to find the most recent "started" event, and you can use that information to filter what you get back from -logdetail.
It's probably easier to do this with a DataStage BASIC routine than with dsjob commands.
The -lognewest allows you to find the most recent "started" event, and you can use that information to filter what you get back from -logdetail.
It's probably easier to do this with a DataStage BASIC routine than with dsjob commands.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Hi Ray,ray.wurlod wrote:So?
The -lognewest allows you to find the most recent "started" event, and you can use that information to filter what you get back from -logdetail.
It's probably easier to do this with a DataStage BASIC routine than with dsjob commands.
-lognewest yields:
Code: Select all
dsjob -lognewest MY_PROJECT MyJob
Newest id = 44
Status code = 0
So I go into Director, open the last entry for that job, and see that it's #44.
The first entry for the last run of that job (the first "black" line in the log) is #17.
I need to list -logdetail for entry #17 - #44, filtering for warnings only (the actual job has thousands of lines in the log).
I can leave the last entry off the -logdetail command, but how do I get the first entry (thanks Kim).
Now, wouldn't
Code: Select all
dsjob -logdetail -lastrun MY_PROJECT MyJob
If anyone has written a Unix wrapper script that does this, please post it. Otherwise, if i write one, I'll post it.
I just need to do this as a one off email to the source file providers, so I'm not getting how a BASIC routine would be easier.
Thanks,
Scott
Some people are the Michael Jordan's of Datastage. I'm more like Muggsy Bogues
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Code: Select all
#!/bin/ksh
#######
################################################################################
#######
####### FILE: dsjob_logwarns.ksh
#######
####### DESCRIPTION: Retrieve Warning and Fatal error messages from datastage job
####### log for latest run
#######
####### Steps:
####### 1) Use dsjob -logsum to find the START EVENT ID of the most
####### recent run of the job
####### 2) Use dsjob -logsum to find all the WARNING EVENT IDs on the log
####### 3) Use dsjob -logsum to find all the FATAL EVENT IDs on the log
####### 4) Loop through the warnings and fatal events and get the details
####### for each log entry which is in the most recent run using
####### dsjob -logdetail
####### 5) Write all these out to a file ${PROG}_${JOBID}.out
#######
#######
####### PARAMETERS: SERVERID="${1}"
####### USERID="${2}"
####### PASSWORD="${3}"
####### PROJECTID="${4}"
####### JOBID="${5}"
#######
####### Date Developer Description
####### ---------- -------------- ----------------------------------------------
####### 2009-05-06 Mike Favero Initial Release
#######
#######
PROG="`basename ${0}`"
Now=`date`
ReturnValue=0
#
# Check for correct number of parameters
#
if [ ${#} -ne 5 ]
then
ReturnValue=1
echo "${Now} ${PROG} : Invalid parameter list."
echo "${Now} ${PROG} : The script needs 5 parameters: 1. SERVERID=${1}"
echo "${Now} ${PROG} : 2. USERID=${2}"
echo "${Now} ${PROG} : 3. PASSWORD=${3}"
echo "${Now} ${PROG} : 4. PROJECTID=${4}"
echo "${Now} ${PROG} : 5. JOBID=${5}"
echo "${Now} ${PROG} Exiting with ReturnValue of: ${ReturnValue}"
exit ${ReturnValue}
fi
SERVERID="${1}"
USERID="${2}"
PASSWORD="${3}"
PROJECTID="${4}"
JOBID="${5}"
START_ID=`dsjob -server ${SERVERID} -user ${USERID} -password ${PASSWORD} -logsum -type STARTED ${PROJECTID} ${JOBID} | nawk 'ORS=(FNR%2)?FS:RS' | grep Starting | tail -1 | awk '{print $1 }'`
WARN_IDS=`dsjob -server ${SERVERID} -user ${USERID} -password ${PASSWORD} -logsum -type WARNING ${PROJECTID} ${JOBID} | grep WARNING | awk '{print $1 }'`
FATAL_IDS=`dsjob -server ${SERVERID} -user ${USERID} -password ${PASSWORD} -logsum -type FATAL ${PROJECTID} ${JOBID} | grep FATAL | awk '{print $1 }'`
echo "Job ${PROJECTID} ${JOBID} warning and fatal messages:" > ${PROG}_${JOBID}.out
ALL_IDS=`echo "${WARN_IDS} ${FATAL_IDS}"`
for TEST_ID in ${ALL_IDS}
do
if [[ "${TEST_ID}" -gt "${START_ID}" ]]
then
WARN_DTL=`dsjob -server ${SERVERID} -user ${USERID} -password ${PASSWORD} -logdetail ${PROJECTID} ${JOBID} ${TEST_ID}`
echo ${WARN_DTL} >> ${PROG}_${JOBID}.out
fi
done
exit ${ReturnValue}
Michael Favero
2852 Humboldt Ave So
Minneapolis, MN 55408
2852 Humboldt Ave So
Minneapolis, MN 55408
Michael, many thanks also.
Question: I've encountered important details in INFO messages, especially from FTP stages. I was wondering do you, Phil or anyone else have an idea of how to identify them in a generic way? My self-interest is in being able to display for non-DS users the log messages, and leaving out the messages coming in from the external process -- which end up in INFO types -- just makes them wake me up to logon and find them.
Question: I've encountered important details in INFO messages, especially from FTP stages. I was wondering do you, Phil or anyone else have an idea of how to identify them in a generic way? My self-interest is in being able to display for non-DS users the log messages, and leaving out the messages coming in from the external process -- which end up in INFO types -- just makes them wake me up to logon and find them.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
-
- Premium Member
- Posts: 1044
- Joined: Wed Sep 29, 2004 3:30 am
- Location: Nottingham, UK
- Contact:
You could easily add INFO messages to this script, but grep them for specific pieces of text. This is particulary true of sequences, e.g. "JobSeq.285.JobControl (DSWaitForJob): Job Job_Name.285 has finished, status = 3 (Aborted)" is an Info.FranklinE wrote:Michael, many thanks also.
Question: I've encountered important details in INFO messages, especially from FTP stages. I was wondering do you, Phil or anyone else have an idea of how to identify them in a generic way? My self-interest is in being able to display for non-DS users the log messages, and leaving out the messages coming in from the external process -- which end up in INFO types -- just makes them wake me up to logon and find them.
That would look something like this:
Code: Select all
INFO_IDS=`dsjob -server ${SERVERID} -user ${USERID} -password ${PASSWORD} -logsum -type INFO ${PROJECTID} ${JOBID} | grep "JobControl.+status = 3.+Aborted" | awk '{print $1 }'`
Code: Select all
ALL_IDS=`echo "${WARN_IDS} ${FATAL_IDS} ${INFO_IDS}"`
Code: Select all
INFO_IDS=`dsjob -server ${SERVERID} -user ${USERID} -password ${PASSWORD} -logsum -type INFO ${PROJECTID} ${JOBID} | grep "(JobControl.+status = 3.+Aborted)|(rsMyRoutine.+Major error in routine)|(some_other_pattern)" | awk '{print $1 }'`
Phil Hibbs | Capgemini
Technical Consultant
Technical Consultant