Link Count

[srimitta]

here what I exactly what you needed from it

is a COPY/PASTE from Craig's response

Short answer: as one long varchar field. Long answer would depend on exactly what you needed from it.

Sorry about that.
SriMitta

[srimitta]

Hi Ray,

To get Job Name I tried Field() function Field(DSLink2.JOBS,':',2) in Derivation.

It's returning Job Name & anything after ':'.

Any how to use system variable @FM to read required data.

Thanks
SriMitta

[AmeyJoshi14]

Sorry earlier reply was not complete, here what I exactly what you needed from it.

JOB_NAME
JOB_START_TIME
JOB_END_TIME
STAGE_NAME
LINK_NAME
ROW_COUNT
JOB_STATUS

Thanks
SriMitta

Hi,
You can run the shell script after the job is finished.In the script you can use "dsjob" command.
The command are as follows:
$DSHOME/bin/dsjob -report $projectname $jobname BASIC
Store the above outptut in some file say tmp.txt and after that you can use various commands like:

Code: Select all

StartTime=`more tmp.txt| fgrep "Job start time" |cut -f2 -d'='`   #Getting Start time
 EndTime=`more $TempDir/tmp.txt| fgrep "Job end time" |cut -f2 -d'='`                    #Getting End time

If you want i will post the detail script !

[srimitta]

Hi Amey
Yeah, please post the detail script and let me see if I can make it happen with your script.

Thanks
SriMitta

[AmeyJoshi14]

In this script you have to pass the Projectname and Jobname as the parameter.You can even hardcode this values as per your requirement.

Code: Select all

#!/bin/ksh
TempDir=PATHNAME
proj=PROJECT_NAME #your project name
jobname=JOB_NAME  # name of the job for which information is required.
 . $DSHOME/dsenv
 $DSHOME/bin/dsjob -report $proj $jobname BASIC > $TempDir/tmp.txt
 StartTime=`more $TempDir/tmp.txt| fgrep "Job start time" |cut -f2 -d'='`                        #Getting Start time
 EndTime=`more $TempDir/tmp.txt| fgrep "Job end time" |cut -f2 -d'='`                            #Getting End time
 TimeConsumed=`more $TempDir/tmp.txt| fgrep "Job elapsed time" |cut -f2 -d'='`                   #Getting Time Consumed
 Status=`more $TempDir/tmp.txt| fgrep "Job status" |cut -f2 -d'='|cut -f2 -d'('|cut -f1 -d')'`   #Status of job
 rundate=`more $TempDir/tmp.txt| fgrep "Generated"|cut -f2 -d':'|cut -f2 -d' '`                  #Getting Status of the job
 #Populating row count
. $DSHOME/dsenv
$DSHOME/bin/dsjob -lstages $proj $jobname > $TempDir/stage.txt          # This will give the name of each and every stage used in the job.
 # I am using loop so that we can get row count for each and every stage used in the job.
cat $TempDir/stage.txt |while read line                        
       do
         stagename=`echo $line`
         echo $stagename
         dsjob -stageinfo $proj $jobname $stagename >> $TempDir/testing.txt
         rowcount=`more $$TempDir/testing.txt | fgrep "In Row Number"|cut -f2 -d':'`     # Getting the Row Count
         LinkName=`$DSHOME/bin/dsjob -llinks $proj $jobname $stagename >> $TempDir/LinkName.txt  # This will give the Link_name for each every stage.
         echo '\n' RowCount for stage $stagename = $rowcount >> $TempDir/rowcount.txt    
         echo $rowcount
       done       

I think this will work fine :D

[srimitta]

Hi Amey, your script is good.
Below is the result of your script, but this is not what I was looking for.

Status code = 0

Status code = 0
Transformer_3

Status code = 0

Status code = 0
0 0 0 0 49 49 49 49 49

I am looking for
JOB_NAME
JOB_START_TIME
JOB_END_TIME
STAGE_NAME
LINK_NAME
ROW_COUNT
JOB_STATUS

Any thoughts

Thanks
SriMitta

[ray.wurlod]

Hi Ray,

To get Job Name I tried Field() function Field(DSLink2.JOBS,':',2) in Derivation.

It's returning Job Name & anything after ':'.

Any how to use system variable @FM to read required data.

Thanks
SriMitta

Code: Select all

Field(Field(DSLink2.JOBS, @FM, 1), ":", 2)

or

Code: Select all

Field(DSLink2.JOBS<1>, ":", 2)

[AmeyJoshi14]

Hi,
As per my understanding you want all the stage names and link names used in the particular job.
I have created test job with name sample which is having sequential input --> transfomer --> Sequential Output.
I have modified the script :

Code: Select all

#!/bin/ksh
TempDir=PATHNAME 
proj=PROJECT_NAME #your project name 
jobname=JOB_NAME  # name of the job for which information is required. 
. $DSHOME/dsenv
 $DSHOME/bin/dsjob -report $proj $jobname BASIC 2>/dev/null > $TempDir/tmp.txt 
 JOB_NAME=$jobname
 JOB_START_TIME=`more $TempDir/tmp.txt| fgrep "Job start time" |cut -f2 -d'='`                        #Getting Start time 
 JOB_END_TIME=`more $TempDir/tmp.txt| fgrep "Job end time" |cut -f2 -d'='`                            #Getting End time 
 JOB_STATUS=`more $TempDir/tmp.txt| fgrep "Job status" |cut -f2 -d'='|cut -f2 -d'('|cut -f1 -d')'`   #Status of job 
 #The following are the stages used in the job.
 . $DSHOME/dsenv 
 $DSHOME/bin/dsjob -lstages $proj $jobname 2>/dev/null > $TempDir/stage.txt          # This will give the name of each and every stage used in the job. 
 # I am using loop so that we can get row count for each and every stage used in the job. 
cat $TempDir/stage.txt |while read line                        
       do 
         stagename=`echo $line` 
         rowcount=`dsjob -stageinfo $proj $jobname $stagename 2>/dev/null | grep "In Row Number" |cut -d':' -f2|awk '{printf $1}'`     # Getting the Row Count 
         LinkName=`dsjob -llinks $proj $jobname $stagename 2>/dev/null >> $TempDir/LinkName.txt`  # This will give the Link_name for each every stage. 
         echo RowCount for stage $stagename = $rowcount >> $TempDir/rowcount1.txt    
       done       
# Displaying the values
echo JOB_NAME=$JOB_NAME
echo JOB_START_TIME=$JOB_START_TIME
echo JOB_END_TIME=$JOB_END_TIME
echo JOB_STATUS=$JOB_STATUS
echo The stages used in the job are :
cat  $TempDir/stage.txt
echo Row count for the stages are:
cat   $TempDir/rowcount1.txt 
echo The Link name are :
cat  $TempDir/LinkName.txt

The output of the script is :

Code: Select all

JOB_NAME=Sample
JOB_START_TIME=2008-03-18 14:42:49
JOB_END_TIME=2008-03-18 14:42:52
JOB_STATUS=Finished OK

The stages used in the job are :
Sequential_File_27
Transformer_35
Sequential_File_48

Row count for the stages are:
RowCount for stage Sequential_File_27 = 4
RowCount for stage Transformer_35 = 4
RowCount for stage Sequential_File_48 = 4

The Link name are :
DSLink29
DSLink29
DSLink43
DSLink43

If you want to mantain your given sequence move all the parameters($JOB_NAME, $JOB_START_TIME etc) into the do while loop.

Thanks!