Server side Force Compile all

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

Post Reply
bmsq
Premium Member
Premium Member
Posts: 32
Joined: Mon Oct 30, 2006 9:19 pm

Server side Force Compile all

Post by bmsq »

Hi all,

We are using Kenneth Bland's compile all job control as part of our deployment process. Works great if we deploy to a fresh project, but jobs will fail with errors related to Transformer .o files when deploying to an existing project.

When these errors occur, a force compile from the client fixes the problem. From what I understand, this would be caused by the C/C++ compiler not recognizing that the transformer has been updated and needs to be re-compiled.

Is there any way to modify the compile all script to force compile px jobs? I can follow the routine but I don't have a deep enough understanding of Universe/PXEngine to know what all of these DSR routines are dooing. My current guess is that there must be an extra parameter for DSR.SUB.JOB.COMPILE which flags a force compile.

The complete routine is below, any help would be greatly appreciated!

Cheers,
Barry

Code: Select all

$INCLUDE DSINCLUDE DSR_UVCONST.H
$INCLUDE DSINCLUDE DSR_COMCONST.H

      Equate LOG.OK.NAME To "OK"
      Equate LOG.WARN.NAME To "!!!"
      MyJobNo = DSJobName
      SUBR = DSR.SUB.JOB
      Key = DSR.SUB.JOB.GETNO
      Call @SUBR(Key, MyJobNo)
      Dummy1 = "" ; Dummy2 = ""

      DEFFUN CallSystemCmd(aCmd, pSystemType) Calling "DSU.CallSystemCmd"

*_______________________________________Define Functions___________________________________________*
*                                                                                                  *

*______________________________________Initialize Variables________________________________________*
*                                                                                                  *
      BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
      JobFileName = "DS_JOBS"
      JobsLike = TRIM(JobsLike)

*___________________________________________Open Files_____________________________________________*
*                                                                                                  *
      Open JobFileName To JobFile Else
         Call DSLogFatal("Error opening file - ":JobFileName, BatchName)
      End

*______________________________________________Main________________________________________________*
*                                                                                                  *

      If NOT(JobsLike) Then
         Call DSLogWarn("No Jobs specified", BatchName)
         Goto Fini
      End
      JobListCnt = DCOUNT(JobsLike,",")

      If JobListCnt > 1 Then
         JobList = JobsLike
         Convert "," To @AM IN JobList
         FormList JobList
      End Else
         Begin Case
            Case JobsLike = "ALL"
               TCLcmd = "SSELECT DS_JOBS"
            Case 1
               TCLcmd = 'SSELECT DS_JOBS LIKE "...':JobsLike:'..."'
         End Case
         If Folder # "" Then TCLcmd := ' WITH F3 LIKE "':Folder:'..."'
         Call DSLogInfo("TCLcmd: ":TCLcmd, BatchName)
         Call DSExecute("TCL", TCLcmd, ScreenOutput, SystemReturnCode)
         If TRIM(ScreenOutput) # "" Then Call DSLogInfo("Screen Output: ":ScreenOutput, BatchName)
      End

      JobList = ""
      JobsSkipped = ""
      JobCnt = 0
      AllJobsDone = @FALSE

      Loop
         Readnext JobName Else AllJobsDone = @TRUE
      Until AllJobsDone Do

         If JobName[1,1] # "\" And JobName[1,22] # BatchName Then
            ProcessJob = @TRUE

            If UncompiledOnly = "Y" Then
               Readv JobNo From JobFile, JobName, 5 Then
                     cmd = "ls -l RT_BP":JobNo:".O | wc -l"
                  Result = CallSystemCmd(cmd, SystemType)
                     Result = OCONV(Result, "MCN")
                     If Result > 1 Then ProcessJob = @FALSE
                  Call DSLogInfo("JobName: ":JobName:"  JobNo: ":JobNo:"   Result: ":Result:"   ProcessJob: ":ProcessJob, BatchName)
               End Else
                  Call DSLogFatal("Error reading ID - ":JobName:"  from file - ":JobFileName, BatchName)
               End
            End

            If ProcessJob = @TRUE Then
               JobList<-1> = JobName
               JobCnt += 1
            End Else
               JobsSkipped<-1> = JobName
            End
         End
      Repeat

      If JobsSkipped Then Call DSLogInfo("Jobs skipped: ":@AM:JobsSkipped, BatchName)

      If JobList Then
         JobsCompiled = ""
         JobsNotCompiled = ""
         Call DSLogInfo("Jobs that will be compiled: ":@AM:JobList, BatchName)
         For J = 1 TO JobCnt
            JobName = JobList<J>
            Gosub CompileJob             ; * compile job
         Next J
         Call DSLogInfo("Jobs compiled: ":@AM:JobsCompiled, BatchName)
         If JobsNotCompiled Then Call DSLogWarn("Jobs NOT compiled: ":@AM:JobsNotCompiled, BatchName)
      End Else
         Call DSLogWarn("There were no jobs compiled", BatchName)
      End

      Goto Fini                          ; *** End of Program



*------------------------------------------------
*--- Subroutines
*------------------------------------------------

CompileJob:
      Key = DSR.SUB.JOB.RESERVE
      Arg2 = JobName
      Call @SUBR(Key, Arg2)
      Call DSD.Init(MyJobNo, Dummy2)
      If Key <> "" Then
         Call DSLogWarn(JobName:" - ERROR: Cannot attach to job!", BatchName)
         JobsNotCompiled<-1> = JobName:"   ":"Cannot attach to job"
         NumFailed += 1
      End Else
         Key = DSR.SUB.JOB.COMPILE
         Arg2 = JobName
         Call @SUBR(Key, Arg2)
         Call DSD.Init(MyJobNo, Dummy2)
         If Key <> "" Or Arg2<1> <> "" Then
            Call DSLogWarn(JobName:" - ERROR: Cannot compile job!", BatchName)
            JobsNotCompiled<-1> = JobName:"   ":"Cannot compile job"
            NumFailed += 1
         End Else
            Call DSLogInfo(JobName:" - Compiled OK", BatchName)
            JobsCompiled<-1> = JobName
            Ignore = 0
            JobNo = DSRGetJob(JobName,Ignore)
            Execute "CLEAR.FILE ":DSR.RTSTATUS.FNAME:JobNo
            Execute "CLEAR.FILE ":DSR.RTLOG.FNAME:JobNo
         End
         Key = DSR.SUB.JOB.RELEASE       ; * unlock job, ignoring any errors
         Arg2 = JobName
         Call @SUBR(Key, Arg2)
         Call DSD.Init(MyJobNo, Dummy2)
      End
      Return


Fini:                                    ; *** finish ***
Post Reply