Page 1 of 1

BATCH JOB & CSV File

Posted: Sun Mar 18, 2012 11:11 pm
by karthi_gana
All,

We are using the server job to execute the parallel jobs which we designed.

The server job will contain only the annotation. We declared a parameter called "ParameterName" (may be any name) and assigned the path of .CSV file which internally have the job names as below.

Code: Select all

Job Name	Controlling Predecessors	Batch
TH_00_FTP_GET_SHARE		BATCH_MJC_PROCESS_XXX_FEEDS
JB_01_XXX_SHARE_LOAD	TH_00_FTP_GET_SHARE	BATCH_MJC_PROCESS_XXX_FEEDS
JB_02_XXX_SHARE_LOAD	JB_01_XXX_SHARE_LOAD	BATCH_MJC_PROCESS_XXX_FEEDS
JB_03_ETF_SHARE_LOAD	JB_02_XXX_SHARE_LOAD	BATCH_MJC_PROCESS_XXX_FEEDS
JB_01_XXX_PERF_LOAD_FA	JB_03_ETF_SHARE_LOAD	BATCH_MJC_PROCESS_XXX_FEEDS
JB_02_XXX_PERF_LOAD_FB	JB_01_XXX_PERF_LOAD_FA	BATCH_MJC_PROCESS_XXX_FEEDS
JB_03_XXX_PERF_LOAD_FC	JB_02_XXX_PERF_LOAD_FB	BATCH_MJC_PROCESS_XXX_FEEDS
JB_04_XXX_PERF_LOAD_F9	JB_03_XXX_PERF_LOAD_FC	BATCH_MJC_PROCESS_XXX_FEEDS
JB_05_XXX_PERF_LOAD	JB_04_XXX_PERF_LOAD_F9	BATCH_MJC_PROCESS_XXX_FEEDS
JB_06_ETF_PERF_LOAD	JB_05_XXX_PERF_LOAD	BATCH_MJC_PROCESS_XXX_FEEDS
All the above are parallel jobs.

How the server job is executing the .csv file?

Posted: Sun Mar 18, 2012 11:31 pm
by ray.wurlod
You tell us. It's your job!

Based on your description I would guess that it's using job control code. What does the annotation tell you?

Posted: Mon Mar 19, 2012 1:51 am
by karthi_gana
yes. You are right!

Code: Select all

   DEFFUN KBAJobControl(A) Calling "DSU.KBAJobControl"
      DEFFUN KBAGetParameterArray(A) Calling 'DSU.KBAGetParameterArray'
      DEFFUN KBAGetParam(A,B) Calling "DSU.KBAGetParam"
      

      
      SystemType = (IF SYSTEM(91) = 0 THEN "UNIX" ELSE "NT")

      ParameterArray = KBAGetParameterArray(ParameterFile)

*
*  Retrieve the next batch processing id
*

      If BatchNumber = "" Then BatchNumber = @DATE:@TIME'R%5'
      If ProcessDate = "1000-01-01" Then ProcessDate = Oconv(@DATE,"D-YMD[4,2,2]")



*
*  Derive any runtime parameters not found in parameter file
*
        ExtraParameters = ""
           
*      ExtraParameters<-1> = "DerivedAtRuntimeParameter1=":Result1
*      ExtraParameters<-1> = "DerivedAtRuntimeParameter2=":Result2
*      ExtraParameters<-1> = "DerivedAtRuntimeParameter3=":Result3
      CONVERT @AM TO @VM IN ExtraParameters
*
*  Configure job control array
*
      JobControlArray = ""
      JobControlArray<1> = ParameterFile
      JobControlArray<2> = 15                ; * How many seconds delay between alarms (minutes * 60 seconds)
      JobControlArray<3> = 30                ; * How many seconds a job can run until the alarm rings(minutes * 60 seconds)
      JobControlArray<4> = 2                 ; * Polling interval in seconds
      JobControlArray<5> = 99                ; * Maximum jobs to have running in parallel (in this batch)
      JobControlArray<6> = "S"               ; * (S) Stop entire jobstream on first job failure (P) Proceed after job failure, truncating that particular sub-jobstream
      JobControlArray<7> = JobHierarchyFile  ; * Job hierarchy file
      JobControlArray<8> = ProcessDate       ; * Common date for all jobs
      JobControlArray<9> = BatchNumber       ; * Batch number
      JobControlArray<10> = SourceSystemList           ; * Source system list (optional)
      JobControlArray<11> = SubjectAreaList  ; * Subject area list (optional)
      JobControlArray<12> = StartingMilestone          ; * Starting milestone (optional)
      JobControlArray<13> = EndingMilestone  ; * Ending milestone (optional)
      JobControlArray<14> = DebugMode        ; * Y = extra output messages
      JobControlArray<15> = JobLinkStatisticChecksFile
      JobControlArray<16> = ExtraParameters  ; * Extra parameters not within the parameter file (optional subarray - @VM separated)
      JobControlArray<17> = ResurrectLogFile

      JobFailedCount = KBAJobControl(JobControlArray)
      If JobFailedCount > 0 Then 
          Call DSLogWarn("Jobs failed", "Msg")
          Abort
      End 
     Else 
         Call DSLogInfo("Jobs finished okay", "Msg")
         vRemoveCmd = "rm -f "
         aLogMsgs = "Deleting Resurrect Log File"
         vCmd = vRemoveCmd:ResurrectLogFile
         vSystemType = "UNIX"
         Call DSExecute(vSystemType, vRemoveCmd:ResurrectLogFile, pScreenOutput, pSystemReturnCode)
         Call DSLogInfo(vLogID:aLogMsgs:ResurrectLogFile:pScreenOutput:pSystemReturnCode:vCmd, vLogID)
      End
i don't understand this code at all. which is the key part to execute the jobs one by one ?

Posted: Mon Mar 19, 2012 3:10 am
by ray.wurlod
You have obtained a copy of Ken Bland Associates job control routines. Therefore, you should also have obtained a copy of the documentation that comes with it.
Track that down on your site, or you can find Ken's website by searching DSXchange.

The "key part" is the invocation of KBAJobControl() function.

Posted: Tue Mar 20, 2012 1:53 am
by karthi_gana
Sure.

Now...If i don't have KBA and if i want to implement the same in parallel job itself instead of using server job with KBA routines...how should i achieve this?

Posted: Tue Mar 20, 2012 5:29 am
by chulett
You wouldn't.

Posted: Tue Mar 20, 2012 2:54 pm
by ray.wurlod
You couldn't (unless you created a couple of custom stage types and/or routines).

Posted: Tue Mar 20, 2012 3:50 pm
by chulett
There is a crap-ton of stuff under the covers that the 'KBA Utilities' encompasses... and I really don't think you could do anything equivalent as a straight parallel implementation. Even with a couple of custom stage types and/or routines.

Posted: Tue Mar 20, 2012 5:10 pm
by ray.wurlod
Also, quite manifestly, you DO have at least some of KBA utilities.

Posted: Tue Mar 20, 2012 5:20 pm
by chulett
Of course, but I assumed the question was can that all be pitched and replaced with strictly 'parallel' components... at least that's where my response was based on.

Posted: Tue Mar 20, 2012 8:46 pm
by ray.wurlod
... and that's just one of the reasons server jobs aren't going away any time soon.