Page 1 of 1
BATCH JOB & CSV File
Posted: Sun Mar 18, 2012 11:11 pm
by karthi_gana
All,
We are using the server job to execute the parallel jobs which we designed.
The server job will contain only the annotation. We declared a parameter called "ParameterName" (may be any name) and assigned the path of .CSV file which internally have the job names as below.
Code: Select all
Job Name Controlling Predecessors Batch
TH_00_FTP_GET_SHARE BATCH_MJC_PROCESS_XXX_FEEDS
JB_01_XXX_SHARE_LOAD TH_00_FTP_GET_SHARE BATCH_MJC_PROCESS_XXX_FEEDS
JB_02_XXX_SHARE_LOAD JB_01_XXX_SHARE_LOAD BATCH_MJC_PROCESS_XXX_FEEDS
JB_03_ETF_SHARE_LOAD JB_02_XXX_SHARE_LOAD BATCH_MJC_PROCESS_XXX_FEEDS
JB_01_XXX_PERF_LOAD_FA JB_03_ETF_SHARE_LOAD BATCH_MJC_PROCESS_XXX_FEEDS
JB_02_XXX_PERF_LOAD_FB JB_01_XXX_PERF_LOAD_FA BATCH_MJC_PROCESS_XXX_FEEDS
JB_03_XXX_PERF_LOAD_FC JB_02_XXX_PERF_LOAD_FB BATCH_MJC_PROCESS_XXX_FEEDS
JB_04_XXX_PERF_LOAD_F9 JB_03_XXX_PERF_LOAD_FC BATCH_MJC_PROCESS_XXX_FEEDS
JB_05_XXX_PERF_LOAD JB_04_XXX_PERF_LOAD_F9 BATCH_MJC_PROCESS_XXX_FEEDS
JB_06_ETF_PERF_LOAD JB_05_XXX_PERF_LOAD BATCH_MJC_PROCESS_XXX_FEEDS
All the above are parallel jobs.
How the server job is executing the .csv file?
Posted: Sun Mar 18, 2012 11:31 pm
by ray.wurlod
You tell us. It's your job!
Based on your description I would guess that it's using job control code. What does the annotation tell you?
Posted: Mon Mar 19, 2012 1:51 am
by karthi_gana
yes. You are right!
Code: Select all
DEFFUN KBAJobControl(A) Calling "DSU.KBAJobControl"
DEFFUN KBAGetParameterArray(A) Calling 'DSU.KBAGetParameterArray'
DEFFUN KBAGetParam(A,B) Calling "DSU.KBAGetParam"
SystemType = (IF SYSTEM(91) = 0 THEN "UNIX" ELSE "NT")
ParameterArray = KBAGetParameterArray(ParameterFile)
*
* Retrieve the next batch processing id
*
If BatchNumber = "" Then BatchNumber = @DATE:@TIME'R%5'
If ProcessDate = "1000-01-01" Then ProcessDate = Oconv(@DATE,"D-YMD[4,2,2]")
*
* Derive any runtime parameters not found in parameter file
*
ExtraParameters = ""
* ExtraParameters<-1> = "DerivedAtRuntimeParameter1=":Result1
* ExtraParameters<-1> = "DerivedAtRuntimeParameter2=":Result2
* ExtraParameters<-1> = "DerivedAtRuntimeParameter3=":Result3
CONVERT @AM TO @VM IN ExtraParameters
*
* Configure job control array
*
JobControlArray = ""
JobControlArray<1> = ParameterFile
JobControlArray<2> = 15 ; * How many seconds delay between alarms (minutes * 60 seconds)
JobControlArray<3> = 30 ; * How many seconds a job can run until the alarm rings(minutes * 60 seconds)
JobControlArray<4> = 2 ; * Polling interval in seconds
JobControlArray<5> = 99 ; * Maximum jobs to have running in parallel (in this batch)
JobControlArray<6> = "S" ; * (S) Stop entire jobstream on first job failure (P) Proceed after job failure, truncating that particular sub-jobstream
JobControlArray<7> = JobHierarchyFile ; * Job hierarchy file
JobControlArray<8> = ProcessDate ; * Common date for all jobs
JobControlArray<9> = BatchNumber ; * Batch number
JobControlArray<10> = SourceSystemList ; * Source system list (optional)
JobControlArray<11> = SubjectAreaList ; * Subject area list (optional)
JobControlArray<12> = StartingMilestone ; * Starting milestone (optional)
JobControlArray<13> = EndingMilestone ; * Ending milestone (optional)
JobControlArray<14> = DebugMode ; * Y = extra output messages
JobControlArray<15> = JobLinkStatisticChecksFile
JobControlArray<16> = ExtraParameters ; * Extra parameters not within the parameter file (optional subarray - @VM separated)
JobControlArray<17> = ResurrectLogFile
JobFailedCount = KBAJobControl(JobControlArray)
If JobFailedCount > 0 Then
Call DSLogWarn("Jobs failed", "Msg")
Abort
End
Else
Call DSLogInfo("Jobs finished okay", "Msg")
vRemoveCmd = "rm -f "
aLogMsgs = "Deleting Resurrect Log File"
vCmd = vRemoveCmd:ResurrectLogFile
vSystemType = "UNIX"
Call DSExecute(vSystemType, vRemoveCmd:ResurrectLogFile, pScreenOutput, pSystemReturnCode)
Call DSLogInfo(vLogID:aLogMsgs:ResurrectLogFile:pScreenOutput:pSystemReturnCode:vCmd, vLogID)
End
i don't understand this code at all. which is the key part to execute the jobs one by one ?
Posted: Mon Mar 19, 2012 3:10 am
by ray.wurlod
You have obtained a copy of Ken Bland Associates job control routines. Therefore, you should also have obtained a copy of the documentation that comes with it.
Track that down on your site, or you can find Ken's website by searching DSXchange.
The "key part" is the invocation of KBAJobControl() function.
Posted: Tue Mar 20, 2012 1:53 am
by karthi_gana
Sure.
Now...If i don't have KBA and if i want to implement the same in parallel job itself instead of using server job with KBA routines...how should i achieve this?
Posted: Tue Mar 20, 2012 5:29 am
by chulett
You wouldn't.
Posted: Tue Mar 20, 2012 2:54 pm
by ray.wurlod
You couldn't (unless you created a couple of custom stage types and/or routines).
Posted: Tue Mar 20, 2012 3:50 pm
by chulett
There is a crap-ton of stuff under the covers that the 'KBA Utilities' encompasses... and I really don't think you could do anything equivalent as a straight parallel implementation. Even with a couple of custom stage types and/or routines.
Posted: Tue Mar 20, 2012 5:10 pm
by ray.wurlod
Also, quite manifestly, you DO have at least some of KBA utilities.
Posted: Tue Mar 20, 2012 5:20 pm
by chulett
Of course, but I assumed the question was can that all be pitched and replaced with strictly 'parallel' components... at least that's where my response was based on.
Posted: Tue Mar 20, 2012 8:46 pm
by ray.wurlod
... and that's just one of the reasons server jobs aren't going away any time soon.