Page 1 of 2
Help needed for FTP stage...
Posted: Fri Jan 02, 2009 7:59 am
by sagar deshmukh
I have a job......
sequential stage---->FTP stage
what i need to do is.....
1) job should run sucessfully...
2)it should not post any black file to remote server if the input has no value....
is it possible?
Posted: Fri Jan 02, 2009 8:33 am
by chulett
Guessing "black" = "blank".
Sequence job. Use a routine to check for an empty file and skip the FTP job if needed. Notification for empty file or job failure.
Posted: Fri Jan 02, 2009 8:34 am
by srinagesh
write a script to Do a wc-l to count the number of lines, FTP the file if file exists and number of lines is greater than zero.
Posted: Fri Jan 02, 2009 8:46 am
by sagar deshmukh
i have a job control for this where i have sequenced it...
can you please share a code how to check the input is null or not in this case?
Posted: Fri Jan 02, 2009 9:10 am
by chulett
You can use DSGetLinkInfo() to check how many records went down the creation link or DSExecute() for O/S commands to check the actual file: "wc -l", "test -s", etc.
Posted: Fri Jan 02, 2009 9:59 am
by sagar deshmukh
Code: Select all
DEFFUN RunRestartableJob(JobName,HASHPATH,BatchNumber,SKIPPARAM) Calling "DSU.RunRestartableJob"
DEFFUN EmailErrorMessage(HASHPATH,BatchNumber) Calling "DSU.EmailErrorMessage"
DEFFUN JobControlReset(JobList) Calling "DSU.JobControlReset"
DEFFUN RunBatch(JobList, Params) Calling "DSU.RunBatch"
DEFFUN ConvDateYYYYMMDD.DashDeltoMMDDYYYY(DATE) Calling "DSU.ConvDateYYYYMMDD.DashDeltoMMDDYYYY"
DEFFUN GetHashValueByKey(File,Key) Calling "DSU.GetHashValueByKey"
BatchStartDts = DSGetJobInfo(DSJ.ME, DSJ.JOBSTARTTIMESTAMP)
** Run hashCheckControlDTS outside of restart framework - want to run on restart **
JobName = "hashCheckControlDTS"
BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
BatchConfig<1> = BatchName ; * current job name or job run id
BatchConfig<2> = 1 ; * Maximum jobs to have running in parallel (in this batch)
BatchConfig<3> = "S" ; * P = parallel batch, S = serial batch
BatchConfig<4> = "JOB" ; * Job List Type (JOB=running jobs, BATCH=running batches)
BatchConfig<5> = "RUN" ; * run type (RUN=run jobs, RESET=reset jobs, VALIDATE=validate)
CONVERT @AM TO "," IN BatchConfig
Errcode = RunBatch(JobName,BatchConfig)
If Errcode Then
GOTO ENDJOB
End
** End of hashCheckControlDTS ***
JavaBatchDTS = GetHashValueByKey(HASHPATH:"hshCCDTS","")
If Not (ConvDateYYYYMMDD.DashDeltoMMDDYYYY(JavaBatchDTS) = ConvDateYYYYMMDD.DashDeltoMMDDYYYY(BatchStartDts)) Then
*abort job
Call DSLogFatal("Batch ran failed! Invoice Java Batch Job has not completed.", BatchName)
End
JobName = "hashProcessControlDTS"
BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
Errcode = RunRestartableJob(JobName,HASHPATH,BATCHNUMBER,"BATCHTIMESTAMP=":BatchStartDts)
If Errcode Then
GOTO ENDJOB
End
JobName = "srcDlyCalcTxns_LN"
BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
Errcode = RunRestartableJob(JobName,HASHPATH,BATCHNUMBER,"BATCHTIMESTAMP=":BatchStartDts)
If Errcode Then
GOTO ENDJOB
result = DSGetLinkInfo(DSJ.ME,DSJ.ME,DSJ.ME,DSJ.LINKROWCOUNT)
End
If Result <> 0 Then
StgFileName = "LNREMIT"
JobName = "xfmDlyCalcTxns_LN"
BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
Errcode = RunRestartableJob(JobName,HASHPATH,BATCHNUMBER,"BATCHTIMESTAMP=":BatchStartDts)
If Errcode Then
GOTO ENDJOB
End
StgFileName = "LiveNationRemit_US"
RemitFileName = "LiveNationRemit_US_":ConvDateYYYYMMDD.DashDeltoMMDDYYYY(BatchStartDts):".csv"
JobName = "ftpLIveNationRemitFile_US"
BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
Errcode = RunRestartableJob(JobName,HASHPATH,BATCHNUMBER,"BATCHTIMESTAMP=":BatchStartDts:",REMITFILENAME=":RemitFileName:",STGFILENAME=":StgFileName)
If Errcode Then
GOTO ENDJOB
End
StgFileName = "LiveNationRemit_CA"
RemitFileName = "LiveNationRemit_CA_":ConvDateYYYYMMDD.DashDeltoMMDDYYYY(BatchStartDts):".csv"
JobName = "ftpLiveNationRemitFile_CA"
BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
Errcode = RunRestartableJob(JobName,HASHPATH,BATCHNUMBER,"BATCHTIMESTAMP=":BatchStartDts:",REMITFILENAME=":RemitFileName:",STGFILENAME=":StgFileName)
If Errcode Then
GOTO ENDJOB
End
JobName = "Batch::LNRemitMaintenance"
BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
Errcode = RunRestartableJob(JobName,HASHPATH,BATCHNUMBER,"BATCHTIMESTAMP=":BatchStartDts)
If Errcode Then
GOTO ENDJOB
End
JobName = "ldiLNRemitProcessControl"
BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
Errcode = RunRestartableJob(JobName,HASHPATH,BATCHNUMBER,"BATCHTIMESTAMP=":BatchStartDts)
If Errcode Then
GOTO ENDJOB
End
ENDJOB:
* Setup DailyPCJobStatsLdiNew, run it, wait for it to finish, and test for success
JobName = "DailyPCJobStatsLdiNew.":BATCHNAME
Result = JobControlReset(JobName)
hJob1 = DSAttachJob(JobName, DSJ.ERRFATAL)
If NOT(hJob1) Then
Call DSLogFatal("Job Attach Failed: DailyPCJobStatsLdiNew", "JobControl")
Abort
End
Errcode1 = DSSetParam(hJob1, "HASHPATH", HASHPATH)
Errcode1 = DSSetParam(hJob1, "STGPATH", PROJDIR:STGPATH)
Errcode1 = DSSetParam(hJob1, "BATCHNUMBER", BATCHNUMBER)
Errcode1 = DSSetParam(hJob1, "SRCDBNAME", SRCDBNAME)
Errcode1 = DSSetParam(hJob1, "SRCUSERID", SRCUSERID)
Errcode1 = DSSetParam(hJob1, "SRCPWD", SRCPWD)
Errcode1 = DSRunJob(hJob1, DSJ.RUNNORMAL)
Errcode1 = DSWaitForJob(hJob1)
Status = DSGetJobInfo(hJob1, DSJ.JOBSTATUS)
If Status = DSJS.RUNFAILED Or Status = DSJS.CRASHED Then
* Fatal Error - No Return
Call DSLogWarn("Job Failed: DailyPCJobStatsLdiNew", "JobControl")
End
If Errcode then
* Result = JobControlReset(BatchName)
Message = EmailErrorMessage(HASHPATH,BATCHNUMBER)
Reply = DSSendMail("From:":MSG_FROM:"\nTo:":MSG_TO:"\nServer:":SMTPSERVER:"\nSubject:":BatchName:" failed\nBody:":Message:"\n")
Call DSLogFatal("Batch ran failed! (see warnings)", BatchName)
End
*Clear Hash File.
OPENPATH HASHPATH:"/":'hshJobInfo':BATCHNUMBER To hashPath Then
CLEARFILE hashPath ON ERROR Call DSLogInfo("Cannot Clear the Log",BatchName)
End
this is my batch job code....
i need to run xfmDlyCalcTxns_LN,LiveNationRemit_US,LiveNationRemit_CA along with below two jobs....only and only if the records flows through 1st job....
else i have to run only "ldiLNRemitProcessControl" after checking 1st job......
i have modified the code bt its not working.....any reason?
or nay changes i need to do?
Posted: Fri Jan 02, 2009 10:31 am
by chulett
Which bit is the modified bit and what's not working about it?
Posted: Fri Jan 02, 2009 10:38 am
by chulett
nm - revisit the online help for the DSGetLinkInfo syntax, it needs Job Handle + Stage and Link names. DSJ.ME is the handle to the job invoking the call, btw - i.e. your job control.
Posted: Fri Jan 02, 2009 10:41 am
by sagar deshmukh
DSGetLinkInfo syntax i checked....as per that i have modified bold part...
JobName = "srcDlyCalcTxns_LN"
BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
Errcode = RunRestartableJob(JobName,HASHPATH,BATCHNUMBER,"BATCHTIMESTAMP=":BatchStartDts)
If Errcode Then
GOTO ENDJOB
result = DSGetLinkInfo(DSJ.ME,DSJ.ME,DSJ.ME,DSJ.LINKROWCOUNT)
End
If Result <> 0 Then
Posted: Fri Jan 02, 2009 10:49 am
by chulett
Check again. You need to establish a handle to the "srcDlyCalcTxns_LN" job and not use DSJ.ME at all. The 2nd & 3rd arguments are Stage Name and Link Name respectively from the job the handle is attached to. And then detach the handle when you are done with it.
Posted: Fri Jan 02, 2009 11:10 am
by sagar deshmukh
Code: Select all
DEFFUN RunRestartableJob(JobName,HASHPATH,BatchNumber,SKIPPARAM) Calling "DSU.RunRestartableJob"
DEFFUN EmailErrorMessage(HASHPATH,BatchNumber) Calling "DSU.EmailErrorMessage"
DEFFUN JobControlReset(JobList) Calling "DSU.JobControlReset"
DEFFUN RunBatch(JobList, Params) Calling "DSU.RunBatch"
DEFFUN ConvDateYYYYMMDD.DashDeltoMMDDYYYY(DATE) Calling "DSU.ConvDateYYYYMMDD.DashDeltoMMDDYYYY"
DEFFUN GetHashValueByKey(File,Key) Calling "DSU.GetHashValueByKey"
BatchStartDts = DSGetJobInfo(DSJ.ME, DSJ.JOBSTARTTIMESTAMP)
** Run hashCheckControlDTS outside of restart framework - want to run on restart **
JobName = "hashCheckControlDTS"
BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
BatchConfig<1> = BatchName ; * current job name or job run id
BatchConfig<2> = 1 ; * Maximum jobs to have running in parallel (in this batch)
BatchConfig<3> = "S" ; * P = parallel batch, S = serial batch
BatchConfig<4> = "JOB" ; * Job List Type (JOB=running jobs, BATCH=running batches)
BatchConfig<5> = "RUN" ; * run type (RUN=run jobs, RESET=reset jobs, VALIDATE=validate)
CONVERT @AM TO "," IN BatchConfig
Errcode = RunBatch(JobName,BatchConfig)
If Errcode Then
GOTO ENDJOB
End
** End of hashCheckControlDTS ***
JavaBatchDTS = GetHashValueByKey(HASHPATH:"hshCCDTS","")
If Not (ConvDateYYYYMMDD.DashDeltoMMDDYYYY(JavaBatchDTS) = ConvDateYYYYMMDD.DashDeltoMMDDYYYY(BatchStartDts)) Then
*abort job
Call DSLogFatal("Batch ran failed! Invoice Java Batch Job has not completed.", BatchName)
End
JobName = "hashProcessControlDTS"
BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
Errcode = RunRestartableJob(JobName,HASHPATH,BATCHNUMBER,"BATCHTIMESTAMP=":BatchStartDts)
If Errcode Then
GOTO ENDJOB
End
[b] JobName = "srcDlyCalcTxns_LN"
BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
Errcode = RunRestartableJob(JobName,HASHPATH,BATCHNUMBER,"BATCHTIMESTAMP=":BatchStartDts)
If Errcode Then
GOTO ENDJOB
result = DSGetLinkInfo(DSJ.JOBNAME,DSJ.xfmDailyCalcTxns,DSJ.Extract_DailyCalcTxns,DSJ.LINKROWCOUNT)
End[/b]
StgFileName = "LNREMIT"
JobName = "xfmDlyCalcTxns_LN"
BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
Errcode = RunRestartableJob(JobName,HASHPATH,BATCHNUMBER,"BATCHTIMESTAMP=":BatchStartDts)
If Errcode Then
GOTO ENDJOB
End
JobName = "ldiLNRemitProcessControl"
BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
Errcode = RunRestartableJob(JobName,HASHPATH,BATCHNUMBER,"BATCHTIMESTAMP=":BatchStartDts)
If Errcode Then
GOTO ENDJOB
End
ENDJOB:
* Setup DailyPCJobStatsLdiNew, run it, wait for it to finish, and test for success
JobName = "DailyPCJobStatsLdiNew.":BATCHNAME
Result = JobControlReset(JobName)
hJob1 = DSAttachJob(JobName, DSJ.ERRFATAL)
If NOT(hJob1) Then
Call DSLogFatal("Job Attach Failed: DailyPCJobStatsLdiNew", "JobControl")
Abort
End
Errcode1 = DSSetParam(hJob1, "HASHPATH", HASHPATH)
Errcode1 = DSSetParam(hJob1, "STGPATH", PROJDIR:STGPATH)
Errcode1 = DSSetParam(hJob1, "BATCHNUMBER", BATCHNUMBER)
Errcode1 = DSSetParam(hJob1, "SRCDBNAME", SRCDBNAME)
Errcode1 = DSSetParam(hJob1, "SRCUSERID", SRCUSERID)
Errcode1 = DSSetParam(hJob1, "SRCPWD", SRCPWD)
Errcode1 = DSRunJob(hJob1, DSJ.RUNNORMAL)
Errcode1 = DSWaitForJob(hJob1)
Status = DSGetJobInfo(hJob1, DSJ.JOBSTATUS)
If Status = DSJS.RUNFAILED Or Status = DSJS.CRASHED Then
* Fatal Error - No Return
Call DSLogWarn("Job Failed: DailyPCJobStatsLdiNew", "JobControl")
End
If Errcode then
* Result = JobControlReset(BatchName)
Message = EmailErrorMessage(HASHPATH,BATCHNUMBER)
Reply = DSSendMail("From:":MSG_FROM:"\nTo:":MSG_TO:"\nServer:":SMTPSERVER:"\nSubject:":BatchName:" failed\nBody:":Message:"\n")
Call DSLogFatal("Batch ran failed! (see warnings)", BatchName)
End
*Clear Hash File.
OPENPATH HASHPATH:"/":'hshJobInfo':BATCHNUMBER To hashPath Then
CLEARFILE hashPath ON ERROR Call DSLogInfo("Cannot Clear the Log",BatchName)
End
Code: Select all
I have made the code little bit short..can you please tell why its not running now....any suggestion
[/quote]
Posted: Fri Jan 02, 2009 11:20 am
by chulett
First tell us what 'not running' means, what issues you are seeing where. I'm not about to
guess what errors you may or may not be getting, especially with all those custom routines.
![Confused :?](./images/smilies/icon_confused.gif)
Posted: Fri Jan 02, 2009 11:31 am
by sagar deshmukh
following Error is coming:----Batch::LNRemitMaster_exp..JobControl (fatal error from Batch::LNRemitMaster_exp): Batch ran failed! (see warnings)
Code: Select all
Is is not at all calling following part of code...just failing....and dont care abt the routines.....those are restartability routines
JobName = "srcDlyCalcTxns_LN"
BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
Errcode = RunRestartableJob(JobName,HASHPATH,BATCHNUMBER,"BATCHTIMESTAMP=":BatchStartDts)
If Errcode Then
GOTO ENDJOB
result = DSGetLinkInfo(DSJ.JOBNAME,DSJ.xfmDailyCalcTxns,DSJ.Extract_DailyCalcTxns,DSJ.LINKROWCOUNT)
End
Posted: Fri Jan 02, 2009 11:45 am
by chulett
No idea what your error has to do with the code you posted but your call to DSGetLinkInfo will
never be executed:
Code: Select all
If Errcode Then
GOTO ENDJOB
result = DSGetLinkInfo(DSJ.JOBNAME,DSJ.xfmDailyCalcTxns,DSJ.Extract_DailyCalcTxns,DSJ.LINKROWCOUNT)
End
Does that formatting help show you why?
Posted: Fri Jan 02, 2009 11:58 am
by sagar deshmukh
Sorry for that stupidity...
Code: Select all
JobName = "srcDlyCalcTxns_LN"
BatchName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)
result = DSGetLinkInfo(DSJ.JOBNAME,DSJ.ME,DSJ.ME,DSJ.LINKROWCOUNT)
Errcode = RunRestartableJob(JobName,HASHPATH,BATCHNUMBER,"BATCHTIMESTAMP=":BatchStartDts)
If Errcode Then
GOTO ENDJOB
End
Can you tell how to use This "RESULT" value for skiping below jobs?