job Parameters for Sequencer Jobs
Moderators: chulett, rschirm, roy
job Parameters for Sequencer Jobs
Is there any way i can make my Sequencer job to read the JobParameters from a Parameter File?
I have 7 jobs to be put in JobSequencer.
All these jobs read the parameters from a Parameter File.
Instead of manually setting the Parameter values in JobSequencer, can i make the Sequncer job to read from a Parameter File?
Thanks in Advance.
I have 7 jobs to be put in JobSequencer.
All these jobs read the parameters from a Parameter File.
Instead of manually setting the Parameter values in JobSequencer, can i make the Sequncer job to read from a Parameter File?
Thanks in Advance.
-
- Premium Member
- Posts: 483
- Joined: Thu Jun 12, 2003 4:47 pm
- Location: St. Louis, Missouri USA
Kenneth,
I think that this answers Poorna_76's question, but since you said that the Sequencer can't do this, I'm not sure if I understood his question correctly. Can you review the scenario I describe below and let me know if I'm off base? Thanks!
Poorna_76
We do this with most of our job sequencers.
We have a parameter file formatted as Name=Value. I wrote a routine that takes a file name and a parameter name and returns the value of the parameter from the file.
We use a Routine Activity stage to execute this routine and retrieve the value of the parameter from the file... One Routine Activity stage for each Job Parameter to retrieve. I normally name the Routine Activity stage the same as the Job Parameter that I'm retrieving. So, if I have a Job Parameter called DB_NAME, I'll have a Routine Activity named DB_NAME.
In the Job Activity stage, where the Job Parameters are set, put the Routine Activity stage name followed by .$ReturnValue. So, in the Job Activity stage for the DB_NAME Job Parameter "Value Expression", I put DB_NAME.$ReturnValue . At runtime, the Routine Activity fetches the Job Parameter value from the file and feeds it to the Job Activity.
I hope this makes sense. If not, please ask more questions and I'll do my best to explain.
tony
I think that this answers Poorna_76's question, but since you said that the Sequencer can't do this, I'm not sure if I understood his question correctly. Can you review the scenario I describe below and let me know if I'm off base? Thanks!
Poorna_76
We do this with most of our job sequencers.
We have a parameter file formatted as Name=Value. I wrote a routine that takes a file name and a parameter name and returns the value of the parameter from the file.
We use a Routine Activity stage to execute this routine and retrieve the value of the parameter from the file... One Routine Activity stage for each Job Parameter to retrieve. I normally name the Routine Activity stage the same as the Job Parameter that I'm retrieving. So, if I have a Job Parameter called DB_NAME, I'll have a Routine Activity named DB_NAME.
In the Job Activity stage, where the Job Parameters are set, put the Routine Activity stage name followed by .$ReturnValue. So, in the Job Activity stage for the DB_NAME Job Parameter "Value Expression", I put DB_NAME.$ReturnValue . At runtime, the Routine Activity fetches the Job Parameter value from the file and feeds it to the Job Activity.
I hope this makes sense. If not, please ask more questions and I'll do my best to explain.
tony
Last edited by tonystark622 on Mon Oct 11, 2004 2:36 pm, edited 1 time in total.
The Sequencer has no capability to feed parameters from a file into a job via the Job Activity stage functionality.
However, if you write a function (which Tony has) which reads a file, parses the contents, and returns the values into fixed return codes, you can send those results into the job parameter values in the Job Activity stage.
BUT, there is no functionality in the Sequencer to do this out of the box. To quote Ray:
However, if you write a function (which Tony has) which reads a file, parses the contents, and returns the values into fixed return codes, you can send those results into the job parameter values in the Job Activity stage.
BUT, there is no functionality in the Sequencer to do this out of the box. To quote Ray:
"You must roll your own."
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Also consider the scalability of a RoutineActivity stage per job parameter. If you have 20+ parameters in a job (paths, credentials, dates, etc), then this becomes an nasty job.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
-
- Premium Member
- Posts: 483
- Joined: Thu Jun 12, 2003 4:47 pm
- Location: St. Louis, Missouri USA
Yes it does, no argument from me. I wish we could stuff all these in a container, but there is no container stage in Sequencer jobsAlso consider the scalability of a RoutineActivity stage per job parameter. If you have 20+ parameters in a job (paths, credentials, dates, etc), then this becomes an nasty job.
Thanks for reviewing my email, Kenneth.
Tony
Ken
I friend of mine Peter Lai invented this routine which you can call as routine activity and from that point on it will change the value of one parameter. I have heavily modified it. Peter's original version would read all the parameters in a sequential file and set any or all of them. I just wanted to set them one at a time. I first have a job which will update a specific hash file named MaxHashKeyHash. The key to this hash file is the parameter name. I wanted to select the max key from a hash file which has the original source keys and then I only need to get keys greater than this from my dimension to build my crossref hash file. This saves reading millions of rows and allows us to sort of treat a hash file like it is persistant but rebuild it if necessary.
Anyway here is the code. Thanks Peter you are the best. This should eliminate the need for batch jobs.
I friend of mine Peter Lai invented this routine which you can call as routine activity and from that point on it will change the value of one parameter. I have heavily modified it. Peter's original version would read all the parameters in a sequential file and set any or all of them. I just wanted to set them one at a time. I first have a job which will update a specific hash file named MaxHashKeyHash. The key to this hash file is the parameter name. I wanted to select the max key from a hash file which has the original source keys and then I only need to get keys greater than this from my dimension to build my crossref hash file. This saves reading millions of rows and allows us to sort of treat a hash file like it is persistant but rebuild it if necessary.
Anyway here is the code. Thanks Peter you are the best. This should eliminate the need for batch jobs.
Code: Select all
* ------------------------------------------------------------
* KgdSetLastKeyParam(ParamName, KeyName, Debug)
* Decription: Set parameters from a file
* Written by: Kim Duke
* Copied From: Peter Lai
* Notes:
* Routine set one parameter value from a hash file.
* ------------------------------------------------------------
$INCLUDE DSINCLUDE JOBCONTROL.H
$INCLUDE DSINCLUDE DSD_STAGE.H
$INCLUDE DSINCLUDE DSD.H
$INCLUDE DSINCLUDE DSD_RTSTATUS.H
*----------------------*
* Initialize Variables *
*----------------------*
Ans = 0 ; * 0=Successful(default), 1=Error occured
ProgName = "Function: KgdSetLastKeyParam"
*------*
* Main *
*------*
HashFileName = "MaxHashKeyHash"
Open HashFileName TO HashFile Else
Ans = 1
Call DSLogWarn('HashFile "':HashFileName:'" does not exist.',ProgName)
goto ProgramEnd
End
* -----------------------------------------------------------------
* Read the named record from the file.
* -----------------------------------------------------------------
Readv ParamValue From HashFile, KeyName, 1 Else
Ans = 1
Call DSLogWarn('"':HashFileName:'", "':KeyName:'" does not exist.',ProgName)
goto ProgramEnd
* ParamValue = 0
End
ParamList = ParamName : @FM : ParamValue
If Debug then Call DSLogInfo('Parameter Name: ':ParamName,ProgName)
If Debug then Call DSLogInfo('Parameter Value: ':ParamValue,ProgName)
JobName = Field(STAGECOM.NAME,'.',1,2)
if Debug then Call DSLogInfo('STAGECOM.NAME=':STAGECOM.NAME,JobName:" ":ProgName)
StatusFileName = FileInfo(DSRTCOM.RTSTATUS.FVAR,1)
StatusId = JobName:'.':STAGECOM.WAVE.NUM
if Debug then Call DSLogInfo('StatusFileName=':StatusFileName,JobName:" ":ProgName)
* Get all job parameter names and values
JobParamList = STAGECOM.JOB.CONFIG<CONTAINER.PARAM.NAMES>
if Debug then Call DSLogInfo('Job Parameter: ':JobParamList,JobName:" ":ProgName)
IF JobParamList <> "" THEN
LOCATE ParamName IN JobParamList<1> SETTING JobParamPos THEN
JobParamName = JobParamList<1,JobParamPos>
JobParamValue = ParamValue
STAGECOM.JOB.STATUS<JOB.PARAM.VALUES,JobParamPos> = JobParamValue
Call DSLogInfo('Parameter: ':JobParamName:' = ':JobParamValue,JobName:" ":ProgName)
END ELSE
Ans = 1
Call DSLogWarn('Parameter "':ParamName:'" does not exist in parameters.',ProgName)
goto ProgramEnd
END
END
ProgramEnd:
print Ans
Mamu Kim
job Parameters for Sequencer Jobs
Kenneth,
To be more clear about my question:
I have 7 jobs to be executed through Sequncer.
Each job takes more than 15 parameters.
Right now i am creating a seperate JobControl job,
which will set the parameters and run my jobs from there.
DSAttach..
DSRunJob..
DSWait....
I thought there might be some thing similar, in SequencerJobs too.
Now i know Sequencer lacks this functionality.
Thanks for all.
To be more clear about my question:
I have 7 jobs to be executed through Sequncer.
Each job takes more than 15 parameters.
Right now i am creating a seperate JobControl job,
which will set the parameters and run my jobs from there.
DSAttach..
DSRunJob..
DSWait....
I thought there might be some thing similar, in SequencerJobs too.
Now i know Sequencer lacks this functionality.
Thanks for all.
tonystark622 wrote:Kenneth,
I think that this answers Poorna_76's question, but since you said that the Sequencer can't do this, I'm not sure if I understood his question correctly. Can you review the scenario I describe below and let me know if I'm off base? Thanks!
Poorna_76
We do this with most of our job sequencers.
We have a parameter file formatted as Name=Value. I wrote a routine that takes a file name and a parameter name and returns the value of the parameter from the file.
We use a Routine Activity stage to execute this routine and retrieve the value of the parameter from the file... One Routine Activity stage for each Job Parameter to retrieve. I normally name the Routine Activity stage the same as the Job Parameter that I'm retrieving. So, if I have a Job Parameter called DB_NAME, I'll have a Routine Activity named DB_NAME.
In the Job Activity stage, where the Job Parameters are set, put the Routine Activity stage name followed by .$ReturnValue. So, in the Job Activity stage for the DB_NAME Job Parameter "Value Expression", I put DB_NAME.$ReturnValue . At runtime, the Routine Activity fetches the Job Parameter value from the file and feeds it to the Job Activity.
I hope this makes sense. If not, please ask more questions and I'll do my best to explain.
tony
Depending on the nature of what you are doing, they are trying to address situations like this. If you have 7.5 there are new Sequencer stages to control "looping" within a Sequencer job and from what I've read the LoopStart stage has some ability to pull information from a delimited source.
I expect functionality in this area to be augmented in future versions.
I expect functionality in this area to be augmented in future versions.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
You do not need the batch job when you use the routine posted above. Your first job could read from a sequence job and write to this hash file. Next is the above routine activity. You could call it 15 times or modify it to send it 15 paramaters or hard code the parameter names. Next have all your jobs wait on this one job and one routine activity. Then start all your jobs.
Peter has one routine that does step one and step two.
This works try it. It is very cool. This will change the values of the parameters in the current sequence.
Peter has one routine that does step one and step two.
Code: Select all
ParamSeqToHash -> LoadParamFromHash -> Wait1 -> StartAllMyJobsHere
Last edited by kduke on Tue Oct 12, 2004 9:10 am, edited 1 time in total.
Mamu Kim
-
- Participant
- Posts: 214
- Joined: Mon Feb 23, 2004 2:10 am
- Location: Dublin, Ireland
- Contact:
Hi All,
well, I have never liked the job scheduler language and the poor dependency management.....
So, I just went the 'easy' way for my own testing and wrote a command scheduler and a DataStage job submission utility for most of my testing....the job submission utility allows the presentation of parameters from any ODBC data source.
These utilities can drive higher utilisation of the machine by intelligently managing dependencies between groups of jobs without the need to updating any code on a restart.....
I don't use it in production yet, but if anyone wants a really good scheduler on win2000 that can also schedule DS jobs feel free to get it from my web page (source code and all) or contact me.....if they get used enough and are reliable enough there is no reason why they could not be used in production environments...
well, I have never liked the job scheduler language and the poor dependency management.....
So, I just went the 'easy' way for my own testing and wrote a command scheduler and a DataStage job submission utility for most of my testing....the job submission utility allows the presentation of parameters from any ODBC data source.
These utilities can drive higher utilisation of the machine by intelligently managing dependencies between groups of jobs without the need to updating any code on a restart.....
I don't use it in production yet, but if anyone wants a really good scheduler on win2000 that can also schedule DS jobs feel free to get it from my web page (source code and all) or contact me.....if they get used enough and are reliable enough there is no reason why they could not be used in production environments...
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
I wrote one of these that's been in production for nearly two years. It uses BCI functions to access an ODBC database (Oracle 7 as it happens). The tricky part was an arbitrarily deep level of nesting in the dependencies.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 483
- Joined: Thu Jun 12, 2003 4:47 pm
- Location: St. Louis, Missouri USA