Job to Reset Jobs

Archive of postings to DataStageUsers@Oliver.com. This forum intended only as a reference and cannot be posted to.

Moderators: chulett, rschirm

Locked
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Job to Reset Jobs

Post by admin »

Im curious if anyone here can help me out with this...

Im hoping I can find a way to write a batch job that could go thru all current jobs in a project and identify which ones are in an aborted state. Once those are encountered, Id like to email myself a list of the jobs that it found in that state (which I can handle) but what I dont know how to do is issue a reset command to the job.

Im just trying to avoid situations where, for whatever reason, a job or series of jobs aborts and then does not run until someone goes in and manually resets the status. Im under the impression that 5.1 can be setup to handle this situation, but Im not sure when Ill be able to move up as were running on a Compaq Tru64 cluster which seems to be dead last in the port list. Current enviroment is 4.2.1, if that makes any difference in the solution.

Id also be curious if people out there have handled this problem in another fashion... this is just an approach that I thought would be workable.

Thanks!

-craig
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Hi craig,
you can test job status code with DSGetJobInfo(JobHandle,InfoType) then you can use DSRunJob(JobHAndel,RunMode) to reset the job. Check help online for more info.

I hope it will help u.

Riccardo

----- Original Message -----
From:
To:
Sent: Friday, November 16, 2001 5:18 PM
Subject: Job to Reset Jobs


>
> Im curious if anyone here can help me out with this...
>
> Im hoping I can find a way to write a batch job that could go thru
> all current jobs in a project and identify which ones are in an
> aborted state. Once those are encountered, Id like to email myself a
> list of the jobs that it found in that state (which I can handle) but
> what I dont know how to do is issue a reset command to the job.
>
> Im just trying to avoid situations where, for whatever reason, a job
> or series of jobs aborts and then does not run until someone goes in
> and manually resets the status. Im under the impression that 5.1 can
> be
setup
> to handle this situation, but Im not sure when Ill be able to move
> up as were running on a Compaq Tru64 cluster which seems to be dead
> last in the port list. Current enviroment is 4.2.1, if that makes
> any difference in the solution.
>
> Id also be curious if people out there have handled this problem in
> another fashion... this is just an approach that I thought would be
> workable.
>
> Thanks!
>
> -craig
>
>
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Hi Craig...

The calls that Riccardo mentions are indeed what you need and are described also in the DataStage Development Kit chapter of the DS Server Developers guide, and have been part of DS since 3.0, so will be in your version of DS.

They are actually part of a command set that can be used in DataStage Basic (within Job Control, for instance), within your own C program, if you are so inclined, or within an operating system shell (they are exposed also in a
command line form). So you can get pretty creative with ways to do the
dynamic resets that you are looking for. All of those options are documented, and Im sure have been used by members of this thread in a lot of ways.

Ernie





-----Original Message-----
From: Riccardo Tani [mailto:rtani@datamat.it]
Sent: Friday, November 16, 2001 11:56 AM
To: datastage-users@oliver.com
Subject: Re: Job to Reset Jobs


Hi craig,
you can test job status code with DSGetJobInfo(JobHandle,InfoType) then you can use DSRunJob(JobHAndel,RunMode) to reset the job. Check help online for more info.

I hope it will help u.

Riccardo

----- Original Message -----
From:
To:
Sent: Friday, November 16, 2001 5:18 PM
Subject: Job to Reset Jobs


>
> Im curious if anyone here can help me out with this...
>
> Im hoping I can find a way to write a batch job that could go thru
> all current jobs in a project and identify which ones are in an
> aborted state. Once those are encountered, Id like to email myself a
> list of the jobs that it found in that state (which I can handle) but
> what I dont know how to do is issue a reset command to the job.
>
> Im just trying to avoid situations where, for whatever reason, a job
> or series of jobs aborts and then does not run until someone goes in
> and manually resets the status. Im under the impression that 5.1 can
> be
setup
> to handle this situation, but Im not sure when Ill be able to move
> up as were running on a Compaq Tru64 cluster which seems to be dead
> last in the port list. Current enviroment is 4.2.1, if that makes
> any difference in the solution.
>
> Id also be curious if people out there have handled this problem in
> another fashion... this is just an approach that I thought would be
> workable.
>
> Thanks!
>
> -craig
>
>
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Thanks, guys, for the very helpful information. I was aware of the routines, but not that I could use them to reset jobs. I also saw that there is a "DSGetProjectInfo" routine where I can request a list of all jobs in the project, which looks like a good way to get all of the jobnames to loop thru using something like FIELD to parse the list.

As an alternative, can someone give me the syntax to read thru DS_JOBS retrieving the Job Name? I know I can say "LIST DS_JOBS" and get everything, but I cant seem to puzzle out the syntax to just select the job name.

-craig
x8885




ernie.ostic@ascentialso
ftware.com To: datastage-users@oliver.com
cc:
11/16/01 11:43 AM Subject: RE: Job to Reset Jobs
Please respond to
datastage-users





Hi Craig...

The calls that Riccardo mentions are indeed what you need and are described also in the DataStage Development Kit chapter of the DS Server Developers guide, and have been part of DS since 3.0, so will be in your version of DS.

They are actually part of a command set that can be used in DataStage Basic (within Job Control, for instance), within your own C program, if you are so inclined, or within an operating system shell (they are exposed also in a
command line form). So you can get pretty creative with ways to do the
dynamic resets that you are looking for. All of those options are documented, and Im sure have been used by members of this thread in a lot of ways.

Ernie





-----Original Message-----
From: Riccardo Tani [mailto:rtani@datamat.it]
Sent: Friday, November 16, 2001 11:56 AM
To: datastage-users@oliver.com
Subject: Re: Job to Reset Jobs


Hi craig,
you can test job status code with DSGetJobInfo(JobHandle,InfoType) then you can use DSRunJob(JobHAndel,RunMode) to reset the job. Check help online for more info.

I hope it will help u.

Riccardo

----- Original Message -----
From:
To:
Sent: Friday, November 16, 2001 5:18 PM
Subject: Job to Reset Jobs


>
> Im curious if anyone here can help me out with this...
>
> Im hoping I can find a way to write a batch job that could go thru
> all current jobs in a project and identify which ones are in an
> aborted
state.
> Once those are encountered, Id like to email myself a list of the
> jobs that it found in that state (which I can handle) but what I dont
> know
how
> to do is issue a reset command to the job.
>
> Im just trying to avoid situations where, for whatever reason, a job
> or series of jobs aborts and then does not run until someone goes in
> and manually resets the status. Im under the impression that 5.1 can
> be
setup
> to handle this situation, but Im not sure when Ill be able to move
> up
as
> were running on a Compaq Tru64 cluster which seems to be dead last in
the
> port list. Current enviroment is 4.2.1, if that makes any
> difference in the solution.
>
> Id also be curious if people out there have handled this problem in
> another fashion... this is just an approach that I thought would be
> workable.
>
> Thanks!
>
> -craig
>
>
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Craig asked, "As an alternative, can someone give me the syntax to read thru DS_JOBS
> retrieving the Job Name?"
>
Sure. At TCL in Telnet you can say, "LIST DS_JOBS NAME ID.SUP" and it will list just the jobnames. The only problem is any jobnames longer than what the data dictionary has for the length within NAME. We have some long jobnames and theyll wrap. But usually its few and far between so editing the results will clean it up.

Sincerely,
Brad Vincent
Data Warehousing with a "health"-y spin
Compuware
c/o The Detroit Medical Center
(313) 966-2176
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

One of the techniques taught on the "Programming with DataStage BASIC" class is not executing a job from a job control routing unless the job is in a runnable state, which may involve resetting it first if it is in a resettable state.
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Open "DS_JOBS" To DSJOBS.fvar
Then
* Process all keys from DS_JOBS file
Select DSJOBS.fvar
Loop
While ReadNext JobName
* Dont process control records from DS_JOBS
If JobName Matches "1A0X"
Then GoSub ProcessJob
Repeat
End
Else
Message = Unable to open "DS_JOBS" file.
Call DSLogFatal(Message, "Controller")
End

Your ProcessJob routine can do whatever you need to.

-----Original Message-----
From: Craig_Hulett@rsausa.com [mailto:Craig_Hulett@rsausa.com]
Sent: Saturday, 17 November 2001 06:46
To: datastage-users@oliver.com
Subject: RE: Job to Reset Jobs



Thanks, guys, for the very helpful information. I was aware of the routines, but not that I could use them to reset jobs. I also saw that there is a "DSGetProjectInfo" routine where I can request a list of all jobs in the project, which looks like a good way to get all of the jobnames to loop thru using something like FIELD to parse the list.

As an alternative, can someone give me the syntax to read thru DS_JOBS retrieving the Job Name? I know I can say "LIST DS_JOBS" and get everything, but I cant seem to puzzle out the syntax to just select the job name.

-craig
x8885




ernie.ostic@ascentialso
ftware.com To:
datastage-users@oliver.com
cc:
11/16/01 11:43 AM Subject: RE: Job to
Reset Jobs
Please respond to
datastage-users





Hi Craig...

The calls that Riccardo mentions are indeed what you need and are described also in the DataStage Development Kit chapter of the DS Server Developers guide, and have been part of DS since 3.0, so will be in your version of DS.

They are actually part of a command set that can be used in DataStage Basic (within Job Control, for instance), within your own C program, if you are so inclined, or within an operating system shell (they are exposed also in a
command line form). So you can get pretty creative with ways to do the
dynamic resets that you are looking for. All of those options are documented, and Im sure have been used by members of this thread in a lot of ways.

Ernie





-----Original Message-----
From: Riccardo Tani [mailto:rtani@datamat.it]
Sent: Friday, November 16, 2001 11:56 AM
To: datastage-users@oliver.com
Subject: Re: Job to Reset Jobs


Hi craig,
you can test job status code with DSGetJobInfo(JobHandle,InfoType) then you can use DSRunJob(JobHAndel,RunMode) to reset the job. Check help online for more info.

I hope it will help u.

Riccardo

----- Original Message -----
From:
To:
Sent: Friday, November 16, 2001 5:18 PM
Subject: Job to Reset Jobs


>
> Im curious if anyone here can help me out with this...
>
> Im hoping I can find a way to write a batch job that could go thru
> all current jobs in a project and identify which ones are in an
> aborted
state.
> Once those are encountered, Id like to email myself a list of the
> jobs that it found in that state (which I can handle) but what I dont
> know
how
> to do is issue a reset command to the job.
>
> Im just trying to avoid situations where, for whatever reason, a job
> or series of jobs aborts and then does not run until someone goes in
> and manually resets the status. Im under the impression that 5.1 can
> be
setup
> to handle this situation, but Im not sure when Ill be able to move
> up
as
> were running on a Compaq Tru64 cluster which seems to be dead last in
the
> port list. Current enviroment is 4.2.1, if that makes any
> difference in the solution.
>
> Id also be curious if people out there have handled this problem in
> another fashion... this is just an approach that I thought would be
> workable.
>
> Thanks!
>
> -craig
>
>
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Hi Craig,
Here is the control loop that will read all the jobs from the DS_JOBS file and let you process each one to checK the status and reset if necessary.
Just place the enclosed code in the JOB CONTROL section of the Job properties section. The place for your code is marked.

I hope this helps you.

Good Luck,
Frank Martyn

----------------------------------------------------------------------------
--------------------
* obtain a list of all jobs - they all start with an alphabetic character
ErrCode =DSLogEvent(DSJ.ME,DSJ.LOGINFO ,"Going into DS_JOBS for the job names")
cmd = "SSELECT DS_JOBS WITH NAME LIKE 1A0X"
execute cmd capturing trash ; * Put list where we can get it,
also grab extraneous output
readlist AllTheJobs else
call DSLogFatal("Unable to read the DS_JOBS
table",DSGetJobInfo(DSJ.ME,DSJ.JobName))
end

NumberOfJobs = dcount(AllTheJobs,@FM)
ErrCode =DSLogEvent(DSJ.ME,DSJ.LOGINFO ,"Found ":NumberOfJobs:" Jobs to process")
* Now the Job names are where we can get at them


* heres where we get each job name
loop
remove JobName from AllTheJobs setting Delim
if JobName = "" then continue ; * just in case there are no jobs
*
ErrCode =DSLogEvent(DSJ.ME,DSJ.LOGINFO ,"Processing ":JobName)
until Delim = 0
repeat


----------------------------------------------------------------------------
--------------------

-----Original Message-----
From: Craig_Hulett@rsausa.com [mailto:Craig_Hulett@rsausa.com]
Sent: Friday, November 16, 2001 2:46 PM
To: datastage-users@oliver.com
Subject: RE: Job to Reset Jobs



Thanks, guys, for the very helpful information. I was aware of the routines, but not that I could use them to reset jobs. I also saw that there is a "DSGetProjectInfo" routine where I can request a list of all jobs in the project, which looks like a good way to get all of the jobnames to loop thru using something like FIELD to parse the list.

As an alternative, can someone give me the syntax to read thru DS_JOBS retrieving the Job Name? I know I can say "LIST DS_JOBS" and get everything, but I cant seem to puzzle out the syntax to just select the job name.

-craig
x8885




ernie.ostic@ascentialso
ftware.com To:
datastage-users@oliver.com
cc:
11/16/01 11:43 AM Subject: RE: Job to
Reset Jobs
Please respond to
datastage-users





Hi Craig...

The calls that Riccardo mentions are indeed what you need and are described also in the DataStage Development Kit chapter of the DS Server Developers guide, and have been part of DS since 3.0, so will be in your version of DS.

They are actually part of a command set that can be used in DataStage Basic (within Job Control, for instance), within your own C program, if you are so inclined, or within an operating system shell (they are exposed also in a
command line form). So you can get pretty creative with ways to do the
dynamic resets that you are looking for. All of those options are documented, and Im sure have been used by members of this thread in a lot of ways.

Ernie





-----Original Message-----
From: Riccardo Tani [mailto:rtani@datamat.it]
Sent: Friday, November 16, 2001 11:56 AM
To: datastage-users@oliver.com
Subject: Re: Job to Reset Jobs


Hi craig,
you can test job status code with DSGetJobInfo(JobHandle,InfoType) then you can use DSRunJob(JobHAndel,RunMode) to reset the job. Check help online for more info.

I hope it will help u.

Riccardo

----- Original Message -----
From:
To:
Sent: Friday, November 16, 2001 5:18 PM
Subject: Job to Reset Jobs


>
> Im curious if anyone here can help me out with this...
>
> Im hoping I can find a way to write a batch job that could go thru
> all current jobs in a project and identify which ones are in an
> aborted
state.
> Once those are encountered, Id like to email myself a list of the
> jobs that it found in that state (which I can handle) but what I dont
> know
how
> to do is issue a reset command to the job.
>
> Im just trying to avoid situations where, for whatever reason, a job
> or series of jobs aborts and then does not run until someone goes in
> and manually resets the status. Im under the impression that 5.1 can
> be
setup
> to handle this situation, but Im not sure when Ill be able to move
> up
as
> were running on a Compaq Tru64 cluster which seems to be dead last in
the
> port list. Current enviroment is 4.2.1, if that makes any
> difference in the solution.
>
> Id also be curious if people out there have handled this problem in
> another fashion... this is just an approach that I thought would be
> workable.
>
> Thanks!
>
> -craig
>
>
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Brad,
There is a simple solution to the wrap problem. Modify your TCL command to read - "LIST DS_JOBS NAME FMT 80L ID.SUP" the FMT key word tell it how to handle the width and justification. in the example it will give the data in a column 80 characters wide, left justified. It you want to right justify just use "R" instead of "T". Note the width and justification is enclosed within quotes.

Regards,
Frank Martyn

-----Original Message-----
From: Vincent, Brad [mailto:BVincent@dmc.org]
Sent: Friday, November 16, 2001 3:00 PM
To: datastage-users@oliver.com
Subject: RE: Job to Reset Jobs


Craig asked, "As an alternative, can someone give me the syntax to read thru DS_JOBS
> retrieving the Job Name?"
>
Sure. At TCL in Telnet you can say, "LIST DS_JOBS NAME ID.SUP" and it will list just the jobnames. The only problem is any jobnames longer than what the data dictionary has for the length within NAME. We have some long jobnames and theyll wrap. But usually its few and far between so editing the results will clean it up.

Sincerely,
Brad Vincent
Data Warehousing with a "health"-y spin
Compuware
c/o The Detroit Medical Center
(313) 966-2176
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

There IS a simple solution - dont use TCL!
See the code fragment I posted earlier, for example.
This also avoids the overhead of invoking the command parser and all that that entails.

Curious why an earlier poster used SSELECT rather than SELECT; if youre going to process all records (job names) then sorting is wasted effort.

-----Original Message-----
From: Frank Martyn [mailto:frankm@mediaone.net]
Sent: Saturday, 17 November 2001 13:09
To: datastage-users@oliver.com
Subject: RE: Job to Reset Jobs


Brad,
There is a simple solution to the wrap problem. Modify your TCL command to read -
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

The simple solution is to reply in the mode the question was asked. Brad was using TCL and asked for a solution to the TCL problem. I was trying to help him, not make him work harder. (In the context of the "RESET JOBS" question the overhead of the command parser is not a major factor).
The purpose of the SSELECT over SELECT is to order the job by name for human tracking if desired. The performance hit of the SSELECT vs. SELECT is not measurable in a file as small as the DS_JOBS will be in any development or production project.

PS RAY, - if you really dont know why someone suggests something you didnt think of, then contact them directly. That way there is no confusion by people who might not know better whats happening.

Regards,
Frank


-----Original Message-----
From: Ray Wurlod [mailto:ray.wurlod@Informix.Com]
Sent: Friday, November 16, 2001 9:21 PM
To: datastage-users@oliver.com
Subject: RE: Job to Reset Jobs


There IS a simple solution - dont use TCL!
See the code fragment I posted earlier, for example.
This also avoids the overhead of invoking the command parser and all that that entails.

Curious why an earlier poster used SSELECT rather than SELECT; if youre going to process all records (job names) then sorting is wasted effort.

-----Original Message-----
From: Frank Martyn [mailto:frankm@mediaone.net]
Sent: Saturday, 17 November 2001 13:09
To: datastage-users@oliver.com
Subject: RE: Job to Reset Jobs


Brad,
There is a simple solution to the wrap problem. Modify your TCL command to read -
Locked