Can the DataStage jobs back-up be automated?

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Shree0410
Participant
Posts: 70
Joined: Tue Nov 29, 2005 7:25 pm

Can the DataStage jobs back-up be automated?

Post by Shree0410 »

Hi all,

I have to take the back-up of all DataStage jobs from the project repository every week, this is a requirement of the client.

I can export the whole project at one shot, but the client want us to export by folder. This takes up more than an hour to carefully export all the jobs every monday.

Can anyone tell me if this can be scheduled to be done every monday or if this can be automated??

Thanks Guys!
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

Do a search for DataStageBackup.bat. This has been covered a lot. This is a script I downloaded from ADN which is no longer available. I have modifed it slightly. It is available on my tips page.
Mamu Kim
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

By the way I would backup develment every night. I would backup production once a week.

We usually backup at about 2 in the morning.
Mamu Kim
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Are you being paid by the hour? :lol:

Seriously, though, yes it can be automated; the script to which Kim referred is a good starting point.

An alternative - and maybe easier - is to back up the entire project then use a DSX cutter to create separate per-category DSX files. Do they require routines, table definitions, etc., also to be backed up as separate folders?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Harini
Premium Member
Premium Member
Posts: 80
Joined: Tue Mar 16, 2010 1:32 am

Re: Can the DataStage jobs back-up be automated?

Post by Harini »

Hi all,

Is there a Unix shell scripting solution to acheive this?

Thanks


Shree0410 wrote:Hi all,

I have to take the back-up of all DataStage jobs from the project repository every week, this is a requirement of the client.

I can export the whole project at one shot, but the client want us to export by folder. This takes up more than an hour to carefully export all the jobs every monday.

Can anyone tell me if this can be scheduled to be done every monday or if this can be automated??

Thanks Guys!
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

No because this kind of backup needs to happen on the client, hence the Windows batch file. There are more options available now if you are on the 8.x release, however.
Last edited by chulett on Mon Jun 21, 2010 7:49 am, edited 1 time in total.
-craig

"You can never have too many knives" -- Logan Nine Fingers
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

In release 8.1 after either the istool patch or Fix Pack 1 you can then use the istool command to backup your system. While it has the benefit of being very fast (it is a server-based command line backup) it has one drawback, namely that it cannot do a granular (item-specific) restore. To get a single item back you need to restore the image to a new project, then copy over the one item you need to restore. Of course with IS Manager you can just drag and drop that one item, which is easy enough.

I have requested a more granular restore mechanism from the product manager - they are looking into it.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

The scripts I give away are all Windows based so far.
Mamu Kim
Harini
Premium Member
Premium Member
Posts: 80
Joined: Tue Mar 16, 2010 1:32 am

Post by Harini »

Thank you all. Yes i use 8.1, but am not sure whether istool patch is installed. I would give it a try if we are able to get it.

From a windows environment, I believe the following command would do the export for me if it is whole project.

C:\IBM\InformationServer\Clients\Classic\dscmdexport /D=servername /U=username /P=password /H=servername projectname D:\bkp1.dsx

If it is a spefic folder from which i have to export the jobs then, Kim's solution would do the job for me, isn't it?

Thanks
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

No, it's either one job or all jobs. You'll need another solution, perhaps something like the DSX Cutter posted here or an iterative loop, for category exports.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Harini
Premium Member
Premium Member
Posts: 80
Joined: Tue Mar 16, 2010 1:32 am

Post by Harini »

chulett wrote:No, it's either one job or all jobs. You'll need another solution, perhaps something like the DSX Cutter posted here or an iterative loop, for category exports. ...
Craig,
Iterating the loop would give me a seperate dsx file or each job. But i would want all the jobs as a single dsx file. Is it possible to do that? Else i shall convey to my Lead that it is not possible.

Thanks
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Harini wrote:But i would want all the jobs as a single dsx file. Is it possible to do that?
It is not possible. You could construct a single DSX file out of the individual job DSX files, which would involve removing the individual files' headers and creating a header for the combined file.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Harini
Premium Member
Premium Member
Posts: 80
Joined: Tue Mar 16, 2010 1:32 am

Post by Harini »

ray.wurlod wrote:
Harini wrote:But i would want all the jobs as a single dsx file. Is it possible to do that?
It is not possible. You could construct a single DSX file out of the individual job DSX files, w ...[/quo

:( It's a premium content..
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Easily remedied. Here, for example.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

It certainly is possible to simply append all of the individual exports into a single dsx without the need to remove all of the 'extra' header information. I haven't done this but seem to remember Uncle Kim does (or has) without issue. Would be simple enough to test, that's for sure.
Last edited by chulett on Thu Jun 24, 2010 5:54 am, edited 1 time in total.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply