Deploy DataStage over Microsoft Cluster

asaf_arbely · Post by **asaf_arbely** » Mon Jul 30, 2007 7:38 am

Hi there,

A customer of mine wishes to deploy DS in production environment over MS Cluster.

Does anybody know if it is possible? Did anyone try it? And maybe most interesting, does anybody know if it is NOT allowed?

Thanks in advance,
Asaf

ray.wurlod · Post by **ray.wurlod** » Mon Jul 30, 2007 3:28 pm

Can you be more precise about what is meant by "deploy"?

In server edition DataStage server can only be on one machine; processing will not be distributed to other machines.

But, in general, files and databases on other machines can be accessed.

No use will be made of the cluster architecture by DataStage server edition.

asaf_arbely · Post by **asaf_arbely** » Tue Jul 31, 2007 2:37 am

Hi Ray,

My customer wishes to have a failover capabilities for its night operators, so in case of a hardware failure, a secondary DS server becomes online. Obviously, the single job that was active during the failure must have cleanup capabilities (other than reset) so it can be lounched again.

The MS cluster provides the same virtual IP for both, primary and secondary DS server. Furthermore, all storage that is used by the DS server is a replicated external EMC, and the same apply to the Oracle repository which is located on other machine and as well is clustered (with virtual IP).

I expect that when the secondary DS server is up and running, it will be as if the primary server was crushed and reloaded. The operator can access via CtrlM the same IP and relounch the failed job.

Is it reasonable?

Please refer to a post made by chulett as a reply to the post DataStage recovery in 24x7

chulett · Post by **chulett** » Tue Jul 31, 2007 6:48 am

Of course it's 'reasonable'... as noted, people have done this before. What's important to understand is that DataStage has no native cluster / fail-over support so nothing about it would be any kind of transparent.

I don't see why pretty much any application can't be made to fail over to another cluster node. As noted in the linked post, we built that for several apps on a Tru64 cluster. You'll need to be familiar enough with whatever cluster control software you are using to be able to script new apps (and dependant objects like crontabs) into it.

And you're right, it should be no different from a single server 'crushing' and coming back up again - all of the same recovery issues will apply.

ray.wurlod · Post by **ray.wurlod** » Tue Jul 31, 2007 4:06 pm

DataStage has particular issues because of its extensive use of shared memory, for example for locks and process control semaphores. It is really tough to fail-over things that are in memory. But it's certainly possible to re-start things on the fail-over machine, provided all the required Repository tables exist on the fail-over machine.

Talk to your vendor about licensing arrangements. It used to be the case that, if the fail-over machine was only on stand-by (that is, not being used) then a discounted licensing price could be had for it, and the main license transferred to it in the event of fail-over.