Node Failover Option in datastage

[sudeepmantri]

Dear all,

shortly we'll be going live with our project and I am trying to explore the option of Node failover option in datastage. For example say I have Environment of 16 nodes and currently my Job is using 2 nodes (Node 1 and Node 2) and say suppose Node-2 Fails. Is there any way that I can use other free nodes instead of this Failed Node-2 and continue with my Job Processing.

Thanks in advance.

Regards,
Sudeep

[nagarjuna]

Node-2 Fails means what ? All the nodes that you mention in configuration file are logical in nature ...

[sudeepmantri]

Node-2 Fails means, the resource/scratch disk allocated to this node fails. Or the section leader itself dies while establishing a communication to this Node!!

[miwinter]

Not from within Datastage itself no. You'd need some logic in place to detect an issue (which could be one of many), and then swap in a new config dynamically to use this in place of the original, excluding the problem node(s). However, this won't avoid having to restart a process from the beginning again and possibly will also require starting further back up the flow within your end-to-end process. The inputs for datasets/filesets and the like won't exist on the nodes it expects them to, so the processes behind producing these would also then have to be re-run to correct this.

In summary, yes you could do it, but not quickly and easily and it would be a bespoke solution rather than out-of-the-box and would involve some fairly complex logic if it were to be entirely automated.

[ray.wurlod]

Talk to IBM about the services they offer for grid-based high availability. There may be something there to lighten your wallet.

[sudeepmantri]

Hmm..yeah ur correct..It can be done, but not straight forward. Problem exists when I have intermediate Filesets and datasets where in I have to change the descriptor accordingly and move the data file to some other place..


Thanks all for your inputs!!