Grid Capability for DataStage

Lali_swam · Post by **Lali_swam** » Wed Dec 14, 2005 2:22 am

Hi,

What do we mean by grid capability in DataStage 7.5.1. Someone asked me if this question "Does DS 7.5.1 have grid capability? If so, what software and components are needed to make our server to be grid enabled". Please enlighten me. Appreciate all your answers.

Thanks,
Lalitha

kcbland · Post by **kcbland** » Wed Dec 14, 2005 8:06 am

You need to google "grid computing". It's a method where computers share their power, think of 50 PCs sitting on desktops around a company doing word processing and email all day. At night, all of the PC's become a larger virtual computer of 50 cpus working together to process data. Those 50 machines are called a "grid computer". The difference between a supercomputer of many clustered servers and a "grid" is that a grid can be just a bunch of desktops, arbitrarily contributing their resources when available.

Check out Seti@home to see grid computing realized in a very large sense. It's people (like me) who volunteer their PCs to work evenings helping process vast amounts of data.

DS does not support grid computing yet.

ray.wurlod · Post by **ray.wurlod** » Wed Dec 14, 2005 3:03 pm

DataStage EE is currently running in production on at least two grids of which I am aware. It treats the grid as an MPP cluster. The operating system looks after a lot of the distribution of work.
I have not seen any official announcement that DS supports grid computing, but at least two sites have it working.

shawn_ramsey · Post by **shawn_ramsey** » Wed Dec 14, 2005 3:56 pm

Lalitha,

There was a good session on grid and DataStage EE at the IBM conference. I would see if your IBM Sales Rep or SE can get you the power point for that session.

vmcburney · Post by **vmcburney** » Wed Dec 14, 2005 4:16 pm

If you hook up the servers and install DataStage what you have is a cluster. A lot of manual configuration is required in the DataStage config files to get the jobs to run over the cluster. If you add grid management software what you get is a grid. This is far more clever at load balancing and allocating jobs to servers and running many server software jobs (not just DataStage) across those grid machines.

It is possible to get a grid up and running without the grid management software but it would require a lot of in house development and configuration.

Lali_swam · Post by **Lali_swam** » Thu Dec 15, 2005 7:01 am

Hi ,
Thank you all for your valuable responses. This question was posted to me by my Manager and infact he also attended the IBM Integration Live. I got the presentation on the grid, The topic is
"Best Practices and Tech Deep Dive - Deploying a Linux Grid" .

Thanks a lot,
Lalitha

Lali_swam · Post by **Lali_swam** » Thu Dec 15, 2005 9:59 pm

Lali_swam wrote:Hi ,
Thank you all for your valuable responses.
Thanks a lot,
Lalitha