Clarification on Grid enablement toolkit for Datastage paral

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
zulfi123786
Premium Member
Premium Member
Posts: 730
Joined: Tue Nov 04, 2008 10:14 am
Location: Bangalore

Clarification on Grid enablement toolkit for Datastage paral

Post by zulfi123786 »

I have read the Redbook "Deploying the Grid Solution with the Infosphere Information Server" though its on linux I guess things are same on AIX too and I have also gone through the code of

osh_conductor
Dynamic_grid.sh
*_PID.wait.sh
*_PID.sh

But few things are still unclear which are mentioned below, Could someone please help in understanding

1. I couldnt find the code which creates *_PID_start (The last point I could trace was *_PID.sh calling GenConfig class file which creates the configuration file and the next step should be creation of _PID_start for which Dynamic_grid.sh is waiting . Is the *_PID_Start file created by GenConfig class ?

2. After the job completes I couldn't find where the held resources are released by resource manager. The documentation says the _PID.sh waits for the *_PID_end file (I could see this) but after it finds the _end file it simply terminates.

Thanks
- Zulfi
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

the PID start file gets generated from the compute node via a piece of JAVA code.

Terminating the listening script which executes on the compute node frees up the slot taken up from the grid resource manager.

The only true use of the grid during execution time is to have a little sleeper program sleeping and waiting for a file to be present in grid_job_dir. It's the place holder.
BI-RMA
Premium Member
Premium Member
Posts: 463
Joined: Sun Nov 01, 2009 3:55 pm
Location: Hamburg

Post by BI-RMA »

Hi zulfi,

one of the really good things about deploying DataStage in a grid environment is that you can add cheap additional linux-boxes to your environment as needed.

Using AIX-systems really takes the low-cost argument out of the equation a bit...
"It is not the lucky ones are grateful.
There are the grateful those are happy." Francis Bacon
Post Reply