Compile and run DS Parallel jobs

rajeev_prabhuat · Post by **rajeev_prabhuat** » Sun Dec 12, 2004 11:47 pm

Hi,

I have developed a job in Parallel and i want to compile and run it, how can i do it, i understand that Parallel jobs can be compiled and run only by unix so how can i do it.

Regards,
Rajeev Prabhu

vmcburney · Post by **vmcburney** » Mon Dec 13, 2004 12:35 am

Parallel jobs can only be run on a Unix or Linux operating system. If you are working on a Windows server you will be able to create your jobs but you will not be able to run your jobs. Do you have a Unix server and DataStage licenses for Unix?

The 7.5.1 release due shortly may have a version of Parallel Extender that runs on Windows.

ray.wurlod · Post by **ray.wurlod** » Mon Dec 13, 2004 1:15 am

You will also require an appropriate C++ compiler on the UNIX machine. Do you have this?

rajeev_prabhuat · Post by **rajeev_prabhuat** » Mon Dec 13, 2004 1:17 am

Hi,

Suppose, we have unix server (Solaris) and DS license for unix. How do you compile and run the jobs ?. [Till now we have not got the DS License but Unix server is ready, we are wating for the same].

Also we have another senario, i.e till now we were working on Server jobs, now since we can do the jobs in Parallel, can we use the same set of serverjobs. If not what is the overload of rework on changing the server job to Parallel.

Regards,
Rajeev Prabhu

vmcburney wrote:Parallel jobs can only be run on a Unix or Linux operating system. If you are working on a Windows server you will be able to create your jobs but you will not be able to run your jobs. Do you have a Unix server and DataStage licenses for Unix?

The 7.5.1 release due shortly may have a version of Parallel Extender that runs on Windows.

rajeev_prabhuat · Post by **rajeev_prabhuat** » Mon Dec 13, 2004 1:18 am

Hi Ray,

What type of C++ compiler ?.

Regards,
Rajeev Prabhu

ray.wurlod wrote:You will also require an appropriate C++ compiler on the UNIX machine. Do you have this?

vmcburney · Post by **vmcburney** » Mon Dec 13, 2004 3:21 am

Now we are getting somewhere! You need to decide whether you want to switch across to parallel/enterprise jobs in a big bang implementation or an incremental approach. In the big bang approach you rewrite all your server jobs as parallel jobs. This is quite a big undertaking. In the incremental approach you just rewrite those server jobs that have a long run time that can be significantly improved by turning them into parallel jobs. The standard and parallel jobs are intermixed and run from the same sequence.

Some DataStage sites stay with a mix of standard and parallel as both types have strengths and weaknesses. Some prefer to just concentrate on the one type of job and move entirely across to parallel.

For example Ken Bland wrote a post about using parallel jobs for source extracts and target loads and keeping standard jobs in the middle for transformation and cleansing. Some databases are suited to parallel loads and huge time savings can be made by just rewriting this part of the sequence.

ray.wurlod · Post by **ray.wurlod** » Mon Dec 13, 2004 2:31 pm

By "appropriate C++ compiler" I mean the one specified in the installation notes for DataStage EE or one with identical characteristics. In general, free or open-source compilers don't cut it.

chand · Post by **chand** » Mon Dec 20, 2004 10:52 am

Hi VMC Burney,

Can Just tell me some thing more about the big bang implementation, like what is the approach and how do we implement the "big bang Theory". I will appreciate your help....it will be a great revolution in my job. I feel like things will be more easier and fast .

Regards,
chand

vmcburney · Post by **vmcburney** » Mon Dec 20, 2004 5:45 pm

I'm assuming from your previous messages that you are currently using DataStage Standard Edition and your company has purchased or is thinking of purchasing DataStage Enterprise Edition. The motivation for moving from Standard to Enterprise is performance and scalability. You should only consider this if you have problems in this area. You have the option to just migrate those jobs that have performance problems and leave the rest as Standard jobs.

While things will run faster they will not run easier. The effort required to migrate all jobs across from Standard to Enterprise makes if far from easy.

If you really want to have an impact on your ETL environment have a good look at DataStage SOA, otherwise known as Real Time Integration. You may find a selected pilot of on demand jobs gives you more impact and more features then a conversion to Enterprise jobs. You can also consider getting the DataStage Enterprise SOA Edition and getting parallel real time jobs!

T42 · Post by **T42** » Mon Dec 27, 2004 6:47 pm

I sense that one thing needs to be clarified:

Parallel jobs does not equal to Server jobs.

You must develop a new job using the Parallel palette in order to take full advantage of EE. Copying over the Server to a Parallel job will not work. You may see a modest performance boost in running Server in Parallel mode (but it's a very modest boost.)

Again, you must start from scratch in developing Parallel jobs to take full advantage of EE (PX).