bad performance

fmartinsferreira · Post by **fmartinsferreira** » Mon Feb 12, 2007 12:05 pm

I have a server with one project.
In this project I have 5 critical servers jobs.

job1
job2
job3
job4
job5

These five jobs start at the same time.

job1, job2 and job3 spend 2 hours.
job4 and job5 spend 10 hours.

We re-build job4 and job5 as a parallel job and now they run in 2 hours,
but job1, job2 and job3 are spending 4 hours.

why?

good point - none another application concurring in datastage server

Regards,

Fernando

narasimha · Post by **narasimha** » Mon Feb 12, 2007 12:10 pm

May be Job 4 and Job 5 are consuming all the resources, leaving very little for Job1, Job2 and job3.
My two cents.

fmartinsferreira · Post by **fmartinsferreira** » Mon Feb 12, 2007 12:21 pm

narasimha wrote:May be Job 4 and Job 5 are consuming all the resources, leaving very little for Job1, Job2 and job3.
My two cents.

Is there any way to separate some resource only for the servers jobs?

Something like

parallel - 70%
server - 30%

Fernando

ArndW · Post by **ArndW** » Mon Feb 12, 2007 12:26 pm

there is no easy way to separate out the Parallel and Server job workloads. Your only steering mechanism is via the APT_CONFIG file and how many nodes you choose to use in the PX jobs. Note that PX jobs inherently get more system resources than server jobs.

I_Server_Whale · Post by **I_Server_Whale** » Mon Feb 12, 2007 12:50 pm

Can job1, job2 and job3 be converted to PX jobs?

Krazykoolrohit · Post by **Krazykoolrohit** » Mon Feb 12, 2007 12:54 pm

tell us the amount of data which you trying to pass

fmartinsferreira · Post by **fmartinsferreira** » Fri Feb 16, 2007 7:31 am

Can job1, job2 and job3 be converted to PX jobs?

Yes, but I can have the same problem with others projects.

tell us the amount of data which you trying to pass

job1 = 5 million registers
job2 = 5 million registers
job3 = 5 million registers
job4 = 85 million registers
job5 = 85 million registers

Fernando

Krazykoolrohit · Post by **Krazykoolrohit** » Fri Feb 16, 2007 10:20 am

and your job design?

if you are using to extract and load into tables in the same job with so much data, i have observed it to be slower in server jobs than in parallel ones.

ray.wurlod · Post by **ray.wurlod** » Fri Feb 16, 2007 2:31 pm

Parallel jobs cosume far more resources than server jobs; these may be starving the server jobs of resources. Have you considered this design in a job sequence?

Code: Select all

Job1  -----+
           |                    +--->  Job4
Job2  -----+--->  Sequencer  ---+
           |         (all)      +--->  Job5
Job3  -----+

Or even:

Code: Select all

Job1  -----+
           |                    
Job2  -----+--->  Sequencer  ----->  Job4  ---->  Job5
           |         (all)     
Job3  -----+