Page 1 of 1

Data Stage Ver 7.5.2 with 4 Quad Core CPU

Posted: Mon Apr 13, 2009 6:19 am
by sysmq
Hi
I have a job that create a hash file and then loads it to the memory for manipulation the output is written to DB

on the old server (2 cpu) windows 2003 sp1 3.5G Ram,
it works great and the cpu is 50-70% work during the job execution

we bought a new server with a 4 quad core CPU with windows 2003 sp1 with 3.5G Ram and the job crushes with "unable_to_alocate_memory()"
during the job execution there is no CPU activity but i have 7.2G Of ram
alocated (the entier RAM and PAGEFILE )

My question is :
Does Version 7.5.2 knows to work with a QuadCore CPU ?

sysmq

Posted: Mon Apr 13, 2009 7:20 am
by chulett
Of course. Your CPUs, quad or otherwise, have nothing to do with this. You simply ran out of allocatable memory, which is nowhere near the 7.2GB you think you have. This is 32bit architecture we're talking about here, after all.

What is the default hashed file memory allocation size on the new server, did you bump it up higher than the default of 128MB? Or at least higher than the old server? That's typically what causes this error, that combined with caching, which is what I assume you mean by "then loads it to memory for manipulation".

Posted: Thu Apr 16, 2009 1:00 am
by sysmq
HI
I know that my server went out of memory
The question is why.
let me elaborate:
In the old server with the same Os and RAM running the same project against the same DB finishes the job on time (2-3 hours) with no errors the CPU is at 50-70 % and memory is within OS limitation (less then 3.5G)

The new server does not finish the job and crashes with the memory issue os finishes after almost 14 hours.
more over the CPU does not participate at all 0-1%
and memory reaches to 7.2G (3.5 RAM and the rest Page File)

The alocated memory in the admin to the project is 512 on both servers
they have the same UVCONFIG file

What else can i check that can help me understand the difference in behaviour?

sysmq
:twisted:

Posted: Thu Apr 16, 2009 6:38 am
by Mike
A server is a lot more than just an OS and RAM. What is different about the new server such that it is not able to supply as many resources to your DataStage application as the old server? What else is consuming resources?

Mike

Posted: Thu Apr 16, 2009 7:11 am
by sysmq
hi
nothing else is consuming the memory
we use performance monitor
the whole project loads more then 16 uvsh process each with 400 mb memory

after the job finishes the memory is reduced to 812 mb total.

the main differance between the server is :
on is 2 cpu P4 3.0 GHz and the new is Quadcore 2.8 GHz exeon

one is a DL 380 server and the new one is a BLADE server

that is all

sysmq

Posted: Thu Apr 16, 2009 7:32 am
by chulett
Not sure how anyone here can really help, seeing as we're not there and not able to examine both servers ourselves. However, you are, which is why that duty falls to you and yours.

Something is different between the two, something other than simply the CPU. Sure, there are teeny differences between the two chips, but you'll not notice a .2GHz drop in speed, nor probably any difference in things like L2 Cache that they may have. And they certainly shouldn't be the prime suspects in an 'out of memory' case IMHO. Look elsewhere. Involve your administrators, the people who's job it is to look after the machines. It could be a difference in patch levels, in spite of the fact that you say both have "SP1" installed. Really shouldn't be DataStage itself if you are certain that both servers have the exact same version installed, with the exact same uvconfig & administrator settings, etc etc.

Who knows? Difficult to diagnose from this side of the glass, which is why this falls squarely on your shoulders.

Posted: Thu Apr 16, 2009 7:58 am
by sysmq
hi
nothing else is consuming the memory
we use performance monitor
the whole project loads more then 16 uvsh process each with 400 mb memory

after the job finishes the memory is reduced to 812 mb total.

the main differance between the server is :
on is 2 cpu P4 3.0 GHz and the new is Quadcore 2.8 GHz exeon

one is a DL 380 server and the new one is a BLADE server

that is all

sysmq

Posted: Thu Apr 16, 2009 8:01 am
by chulett
Not sure how anyone here can really help, seeing as we're not there and not able to examine both servers ourselves. However, you are, which is why that duty falls to you and yours.

Something is different between the two, something other than simply the CPU. Sure, there are teeny differences between the two chips, but you'll not notice a .2GHz drop in speed, nor probably any difference in things like L2 Cache that they may have. And they certainly shouldn't be the prime suspects in an 'out of memory' case IMHO. Look elsewhere. Involve your administrators, the people who's job it is to look after the machines. It could be a difference in patch levels, in spite of the fact that you say both have "SP1" installed. Really shouldn't be DataStage itself if you are certain that both servers have the exact same version installed, with the exact same uvconfig & administrator settings, etc etc.

Who knows? Difficult to diagnose from this side of the glass, which is why this falls squarely on your shoulders.

Posted: Thu Apr 16, 2009 8:01 am
by chulett
Two can play that game. :wink:

Posted: Sun Apr 19, 2009 9:11 am
by chulett
sysmq wrote:My apologies. I send it by mistake due to an error on my browser.
I know, just mess'n with ya, hence the smilie. We see plenty of double posts here, just not all that many 47 minutes apart. Couldn't resist.