Hardware sizing of Dev, Test and Production Environments

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Bill_G
Premium Member
Premium Member
Posts: 74
Joined: Thu Oct 20, 2005 9:34 am

Hardware sizing of Dev, Test and Production Environments

Post by Bill_G »

I am relatively new to the PX dev world, and have taken on the task of implementing Enterprise and updating the job designs from server to PX. I have poured over the px and adv px dev guide and know enough to be dangerous.

We are running into issues where our ETL job designs are limited by the resources available on our Dev/Test box, however the jobs will ultimately run in a much larger Production environment. Our Dev/Test environment is a dual proc native blade on egenera framework with 6 gig of ram. Our Production box is a 4 -way on the same In maximizing our designs on the dual processor Dev box, CPU/Memory is crushed and we run into netstat and I/O failure issues causing broken client connections, job failures etc. The jobs would most likely run efficiently on the larger Production box, but given that the basic design of the jobs is related to the resources available, I am reluctant to proceed. To bolster the obvious, I need to document differences in Hardware between Dev, Test, and Production ETL environments from various environments to make a case for a hardware upgrade to our dev/test environments. IBM was unable to provide specific guidelines for hardware.



Specifically, I would be interested in the Hardware Manufacturer, OS, number of CPUs, RAM, and if you can remember, the amount of available swap space for your Dev/Test and Production environments. If you have an idea of the dollars spent (or at least a %) on Dev/Test vs. Production that would be helpful as well.



Any best practices that have been proven out would be greatly appreciated.





Thanks in Advance for this information.
Post Reply