Plz let us know some best practice steps to improve performance.
In Datastage jobs.
Thanks
Steps to Improve Performance
Moderators: chulett, rschirm, roy
nraj,
the question is far too broad and general to even begin to answer. It is like asking "How do I make my car go faster" -> there are too many variables and places where you can introduce blockages.
If you were to ask a more specific question, such as "I have sequential EBCDIC sources going to a SQL server database and have a number of Oracle tables that I use as lookups. The main lookup table has 1.5 millions rows and the daily input volume is 10 million records. The job runs 30 minutes on my Solarix xxx box on a SAN array. Can I tune this?" you would get some helpful comments on areas to look at.
I would highly recommend actually reading the Ascential administrator guide and taking a look at Ascential's ADN site (plus of course browsing this forum for keywords like tuning and best practices).
If you don't wish to do any of the above, here are some of my suggestions that are generally true:
1. Don't use "SLEEP" statements
2. Don't load large tables to memory that you don't use
3. Add main memory
4. Add CPUs
5. Add faster and parallel disk I/O
6. Repeat 3-5 until satisfied
the question is far too broad and general to even begin to answer. It is like asking "How do I make my car go faster" -> there are too many variables and places where you can introduce blockages.
If you were to ask a more specific question, such as "I have sequential EBCDIC sources going to a SQL server database and have a number of Oracle tables that I use as lookups. The main lookup table has 1.5 millions rows and the daily input volume is 10 million records. The job runs 30 minutes on my Solarix xxx box on a SAN array. Can I tune this?" you would get some helpful comments on areas to look at.
I would highly recommend actually reading the Ascential administrator guide and taking a look at Ascential's ADN site (plus of course browsing this forum for keywords like tuning and best practices).
If you don't wish to do any of the above, here are some of my suggestions that are generally true:
1. Don't use "SLEEP" statements
2. Don't load large tables to memory that you don't use
3. Add main memory
4. Add CPUs
5. Add faster and parallel disk I/O
6. Repeat 3-5 until satisfied
</a>-
Sainath.Srinivasan
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
Sainath - I didn't realize that the question was in a PX forum.
As you are working with PX the basic design of your processes needs to reflect the scalability and implicit parallelism. You can create the most elegant, pretty, concise and clear code in Px that will run incredibly slowly if you don't understand the under-the-covers-actions that Px takes.
As you are working with PX the basic design of your processes needs to reflect the scalability and implicit parallelism. You can create the most elegant, pretty, concise and clear code in Px that will run incredibly slowly if you don't understand the under-the-covers-actions that Px takes.