Performance degraded after migration from Datstage 7.5 to 8.
Moderators: chulett, rschirm, roy
Performance degraded after migration from Datstage 7.5 to 8.
Hi All,
We have recently migrated Datastage Project from 7.5 to 8.5, But we are facing performance issues, The old project and new project, both are on same UNIX box. There are some jobs in new project taking as much as double the time of what they were taking in old project.
We have taken a job, having performance issue . The Design of the job is as given below.
DRS_SATGE --> Transformer + Lookup --> DRS_STAGE
The Source and target tables are in Oracle database.
and in order to find out bottleneck, we devided the Job into three jobs
Job1
------------------------------------------------
DRS Stage --> Sequential_File_1
Job2
-------------------------------------------
Sequential_File_1 --> Transformer + Lookup --> Sequential_File_2
Job 3
---------------------------------------------------
Sequential_File_2 --> DRS_STage
We run these three jobs in Datastage 7.5 and 8.5
The Performance of Job2 is almost same in both the version of Datastage,
But the performance of Job_1 and Job_3 in 8.5 side is not comparable with 7.5 side.
We replace DRS stage with OCI stage but it didn't help much.
Are there any settings or parameters which need to fix in 8.5 .
Please suggest.
Thanks,
Nitin Swarnkar
We have recently migrated Datastage Project from 7.5 to 8.5, But we are facing performance issues, The old project and new project, both are on same UNIX box. There are some jobs in new project taking as much as double the time of what they were taking in old project.
We have taken a job, having performance issue . The Design of the job is as given below.
DRS_SATGE --> Transformer + Lookup --> DRS_STAGE
The Source and target tables are in Oracle database.
and in order to find out bottleneck, we devided the Job into three jobs
Job1
------------------------------------------------
DRS Stage --> Sequential_File_1
Job2
-------------------------------------------
Sequential_File_1 --> Transformer + Lookup --> Sequential_File_2
Job 3
---------------------------------------------------
Sequential_File_2 --> DRS_STage
We run these three jobs in Datastage 7.5 and 8.5
The Performance of Job2 is almost same in both the version of Datastage,
But the performance of Job_1 and Job_3 in 8.5 side is not comparable with 7.5 side.
We replace DRS stage with OCI stage but it didn't help much.
Are there any settings or parameters which need to fix in 8.5 .
Please suggest.
Thanks,
Nitin Swarnkar
-
- Participant
- Posts: 26
- Joined: Fri Feb 19, 2010 2:08 am
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
There is always some performance hit in server jobs on version 8 over version 7.5 because of the new environment in which things are operating. I've posted about this in the past. In short, though, everything executes in a service-oriented architecture running on a parallel execution engine. To communicate with this, server jobs have to undergo a "translation" process.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
In my humble opinion, the fact that the only jobs that are connecting to the database are significantly slower, implies that the problem is your database connection.
I can't say if the actual connection or perhaps some database optimezer issue or your network configuration, is the one rsponsible but those seems to be the the ones to check.
IHTH(I Hope This Helps),
I can't say if the actual connection or perhaps some database optimezer issue or your network configuration, is the one rsponsible but those seems to be the the ones to check.
IHTH(I Hope This Helps),
Roy R.
Time is money but when you don't have money time is all you can afford.
Search before posting:)
Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Time is money but when you don't have money time is all you can afford.
Search before posting:)
Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
-
- Premium Member
- Posts: 483
- Joined: Thu Jun 12, 2003 4:47 pm
- Location: St. Louis, Missouri USA
Hi All,
The IBM suggested to disable the NLS settings and that does improve the performance.
In Datastage 7.5, job was processing 11000 rows/second. In Datastage 8.5, with NLS enabled it was processing 4500 rows/sec and after desabling NLS, it become 8500 row/sec.
Admins are still in touch with IBM. I will keep you posted.
Thanks,
Nitin Swarnkar
The IBM suggested to disable the NLS settings and that does improve the performance.
In Datastage 7.5, job was processing 11000 rows/second. In Datastage 8.5, with NLS enabled it was processing 4500 rows/sec and after desabling NLS, it become 8500 row/sec.
Admins are still in touch with IBM. I will keep you posted.
Thanks,
Nitin Swarnkar
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: