my job is taking 45 minits to load the data
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 86
- Joined: Wed Mar 03, 2010 3:09 am
my job is taking 45 minits to load the data
Hi All,
My job design is like this
dataset----->transformer------->odbc connector
my job is having 169000 records,i am using update then insert to load the data.It is taking 45 minits to load the data.I am using HASH partitioning in transformer and odbc stages.
can anyone please suggest me how to improve the performance.
thanks for your help in advance
My job design is like this
dataset----->transformer------->odbc connector
my job is having 169000 records,i am using update then insert to load the data.It is taking 45 minits to load the data.I am using HASH partitioning in transformer and odbc stages.
can anyone please suggest me how to improve the performance.
thanks for your help in advance
-
- Participant
- Posts: 86
- Joined: Wed Mar 03, 2010 3:09 am
-
- Premium Member
- Posts: 138
- Joined: Wed Jul 16, 2008 9:51 pm
- Location: Kolkata
-
- Participant
- Posts: 86
- Joined: Wed Mar 03, 2010 3:09 am
-
- Premium Member
- Posts: 138
- Joined: Wed Jul 16, 2008 9:51 pm
- Location: Kolkata
Hard coding of columns is not known to be a performance breaker but type conversions if not done with care will have an impact.
Share the column metadata which is used before and after type conversion.
Share the column metadata which is used before and after type conversion.
TONY
ETL Manager
Infotrellis India
"Do what you can, with what you have, from where you are and to the best of your abilities."
ETL Manager
Infotrellis India
"Do what you can, with what you have, from where you are and to the best of your abilities."
-
- Premium Member
- Posts: 120
- Joined: Thu Oct 28, 2004 4:24 pm
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
You can prove the location of the bottleneck by writing to a Sequential File rather than Oracle. If that's no faster, the problem is in DataStage. If it is markedly faster, then the problem is in Oracle. Updating tables where the columns mentioned in the WHERE clause are not indexed is notoriously slow in any database.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 86
- Joined: Wed Mar 03, 2010 3:09 am
Thanks ray,sorry for the late reply,i was on leave
i have done that,when i am loading into sequential file it is not taking time.
whenever i am trying to load the data into ORACLE i am getting this problem.
i think there may be some indexes on key columns,i will check and let you know guys.
thanks again
i have done that,when i am loading into sequential file it is not taking time.
whenever i am trying to load the data into ORACLE i am getting this problem.
i think there may be some indexes on key columns,i will check and let you know guys.
thanks again
-
- Participant
- Posts: 86
- Joined: Wed Mar 03, 2010 3:09 am