Hi All
I am new to datastage.I've designed a job which populates a table.It extracts the data from three different tables,90% of the rows come from a particular table.In the transformer stage I perform many substring functions on particular input column which feed the appropriate columns in the target.I also have two subroutines which feed the target column upon satisfying the particular condition.Now,my problem is,it is taking lot of time to load the target.When i monitor the transformer stage it shows 1 row/sec initially and after that it doesn't load for quite long time it took some 30 Hrs to load 3500 records.Can anyone please tell me the solution to improve the performance.
Thanks in advance.
Shivakumar
Performance
Moderators: chulett, rschirm, roy
Hi,
I am not sure that this is performance problem.
I run into this behavior once and I did next to spot the problem :
1. Check performance after getting rid of routines in transformer.
If it works better try simplify your logic in routine.
2. If not, continue getting rid of rest of logic in your yob untill
you identify where is the problem.
Once I made whole this work around but job continued to run with 1 record per sec speed. So finally I rebuilt job from the scratch with completly same logic as it was before and job worked perfect.
In this case I made conclusion that it was DataStage bug.
Regards.
I am not sure that this is performance problem.
I run into this behavior once and I did next to spot the problem :
1. Check performance after getting rid of routines in transformer.
If it works better try simplify your logic in routine.
2. If not, continue getting rid of rest of logic in your yob untill
you identify where is the problem.
Once I made whole this work around but job continued to run with 1 record per sec speed. So finally I rebuilt job from the scratch with completly same logic as it was before and job worked perfect.
In this case I made conclusion that it was DataStage bug.
Regards.
Shiva
Routines are not slow usually. Send me a copy of the routine and I will tell you what the problem is. Also send the export of the one job and I will be able to evaluate it better.
Thanks Kim.
quote:Originally posted by shiva459
[br]Hi All
I am new to datastage.I've designed a job which populates a table.It extracts the data from three different tables,90% of the rows come from a particular table.In the transformer stage I perform many substring functions on particular input column which feed the appropriate columns in the target.I also have two subroutines which feed the target column upon satisfying the particular condition.Now,my problem is,it is taking lot of time to load the target.When i monitor the transformer stage it shows 1 row/sec initially and after that it doesn't load for quite long time it took some 30 Hrs to load 3500 records.Can anyone please tell me the solution to improve the performance.
Thanks in advance.
Shivakumar
Kim Duke
Routines are not slow usually. Send me a copy of the routine and I will tell you what the problem is. Also send the export of the one job and I will be able to evaluate it better.
Thanks Kim.
quote:Originally posted by shiva459
[br]Hi All
I am new to datastage.I've designed a job which populates a table.It extracts the data from three different tables,90% of the rows come from a particular table.In the transformer stage I perform many substring functions on particular input column which feed the appropriate columns in the target.I also have two subroutines which feed the target column upon satisfying the particular condition.Now,my problem is,it is taking lot of time to load the target.When i monitor the transformer stage it shows 1 row/sec initially and after that it doesn't load for quite long time it took some 30 Hrs to load 3500 records.Can anyone please tell me the solution to improve the performance.
Thanks in advance.
Shivakumar
Kim Duke
-
ray.wurlod
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: