Parallel job hangs
Moderators: chulett, rschirm, roy
Parallel job hangs
Hi everybody,
there's something curiously going on in my datastage jobs:
I have some (data intensive) SCD1-jobs with a lot of look ups. Most of the time they finish, but very often I have the last log entry for 6 o'clock when I check it at 9 o'clock and it should run approx. 1 hour. When I stop and restart the job, it works without any problems.
This problem does not only concern one job, on Monday it's job 1 which doesn't finish, on Friday it's job 2 which doesn't finish,...
Do you have any ideas?
Best regards,
Kerstin
there's something curiously going on in my datastage jobs:
I have some (data intensive) SCD1-jobs with a lot of look ups. Most of the time they finish, but very often I have the last log entry for 6 o'clock when I check it at 9 o'clock and it should run approx. 1 hour. When I stop and restart the job, it works without any problems.
This problem does not only concern one job, on Monday it's job 1 which doesn't finish, on Friday it's job 2 which doesn't finish,...
Do you have any ideas?
Best regards,
Kerstin
-
- Premium Member
- Posts: 536
- Joined: Thu Oct 11, 2007 1:48 am
- Location: Bangalore
Hi,
What is your source,target and lookup stages?
What is your source,target and lookup stages?
Thanks
Prasoon
ETL Consultant
LinkedIn :- http://www.linkedin.com/profile/view?id ... ab_pro_top
Blog:- http://dsshar.blogspot.com/
Prasoon
ETL Consultant
LinkedIn :- http://www.linkedin.com/profile/view?id ... ab_pro_top
Blog:- http://dsshar.blogspot.com/
Any time I see symptoms like that - a random issue appears that does not repeat when the job is rerun on its own - it says "resource issue" to me. How many jobs are running each night? Knowing nothing else about your situation, first suggestion would be to scale back how many run at the same time.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Premium Member
- Posts: 1735
- Joined: Thu Mar 01, 2007 5:44 am
- Location: Troy, MI
In cases where it works sometimes it may be resource issue so talk to your database admin and system admin to see if there is any such issue they see in Trace file/ASH Report. SGA/PGA value can also play a part in such scenario.
I had similar issues with Oracle Connector even if there is 1K to 2K records and used to finish in 1 second using oracle enterprise stage. But in my case it stopped working completely for few jobs. I did raise a PMR with IBM and instead of fixing the issue, they asked us to use Oracle enterprise stage as Oracle connector is new and may create problems sometimes. This happened on 8.5, I believe they don't want to release a patch since 8.7 was already in market. Since we were planing to move to 8.7, we did not pursue it any further.
I had similar issues with Oracle Connector even if there is 1K to 2K records and used to finish in 1 second using oracle enterprise stage. But in my case it stopped working completely for few jobs. I did raise a PMR with IBM and instead of fixing the issue, they asked us to use Oracle enterprise stage as Oracle connector is new and may create problems sometimes. This happened on 8.5, I believe they don't want to release a patch since 8.7 was already in market. Since we were planing to move to 8.7, we did not pursue it any further.
Priyadarshi Kunal
Genius may have its limitations, but stupidity is not thus handicapped.
Genius may have its limitations, but stupidity is not thus handicapped.
We made an additional limitation for the data to work with and the error has never come again. So I agree with you, that it is a space limitation/performance problem. Unfortunately I have no access or answer from a db admin, which would have been very interestering and helpful for the upcoming projects. Nevertheless thanks a lot for your help!