Datastage-Unix I/O transfer problem ?

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Guess again. If it was we'd have solved this by now. :wink:

So... this process launches every five minutes, eh? And yet your stream of jobs runs anywhere from 10 to 50 minutes? How are you ensuring there's no contention there, no locks or collisions amongst the processes?

And this script... after it 'clubs' together these teeny little files for processing, does it launch one 'master' job? Something like a Sequence or job control job that runs the other 21 or 22 jobs? And it's only that first invocation that takes extra time and the hand-off between all of the others is fine? Or the first invocation from script to job takes time and all of the internal Job Activity invocations take time as well? Or the script is the 'master job' and runs all of the jobs in sequential order, and they all take longer than they should? :?

As you can see there's still plenty of secrets, plenty of mysteries here. There's an expression about "pulling teeth" that comes to mind. Sure, the lights have come up a little bit and the darkness slightly receded, but we're still waving our hands around in the dimness pointing in random directions shouting "perhaps the answer lies... over there!"

Don't expect us to somehow hand you The Answer, we're not there and seeing what you're seeing. Like any good leaders, we'll do our best to point you in the right direction, what things to check or look for but still ultimately it comes down to you - the troops on the ground - to fight this battle and flush out the enemy.
-craig

"You can never have too many knives" -- Logan Nine Fingers
suneyes
Participant
Posts: 82
Joined: Mon Jul 21, 2008 8:42 am

Post by suneyes »

Hey guys,
Thanks a lot for your help.
The delay was being induced by ever increasing job logs.
We had logs over 2 years old. The huge size of the logs was causing the performance drop.
Once the older logs have been purged, The performance is good again.

Thanks again for your support.
sun
Post Reply