Hello,
Has anyone had any experience running DSEE/PX jobs or sequences over longer periods of time (several days)?
If so, have you had any bad experiences with the jobs hanging, memory overflows, etc.?
In my specific case, I'm not intending to do anything complicated during most of the run - but I do want to "sleep", mid-sequence, for up to 72 hours before resuming processing. Can anyone please comment on whether or not that would be a good idea?
TIA,
tpb
DataStage newbie
DS Stability over Longer Run Times
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
There's no reason in theory that you shouldn't do what you want, but you're taking a risk that the system might be stopped during your long sleep. Or that an administrator may clean up apparently idle processes.
I'd advise researching a different approach, perhaps finish gracefully now and schedule another job to start 72 hours later. Scheduled jobs are queued, and survive system shutdowns and the like.
I'd advise researching a different approach, perhaps finish gracefully now and schedule another job to start 72 hours later. Scheduled jobs are queued, and survive system shutdowns and the like.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
DataStage (especially with EE) are VERY solid with their memory usage. Memory leaks are VERY rare out in the field, and jobs could practically run for years if you let it.
However, do pay attention to what Ray said. Do you REALLY want to let DataStage, a pure ETL box, to handle scheduling duties? Why not invest into a good third party scheduler and let that tool do the job? Ascential do recommend that if you want an enterprise scheduler.
However, do pay attention to what Ray said. Do you REALLY want to let DataStage, a pure ETL box, to handle scheduling duties? Why not invest into a good third party scheduler and let that tool do the job? Ascential do recommend that if you want an enterprise scheduler.
-
- Participant
- Posts: 82
- Joined: Thu Dec 02, 2004 10:27 pm
- Location: INDIA
Another option is to use your 3rd party scheduler to schedule the job regularly. Daily or hourly. To control from your end you can have a file . IF the file is present the job will not start. Once you remove the file, the job will run regularly. I am assuming you are starting you sequencer using a unix shell script.
This way you never have to ask the scheduling team to stop or start your jobs.
This way you never have to ask the scheduling team to stop or start your jobs.
dsxuserrio
Kannan.N
Bangalore,INDIA
Kannan.N
Bangalore,INDIA
Nice try. But then I'd have to go through the file control team!Another option is to use your 3rd party scheduler to schedule the job regularly. Daily or hourly. To control from your end you can have a file . IF the file is present the job will not start. Once you remove the file, the job will run regularly. I am assuming you are starting you sequencer using a unix shell script.
This way you never have to ask the scheduling team to stop or start your jobs.
After further review, I've decided I'm just going to have to create 3 job sequences that can be run by the 3rd-party scheduler; one for M-Th that runs entirely to completion, one for Friday nights that runs part way, and another for Sunday PM that finishes up where Friday night left off.
Oh well, just 200% more things to keep track of.
tpb
DataStage newbie