There seems to be a constant delay of 10-12 secs while a DS job closes.
The actual job itself takes only 5 secs. This can add up when you have a large number of DS jobs in a workflow.
Is there any way of reducing this delay ?
Peter
Inter-job delays
Cleaning up the &PH& directory can help (if you are on NT, anyway).
-----Original Message-----
From: Peter Oates [mailto:Peter.Oates@macquarie.com]
Sent: Wednesday, November 07, 2001 2:05 PM
To: datastage-users@oliver.com
Subject: Inter-job delays
There seems to be a constant delay of 10-12 secs while a DS job closes.
The actual job itself takes only 5 secs. This can add up when you have a large number of DS jobs in a workflow.
Is there any way of reducing this delay ?
Peter
*************************************************************************
This e-mail and any files transmitted with it may be confidential and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in
error, please notify the sender by return e-mail, and delete this e-mail from your in-box. Do not copy it to anybody else
*************************************************************************
-----Original Message-----
From: Peter Oates [mailto:Peter.Oates@macquarie.com]
Sent: Wednesday, November 07, 2001 2:05 PM
To: datastage-users@oliver.com
Subject: Inter-job delays
There seems to be a constant delay of 10-12 secs while a DS job closes.
The actual job itself takes only 5 secs. This can add up when you have a large number of DS jobs in a workflow.
Is there any way of reducing this delay ?
Peter
*************************************************************************
This e-mail and any files transmitted with it may be confidential and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in
error, please notify the sender by return e-mail, and delete this e-mail from your in-box. Do not copy it to anybody else
*************************************************************************
How do you measure the 10-12 seconds? Much of the "delay" between the finish of the final active stage and the finish of the job is involved in closing things - either files or connections to databases. Close is an expensive operation, since it has to update the date/time accessed (and, possibly, date/time modified) not only of the objects accessed, but also the directory, its parent directory, and so on. Further, if you were using memory-mapped hashed files, these need to be flushed back to disk. You can reduce this time somewhat by being as selective as possible about what you actually load into the hashed file.
-----Original Message-----
From: Peter Oates [mailto:Peter.Oates@macquarie.com]
Sent: Wednesday, 07 November 2001 15:05
To: datastage-users@oliver.com
Subject: Inter-job delays
There seems to be a constant delay of 10-12 secs while a DS job closes. The actual job itself takes only 5 secs. This can add up when you have a large number of DS jobs in a workflow.
Is there any way of reducing this delay ?
Peter
-----Original Message-----
From: Peter Oates [mailto:Peter.Oates@macquarie.com]
Sent: Wednesday, 07 November 2001 15:05
To: datastage-users@oliver.com
Subject: Inter-job delays
There seems to be a constant delay of 10-12 secs while a DS job closes. The actual job itself takes only 5 secs. This can add up when you have a large number of DS jobs in a workflow.
Is there any way of reducing this delay ?
Peter
Unix also has a &PH& directory - just dont know if it will have the same affect. Tech Support e-mailed me a procedure the other day on how to clean up this directory in Unix.
I quote the e-mail:
"As per our phone conversation, these are the instructions to clear the &PH&
directory:
1) Open DS Administrator
2) Highlight your project and select Command button
3) Issue the following command CLEAR.FILE &PH& and execute."
(...probably the same for NT)
Dirk
-----Original Message-----
From: David Barham [mailto:David.Barham@Anglocoal.com.au]
Sent: Wednesday, November 07, 2001 6:08 AM
To: datastage-users@oliver.com
Subject: RE: Inter-job delays
Cleaning up the &PH& directory can help (if you are on NT, anyway).
-----Original Message-----
From: Peter Oates [mailto:Peter.Oates@macquarie.com]
Sent: Wednesday, November 07, 2001 2:05 PM
To: datastage-users@oliver.com
Subject: Inter-job delays
There seems to be a constant delay of 10-12 secs while a DS job closes. The actual job itself takes only 5 secs. This can add up when you have a large number of DS jobs in a workflow.
Is there any way of reducing this delay ?
Peter
*************************************************************************
This e-mail and any files transmitted with it may be confidential and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender by return e-mail, and delete this e-mail from your in-box. Do not copy it to anybody else
*************************************************************************
==========================================================
This message contains information intended for the perusal, and/or use (if so stated), by the stated addressee(s) only. The information is confidential and privileged. If you are not an intended recipient, do not peruse, use, disseminate, distribute, copy or in any manner rely upon the information contained in this message (directly or indirectly). The sender and/or the entity represented by the sender shall not be held accountable in the event that this prohibition is disregarded. If you receive this message in error, notify the sender immediately by e-mail, fax or telephone representations contained in this message, whether express or implied, are those of the sender only, unless that sender expressly states them to be the views or representations of an entity or person, who shall be named by the sender and who the sender shall state to represent. No liability shall otherwise attach to any other entity or person. ==========================================================
I quote the e-mail:
"As per our phone conversation, these are the instructions to clear the &PH&
directory:
1) Open DS Administrator
2) Highlight your project and select Command button
3) Issue the following command CLEAR.FILE &PH& and execute."
(...probably the same for NT)
Dirk
-----Original Message-----
From: David Barham [mailto:David.Barham@Anglocoal.com.au]
Sent: Wednesday, November 07, 2001 6:08 AM
To: datastage-users@oliver.com
Subject: RE: Inter-job delays
Cleaning up the &PH& directory can help (if you are on NT, anyway).
-----Original Message-----
From: Peter Oates [mailto:Peter.Oates@macquarie.com]
Sent: Wednesday, November 07, 2001 2:05 PM
To: datastage-users@oliver.com
Subject: Inter-job delays
There seems to be a constant delay of 10-12 secs while a DS job closes. The actual job itself takes only 5 secs. This can add up when you have a large number of DS jobs in a workflow.
Is there any way of reducing this delay ?
Peter
*************************************************************************
This e-mail and any files transmitted with it may be confidential and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender by return e-mail, and delete this e-mail from your in-box. Do not copy it to anybody else
*************************************************************************
==========================================================
This message contains information intended for the perusal, and/or use (if so stated), by the stated addressee(s) only. The information is confidential and privileged. If you are not an intended recipient, do not peruse, use, disseminate, distribute, copy or in any manner rely upon the information contained in this message (directly or indirectly). The sender and/or the entity represented by the sender shall not be held accountable in the event that this prohibition is disregarded. If you receive this message in error, notify the sender immediately by e-mail, fax or telephone representations contained in this message, whether express or implied, are those of the sender only, unless that sender expressly states them to be the views or representations of an entity or person, who shall be named by the sender and who the sender shall state to represent. No liability shall otherwise attach to any other entity or person. ==========================================================
The reason I specifically mentioned NT is I believe that the way NT handles attempts to access locked files causes excessive delays to the job close down when there are a lot of jobs running with lots of active stages.
In my early DataStage days, when running a dozen or so short jobs at the same time, I witnessed delays of up to 2 minutes or more to close down a job which took 10 seconds to execute. When running these jobs by themselves, they closed down in a couple of seconds.
I had a quick look at last nights logs. I have a job which runs repeated. The close down time for it varies (randomly?) from 1 second to about 20 seconds.
-----Original Message-----
From: Dirk Moolman [mailto:dirkm@reach.co.za]
Sent: Wednesday, 7 November 2001 4:34 PM
To: datastage-users@oliver.com
Subject: RE: Inter-job delays
Unix also has a &PH& directory - just dont know if it will have the same affect. Tech Support e-mailed me a procedure the other day on how to clean up this directory in Unix.
I quote the e-mail:
"As per our phone conversation, these are the instructions to clear the &PH&
directory:
1) Open DS Administrator
2) Highlight your project and select Command button
3) Issue the following command CLEAR.FILE &PH& and execute."
(...probably the same for NT)
Dirk
-----Original Message-----
From: David Barham [mailto:David.Barham@Anglocoal.com.au]
Sent: Wednesday, November 07, 2001 6:08 AM
To: datastage-users@oliver.com
Subject: RE: Inter-job delays
Cleaning up the &PH& directory can help (if you are on NT, anyway).
-----Original Message-----
From: Peter Oates [mailto:Peter.Oates@macquarie.com]
Sent: Wednesday, November 07, 2001 2:05 PM
To: datastage-users@oliver.com
Subject: Inter-job delays
There seems to be a constant delay of 10-12 secs while a DS job closes. The actual job itself takes only 5 secs. This can add up when you have a large number of DS jobs in a workflow.
Is there any way of reducing this delay ?
Peter
*************************************************************************
This e-mail and any files transmitted with it may be confidential and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender by return e-mail, and delete this e-mail from your in-box. Do not copy it to anybody else
*************************************************************************
==========================================================
This message contains information intended for the perusal, and/or use (if so stated), by the stated addressee(s) only. The information is confidential and privileged. If you are not an intended recipient, do not peruse, use, disseminate, distribute, copy or in any manner rely upon the information contained in this message (directly or indirectly). The sender and/or the entity represented by the sender shall not be held accountable in the event that this prohibition is disregarded. If you receive this message in error, notify the sender immediately by e-mail, fax or telephone representations contained in this message, whether express or implied, are those of the sender only, unless that sender expressly states them to be the views or representations of an entity or person, who shall be named by the sender and who the sender shall state to represent. No liability shall otherwise attach to any other entity or person. ==========================================================
In my early DataStage days, when running a dozen or so short jobs at the same time, I witnessed delays of up to 2 minutes or more to close down a job which took 10 seconds to execute. When running these jobs by themselves, they closed down in a couple of seconds.
I had a quick look at last nights logs. I have a job which runs repeated. The close down time for it varies (randomly?) from 1 second to about 20 seconds.
-----Original Message-----
From: Dirk Moolman [mailto:dirkm@reach.co.za]
Sent: Wednesday, 7 November 2001 4:34 PM
To: datastage-users@oliver.com
Subject: RE: Inter-job delays
Unix also has a &PH& directory - just dont know if it will have the same affect. Tech Support e-mailed me a procedure the other day on how to clean up this directory in Unix.
I quote the e-mail:
"As per our phone conversation, these are the instructions to clear the &PH&
directory:
1) Open DS Administrator
2) Highlight your project and select Command button
3) Issue the following command CLEAR.FILE &PH& and execute."
(...probably the same for NT)
Dirk
-----Original Message-----
From: David Barham [mailto:David.Barham@Anglocoal.com.au]
Sent: Wednesday, November 07, 2001 6:08 AM
To: datastage-users@oliver.com
Subject: RE: Inter-job delays
Cleaning up the &PH& directory can help (if you are on NT, anyway).
-----Original Message-----
From: Peter Oates [mailto:Peter.Oates@macquarie.com]
Sent: Wednesday, November 07, 2001 2:05 PM
To: datastage-users@oliver.com
Subject: Inter-job delays
There seems to be a constant delay of 10-12 secs while a DS job closes. The actual job itself takes only 5 secs. This can add up when you have a large number of DS jobs in a workflow.
Is there any way of reducing this delay ?
Peter
*************************************************************************
This e-mail and any files transmitted with it may be confidential and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender by return e-mail, and delete this e-mail from your in-box. Do not copy it to anybody else
*************************************************************************
==========================================================
This message contains information intended for the perusal, and/or use (if so stated), by the stated addressee(s) only. The information is confidential and privileged. If you are not an intended recipient, do not peruse, use, disseminate, distribute, copy or in any manner rely upon the information contained in this message (directly or indirectly). The sender and/or the entity represented by the sender shall not be held accountable in the event that this prohibition is disregarded. If you receive this message in error, notify the sender immediately by e-mail, fax or telephone representations contained in this message, whether express or implied, are those of the sender only, unless that sender expressly states them to be the views or representations of an entity or person, who shall be named by the sender and who the sender shall state to represent. No liability shall otherwise attach to any other entity or person. ==========================================================