Memory Usage

[Coles]

Hi All,

We have encountered an issue with one of our Data Stage jobs in our Production environment...

The job does a looping in the transformer by comparing the date values between Start and End....

Ideally the start date should always be less than end date..

We have identified few instances in the data where the start date was greater than end date and this resulted a negative value in the loop variable...

Because of the negative value in the loop job was running longer and using all of the available memory on the server and eventually aborted which also caused every other job running on the server to abort...

Attached below is the error message

main_program: APT_PMsectionLeader(2, node2), player 4 - Unexpected termination by Unix signal 9(SIGKILL).

We fixed the issue by removing the error records...

My question around this is

Can some one pleas let us know if there is a way to restrict a job from using all of the available memory...

Regards
Coles

[ray.wurlod]

There is not, at least not within DataStage. DataStage jobs run as processes. There may be operating system tools that can restrict the memory demanded by (or supplied to) a particular process, but these are not part of DataStage.

Surely the better solution would be to detect the abnormal condition (start date later than end date) earlier in the processing, before traversing the loop.

[priyadarshikunal]

Can't check GUI but there should be a number to limit the number of loops. In your case, How many time you want the loop to run in-case it encounters a negative value? That being said, I would handle it in a case statement for 0 or -ve values for deciding on the number loops.

[atul9806]

Hi Coles
Would like to know what you are doing if loop variable is get negative value? I agree with priyadarshikunal, we should handle this by storing the loop variable value in another variable and if it get negative value use the backup value +1 else use loop variable