The number of logs for a mult-instantiated parallel job

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

aluthra48
Participant
Posts: 66
Joined: Thu Mar 05, 2009 9:59 am

The number of logs for a mult-instantiated parallel job

Post by aluthra48 »

Hello,

I have a multi-instantiated parallel job that is executed repeatedly in many sequence jobs. The problem I have is that I seem to be able to retain the logs for the latest 6 executions. That is, if there are 10 sequence jobs, each calling the multi-instantiated job once, I expect to see 10 logs, one for each invocation id. Yet I see only the latest 6.

Is there a limit that needs to be set? If so, how?

Thanks
nagarjuna
Premium Member
Premium Member
Posts: 533
Joined: Fri Jun 27, 2008 9:11 pm
Location: Chicago

Post by nagarjuna »

Is invocation ID different for all 10 jobs ??
Nag
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

You most likely have autopurge settings for this job. Go to the Director -> Clear Log and set the Autopurge to none.
aluthra48
Participant
Posts: 66
Joined: Thu Mar 05, 2009 9:59 am

Post by aluthra48 »

Yes, each as a different invocation id.

As suggested, I plan to view log for the multi-instantiated job, check auto-purge and "auto purge enabled for this job" and select 25 for "Up to previous (job runs)", as it requires a number from 1 thru 25.

I assume this is what the suggestion implied.

I will report back with my results in the next 1 hour.

Thanks
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Multi-instance jobs all share the same log and each individual invocation id run has (in essence) a view into the main log which filters out just their entries. So you need to take that into account when setting auto-purge for them.
-craig

"You can never have too many knives" -- Logan Nine Fingers
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

It would be best to turn off autopurge completely for the job.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Because of the associated bug related to that in 8.x, I assume?
-craig

"You can never have too many knives" -- Logan Nine Fingers
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Craig - Yes. I should have explained it. There is a sporadic issue with enabled auto-purge at 8.x which causes jobs to abort. We have 4 multiinstance jobs, each running with several instances and the jobs start once a minute or so and sometimes nothing (bad) happens for a day or two, but once issues start they don't disappear until the logs are either cleared or purge is turned off.
priyadarshikunal
Premium Member
Premium Member
Posts: 1735
Joined: Thu Mar 01, 2007 5:44 am
Location: Troy, MI

Post by priyadarshikunal »

This bug was reported for 8.0 and I recall them addressing it in FixPack 2. Does that problem exists in later releases?
Priyadarshi Kunal

Genius may have its limitations, but stupidity is not thus handicapped. :wink:
aluthra48
Participant
Posts: 66
Joined: Thu Mar 05, 2009 9:59 am

Post by aluthra48 »

In order to disable auto-purge, I selected "view log", then Job====>clear log. I checked "Auto-purge" and unchecked "Auto purge enabled for this job".

However, after saving, I return to clear log, and I see "Immediate purge" still checked.

I can do the same, without going to view log. Just highlight the job and select clear log from the job tab. When I return to clear log, I find "Immediate purge" still checked.

What could I be doing wrong?

Thanks
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

That's normal so nothing wrong at all. Immediate Purge is the default when doing this manually.
-craig

"You can never have too many knives" -- Logan Nine Fingers
aluthra48
Participant
Posts: 66
Joined: Thu Mar 05, 2009 9:59 am

Post by aluthra48 »

Thanks.

I will report back soon.
aluthra48
Participant
Posts: 66
Joined: Thu Mar 05, 2009 9:59 am

Post by aluthra48 »

Hello,

That solution did not work. What was curious though, was that on some occassions, as I went about disabling auto-auto purge, it would return with an error message "Unable to return an attached job, Cannot open file RT_LOG". I assume it is due to the log being locked at that point in time, though I am not sure, as at other times it has saved the settings with no such error message.

Anyhow, the results are negative.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Did not work in what sense? What exact problem are you trying to solve again? Your RT_LOG issue sounds more like a corruption issue with the log hashed files themselves. I would suggest exporting and reimporting any job that does that to fix things of that nature.
-craig

"You can never have too many knives" -- Logan Nine Fingers
aluthra48
Participant
Posts: 66
Joined: Thu Mar 05, 2009 9:59 am

Post by aluthra48 »

Sorry that I didn't quite explain myself.

In disabling the auto-purge, I occassionally got RT LOG error I mentioned. On attempting to do so again, it would apply the change without giving any errors.

Once it did not give any errors, I assumed the auto-purge had been disabled. Then I ran the sequence jobs that call the multi-instantiated job, expecting to see all 10 (or more) logs, one per invocation id.

When I said that the results were negative, I meant that I did not see as many logs as the invocation id's.

Thanks
Post Reply