Reducing Joblogsize after Purge?

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
DSUser2000
Participant
Posts: 42
Joined: Tue Oct 20, 2009 8:36 am

Reducing Joblogsize after Purge?

Post by DSUser2000 »

Is it possible to reduce the size of the joblog after purging log entries (I've purged all entries older than 30 days)? Perhaps by some kind of Universe Database command like "Optimize Database" or so (I already know the "RT_LOGXXX" table which I need via uvsh)?
I know that it's possible to delete the job and create it new but I don't want to loose all entries...
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

They're Type30 dynamic hashed files, so that purge should automatically be reducing the log's size as well. Are you certain that it is not? :?
-craig

"You can never have too many knives" -- Logan Nine Fingers
DSUser2000
Participant
Posts: 42
Joined: Tue Oct 20, 2009 8:36 am

Post by DSUser2000 »

We've built a job which lists logfiles which get too huge. The job is a MultipleInstance job which got > 1GB. After deleting entries older than 30 days via the director, the size stayed almost exactly the same (the older log entries are really deleted): 1207MB before, 1206MB now. The deletions made should be definately more than 1MB...normally it should be about the half the size or smaller.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Multi-instance jobs are a little trickier - where did you do the purge from, the 'base' job or from one / all of the invocation ids? How long had it been since the last '30 days' purge?
-craig

"You can never have too many knives" -- Logan Nine Fingers
DSUser2000
Participant
Posts: 42
Joined: Tue Oct 20, 2009 8:36 am

Post by DSUser2000 »

I've done the purge from the base job. In the instance logs, the option to clear the log is greyed out...
The log exists since mid october (job was created then), so it's been 2 months now. Thus, I though that deleting 30 days would reduce the size by about the half...
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

True, forgot you can only do it from the main job. Did you get a count of log entries before and after? That should confirm the amount of records that were actually deleted.
-craig

"You can never have too many knives" -- Logan Nine Fingers
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Check out the RESIZE command. Not sure the purge affects the OVER.30 and DATA.30 goes down in Page Size increments but resize should get both, I do believe.
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Transactional DELETE does not free up disk space - it remains within the hashed file structure for re-use.

Commands CLEAR.FILE and RESIZE are non-transactional and do free up disk space. An appropriate syntax is

Code: Select all

RESIZE RT_LOGnnn * * *
This does not alter any of the hashed file's tuning parameters, but does free up disk space.

If disk space is at a premium in the project directory, add a USING clause to the RESIZE command to use a different file system as temporary workspace.

Code: Select all

RESIZE RT_LOGnnn * * * USING /usr/tmp
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
DSUser2000
Participant
Posts: 42
Joined: Tue Oct 20, 2009 8:36 am

Post by DSUser2000 »

uvsh "RESIZE RT_LOGxxxx *"
worked so far and recovered about 200MB.
Analyze.File before resize:
File name .................. RT_LOGxxxx
Pathname ................... RT_LOGxxxx
File type .................. DYNAMIC
NLS Character Set Mapping .. NONE
Hashing Algorithm .......... GENERAL
No. of groups (modulus) .... 6544 current ( minimum 1 )
Large record size .......... 1628 bytes
Group size ................. 2048 bytes
Load factors ............... 80% (split), 50% (merge) and 50% (actual)
Total size ................. 1265258496 bytes
After:
File name .................. RT_LOGxxxx
Pathname ................... RT_LOGxxxx
File type .................. DYNAMIC
NLS Character Set Mapping .. NONE
Hashing Algorithm .......... GENERAL
No. of groups (modulus) .... 4040 current ( minimum 1 )
Large record size .......... 1628 bytes
Group size ................. 2048 bytes
Load factors ............... 80% (split), 50% (merge) and 80% (actual)
Total size ................. 1088081920 bytes
I think the reason why not more was recovered is this load factor. I think I'll have to live with that and just delete more...
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

There should be three stars after the resize, not sure what difference it makes however. Please mark this as Resolved using the button at the top of the page.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply