DSXchange

Hi Friends,

I am facing this issue of Datastage server directory on UNIX "Ascential/Datastage" running out of space frequently. My system administrator keeps increasing the size of the directory but in a matter of 10 days the directory has grown from 8 GB to 11 GB in size!!! I am baffled because I cant understand the reason for this. The file sytem for writing the datafiles is completely different, so there is no chance of data files being written to datastage project directory.

I have also eneabled Auto purge job log feature in datastage admin. So the log files hould not create a prob either..

Can any of you shed some light on this peculiar issue? Your help will be highly appreciated.

Not sure but huge Hashed files??

Hi Deepak, as i told earlier the files are written to a completely different directory meant for storing all kinds of data and hashed files.. So why should the datastage direcoty get filled up?

Purge the log files periodically, you can set the options also from the Administrator of auto purge.

Are there any jobs running with large number of warnings, if yes, then try to modify the jobs in such a way that you don't get warnings at all.

loveojha2 wrote:Are there any jobs running with large number of warnings, if yes, then try to modify the jobs in such a way that you don't get warnings at all.

Hi Loveojha2,

As long as the Auto purge feature is enabled in ds admin, it dsnt matter even if the job gives warnings.. I have already tried all that..

Any other suggestions?

Check which files are relatively big in size and what is their type (log,RTBP,temp file)?

Just classify the space utilized by the different types of file, that may lead to some clue.

Adjust the size down for smaller than 2GB files.

loveojha2 wrote:Check which files are relatively big in size and what is their type (log,RTBP,temp file)?

Just classify the space utilized by the different types of file, that may lead to some clue.

There are a lot of 'TEMP*', 'RT_LOG*' and 'RT_STATUS*' files in the project directory, But I feel these files are generated internally by ds processes and are required.. Do you mean to say that I can DELETE these files?

Do you mean to say that I can DELETE these files?

No
Otherwise you will lose your project

loveojha2 wrote:

Do you mean to say that I can DELETE these files?

No
Otherwise you will lose your project

Then what are you exaclty suggesting...
I would appreciate if you be clear in your suggestions...

How many jobs are there in the project?
If they are not hugh in numbers.
My advise could be taking export of the entire project with the executable and creating a different project with that, then you can make a comparison actually, since you have said that you are auto purging the log and you are not maintaining Hashed Files in the project directory, so there should not be much of a difference.

loveojha2 wrote:How many jobs are there in the project?
If they are not hugh in numbers.
My advise could be taking export of the entire project with the executable and creating a different project with that, then you can make a comparison actually, since you have said that you are auto purging the log and you are not maintaining Hashed Files in the project directory, so there should not be much of a difference.

Ok.. can you tell me if it is usual for a project with approx 2000 jobs and sequences to occupy 11 GB of space????

Ok.. can you tell me if it is usual for a project with approx 2000 jobs and sequences to occupy 11 GB of space????

We have around 10K maps in one project with around 1 GB of space.

loveojha2 wrote:

Ok.. can you tell me if it is usual for a project with approx 2000 jobs and sequences to occupy 11 GB of space????

We have around 10K maps in one project with around 1 GB of space.

So you mean to say that it is normal?