Hello there...
Well I have this job running at the client end and it processes real heavy gusts of data...
Now some time back the jobs fails saying "APT_CombinedOperatorController,0: write failed: Output file full, and no more output files
write failed: Output file full, and no more output files"
Now This message says that Output file is full but this does not indicate that whether it is Datasets or Some Temporary Files being full.
Doubt arises when we see the error message in the next run
APT_CombinedOperatorController,0: Fatal Error: Tsort merger aborting: Scratch space full
(there is some sorting logic in the job)
1)Now I need to know what is the Scratch Space
2) Also when I see the Environment settings of my project I see that
something called as resource disk also being used.
Can you tell me what exactly are Scratch Disk and the Resource Disk.
Also there is an all together different temporary directory being used in Datastage in the variable TMPDIR
This TMPDIR is not specific to our project it is standard to Datastage.
and can bee seen in the administrator's general Settings Tag
So if this TMPDIR is there then what is the need of Scratch Space to be used
[If my assumption that scratch space is a temporary space used while executing a job is correct ]
Also I tried to run the job again...
The job is failing again saying that "APT_CombinedOperatorController,0: write failed: Output file full, and no more output files"
But when I check the corresponding UNIX directory I see only that 15% of the total disk space in TMPDIR location is used.
So Is there any Settings in datastage which limits the usage of the TMPDIR space.
Please let me know it is very important for me to know this to get out of the quagmire I'm currently trapped in
First off, disable operator combination so you can work out which operator is having the problem.
Resource disk is used only for Data Sets and File Sets. If it turns out that the stage having the problem is a Data Set stage, a File Set stage or a Lookup File Set stage, then you're close to a solution.
If it's any other stage then it's probably scratch disk that is filling. You have to monitor this while the job is running, not once it's finished.
TMPDIR is not used until and unless scratch disk fills.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Operator combination is an optimization whereby multiple adjacent parallel operators are caused to execute in the same process, provided certain conditions are met.
Disable it either using the APT_DISABLE_COMBINATION environment variable or setting the combination setting in individual stages' Advanced tab to "Don't combine".
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.