Hi All,
Pls clarify me the difference between Scratch Space and Temporary Storage Space. Does Temporary Space includes scratch space? Or they are mutually exculsive?
I think we have to specify scratch space for each node. But does this mean that that scratch space will be present in Temporary Storage space?
Thanks in Advance for your help.
Regards
Pinkesh
Are Scratch Space and Temporary Space Different
Moderators: chulett, rschirm, roy
-
ray.wurlod
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Can you provide a more exact context? The terms can be equivalent, but in some specific circumstances they can mean different things. For one thing, "scratch space" may been that memory has been allocated, rather than disk, for example for data type conversions.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Re: Are Scratch Space and Temporary Space Different
For PX, we have several locations - Scratch, Datasets, and Temp.
Scratch spaces are utilized by the configuration files, along with Datasets, for typical PX processing. It is technically temporary space, but also contains output information produced by some stages such as Oracle Stage (This is an excellent place to dig around if your Oracle Stage give you an unclear error). This is also where Tsort place the actual data files during its sorting. All kind of stuff goes in there, and can be very useful during a failure's diagnosis process.
Datasets space is to be considered more or less stable/permanent - it is where all the actual data for datasets are placed during runs, including internally created datasets. This is what you may be referring to as Temporary Storage Space.
Temp spaces are used by Orchestrate for the actual storage of data, including the header files for temporary datasets.
Configurations are obviously different for each client. The way I am currently setting up the spaces are to provide at least 1 Scratch and 1 Dataset space per node. No fancy specific sort spaces, or anything else (since all of our drives are very uniform in performance, and are really all part of a massive RAID system). If you have any further configuration file questions, let us know.
There is a lot more to this, but these probably should answer your questions.
-T.J.
Scratch spaces are utilized by the configuration files, along with Datasets, for typical PX processing. It is technically temporary space, but also contains output information produced by some stages such as Oracle Stage (This is an excellent place to dig around if your Oracle Stage give you an unclear error). This is also where Tsort place the actual data files during its sorting. All kind of stuff goes in there, and can be very useful during a failure's diagnosis process.
Datasets space is to be considered more or less stable/permanent - it is where all the actual data for datasets are placed during runs, including internally created datasets. This is what you may be referring to as Temporary Storage Space.
Temp spaces are used by Orchestrate for the actual storage of data, including the header files for temporary datasets.
Configurations are obviously different for each client. The way I am currently setting up the spaces are to provide at least 1 Scratch and 1 Dataset space per node. No fancy specific sort spaces, or anything else (since all of our drives are very uniform in performance, and are really all part of a massive RAID system). If you have any further configuration file questions, let us know.
There is a lot more to this, but these probably should answer your questions.
-T.J.
Developer of DataStage Parallel Engine (Orchestrate).