Page 1 of 1
internal storage
Posted: Wed Feb 27, 2008 11:19 pm
by mcs_suman
Does data stage has internal storage place to store data.
Posted: Wed Feb 27, 2008 11:28 pm
by Maveric
On a temporary basis, yes. Datasets. That will be stored in DataStage internal format, in the resource disc. They are the most efficient way in terms of space, read/write efficiency to store data. You also have hashed files, file sets and sequential files.
Posted: Wed Feb 27, 2008 11:36 pm
by ray.wurlod
In parallel jobs you don't really have hashed files readily available.
Posted: Thu Feb 28, 2008 12:35 am
by mcs_suman
could you explain me about data stage internal format.
Posted: Thu Feb 28, 2008 12:42 am
by ray.wurlod
As much as possible, in parallel jobs, we strive not to use internal storage (that is, not to touch data down to disk), as this slows things down. The structure of data in memory is given by the record schema that you can see in the generated OSH and the score. The same structure is used in Data Sets. Numeric values are stored in binary formats, while variable-length strings have a prefix showing how many characters there are in the string. If NLS is enabled internal storage is Unicode, if NLS is not enabled internal storage is "eight-bit ASCII".