how is sratch space calculated

goriparthi · Post by **goriparthi** » Mon Oct 23, 2006 7:07 am

Hi All,

I am running a parallel job with a remove duplicate stage on a input source of 5 columns. I ended up with error saying not enough space.

One of the column is varchar2(4000). If i calculate space needed with columns length as 4000 then the error is right, If i consider with the length of data in that column(around 20) then it is wrong.

Can anyone confirm if Datastage consider 4000 or the length of the data in it

thanks

ArndW · Post by **ArndW** » Mon Oct 23, 2006 7:12 am

When DataStage allocates structures and space it needs to use the defined maximum length. When it writes data to disk it will use the actual contents unless you specify fixed length. I think that the remove duplicates is allocating space based on the metadata and doing it's comparisons as if the string were fixed length CHARs, which explains your problem.