DSXchange

I am attempting to understand one of the sample jobs provided with Datastage and have come accross the folllowing code:

*Get the maximum length of LONG column data
*If $mam is nonzero, use it to determine array size
IF ($mam NE 0) THEN
ARG1 = $mam*1048576
OPENSEQ PARAMETERFILE TO PFILE ELSE CALL DSLogInfo("File not found ":PARAMETERFILE:"", "JobControl")
READSEQ ColLength FROM PFILE ELSE CALL DSLogInfo("Error reading ":PARAMETERFILE:"", "JobControl")
DATALENGTH = TRIMB(ColLength)

CLOSESEQ PFILE

* 0.3 is a number of factor if there are only one DRS as source stage and
* another DRS as target stage. With more than two DRS stages in the ETL job,
* smaller number should be considered.

MAXMEM = ARG1*0.3
* 6000 is for rows size of other columns
ROWSIZE = DATALENGTH + 6000
* If the maximum length of the long column data exceeds MAXMEM,
* MAXMEM will be the column precision for ETL job
IF (DATALENGTH > MAXMEM) THEN
DATALENGTH = MAXMEM
END

ARRAYSIZE = DIV(MAXMEM, ROWSIZE)

IF (ARRAYSIZE < 1) THEN
ARG1 = 1
END

*32767 is the biggest arraysize allowed
IF (ARRAYSIZE > 32767) THEN
ARG1 = 32767
END
IF(ARRAYSIZE >= 1 AND ARRAYSIZE <= 32767) THEN
ARG1 = ARRAYSIZE
END

ARG2 = DATALENGTH

I have searched the documentation for array size limitation, for either dynamic or dimensioned arrays and couldn't find anything.

Could the 32767 mentioned here be referring to a signed half word

(UCS-2 characters, C int16_t, Java char, Java short) ?

Would be grateful for any enlightenment here...Thanks

DS/Basic arrays can be dimensioned more than 32K. The code references "$mam" - what is that value at runtime?
Perhaps the array size referenced here is for your transfer buffer to/from the database.

ArndW wrote:DS/Basic arrays can be dimensioned more than 32K. The code references "$mam" - what is that value at runtime?
Perhaps the array size referenced here is for your transfer buffer to/from the database.

Thanks for your comment. I haven't tried running this so don't have any runtime values. I am just attempting to understand the significance of the array size limitation. I suspect it may be just a poorly worded comment for the code, and that the author probably meant to say something else.

ArndW wrote:Perhaps the array size referenced here is for your transfer buffer to/from the database.

Yes, this looks like some sort of custom algorithm used to set the Array Size option in a database stage, specifically the DRS stage from the comments. Nothing to do with BASIC arrays but that passive stage option has a maximum settable value of 32767.

chulett wrote:

ArndW wrote:Perhaps the array size referenced here is for your transfer buffer to/from the database.

Yes, this looks like some sort of custom algorithm used to set the Array Size option in a database stage, specifically the DRS stage from the comments. Nothing to do with BASIC arrays but that passive stage option has a maximum settable value of 32767.

Thanks for that. Sounds more like what is happening, as the passive stage option has a max settable value.

Many thanks....

Even though it's not related to this problem, DataStage BASIC dimensioned arrays do have an upper limit of 65535 elements.

Someone might be searching for this one day.

Dynamic arrays may have an arbitrary (and unlimited) number of elements, until you run out of memory.

ray.wurlod wrote:Even though it's not related to this problem, DataStage BASIC dimensioned arrays do have an upper limit of 65535 elements.

Someone might be searching for this one day.

Dynamic arrays may have an arbitrary (and unlimited) number of elements, until you run out of memory.

Thanks Ray, I didn't know that...