DataStage Basic Routine behaves different in 8.7...
Posted: Fri Jun 01, 2012 11:58 am
We have a routine (wrote in Basic) which works correct in 7.5 but not in 8.7. Are there any changes to the DataStage Basic in the newer version ? This rouutine reads a Master Mainframe EBCDIC file and breaks it into multiple files with base and riders.
We are in the process of upgrading from 7.5 Parallel to 8.7 and during the testing we found the routine creates files completely different from older version. Different meaning file size and even number of files it produces is different. Following is the routine.
Please let me know if you see something in this routine which will cause issues in new version.
Appreciate your time..
Thanks
We are in the process of upgrading from 7.5 Parallel to 8.7 and during the testing we found the routine creates files completely different from older version. Different meaning file size and even number of files it produces is different. Following is the routine.
Code: Select all
***DSLogInfo will display to Log in Director
Call DSLogInfo("Processing Started ":TIMEDATE(),"JobControl")
Call DSLogInfo("Process Parameters ":ProjectCategoryParmFile,"JobControl")
***Establish Parameter Values
prm_project_path =FIELD(ProjectCategoryParmFile,';',1,1)
prm_path_category=FIELD(ProjectCategoryParmFile,';',2,1)
prm_source =FIELD(ProjectCategoryParmFile,';',3,1)
****Exit on missing parameter value
var_tmp="Parameter Missing: "
IF prm_project_path="" OR ISNULL(prm_project_path)
THEN Call DSLogFatal(var_tmp:"Project Path","JobControl")
ELSE IF prm_path_category="" OR ISNULL(prm_path_category)
THEN Call DSLogFatal(var_tmp:"Category Path","JobControl")
ELSE IF prm_source="" OR ISNULL(prm_source)
THEN Call DSLogFatal(var_tmp:"Source File Name","JobControl");
****Read Parameter File
var_FileName=prm_project_path:prm_path_category:FIELD(prm_source,'.',1,1):"_MFP.prm"
OPENSEQ var_FileName TO var_InFile ELSE Call DSLogFatal(var_FileName:" Parameter File OPEN ERROR","JobControl")
var_EOF=0
LOOP
READSEQ var_InRec FROM var_InFile ELSE var_EOF=1
UNTIL var_EOF
IF var_InRec = "" OR var_InRec[1,1]="*" OR var_InRec[1,1]="#" ELSE
var_param=FIELD(var_InRec,'=',1,1)
var_value=FIELD(var_InRec,'=',2,1)
BEGIN CASE
CASE var_param = "prm_BaseLen"
prm_BaseLen = var_value
CASE var_param = "prm_BaseIdStart"
prm_BaseIdStart = var_value
CASE var_param = "prm_BaseIdLen"
prm_BaseIdLen = var_value
CASE var_param = "prm_KeyStart"
prm_KeyStart = var_value
CASE var_param = "prm_KeyLen"
prm_KeyLen = var_value
CASE var_param = "prm_SkipRecs"
prm_SkipRecs = var_value
CASE var_param = "prm_BaseIds"
prm_BaseIds = var_value
CASE var_param = "prm_TrailerIds"
prm_TrailerIds = var_value
CASE var_param = "prm_TrailerLens"
prm_TrailerLens = var_value
CASE var_param = "prm_TrailLenByteLen"
prm_TrailLenByteLen = var_value
CASE var_param = "prm_RecLen"
prm_RecLen = var_value
CASE 1
Call DSLogInfo("Unknown File Parameter ":var_param,"JobControl")
END CASE
END;
REPEAT
CLOSESEQ var_InFile
****Exit on missing or unassigned parameter value
var_tmp="Parameter is Missing from Parameter File: "
IF UNASSIGNED(prm_BaseLen) OR prm_BaseLen=""
THEN Call DSLogFatal(var_tmp:"prm_BaseLen","JobControl")
ELSE IF UNASSIGNED(prm_BaseIdStart) OR prm_BaseIdStart=""
THEN Call DSLogFatal(var_tmp:"prm_BaseIdStart","JobControl")
ELSE IF UNASSIGNED(prm_BaseIdLen) OR prm_BaseIdLen=""
THEN Call DSLogFatal(var_tmp:"prm_BaseIdLen","JobControl")
ELSE IF UNASSIGNED(prm_KeyStart) OR prm_KeyStart=""
THEN Call DSLogFatal(var_tmp:"prm_KeyStart","JobControl")
ELSE IF UNASSIGNED(prm_KeyLen) OR prm_KeyLen=""
THEN Call DSLogFatal(var_tmp:"prm_KeyLen","JobControl")
ELSE IF UNASSIGNED(prm_SkipRecs) OR prm_SkipRecs=""
THEN Call DSLogFatal(var_tmp:"prm_SkipRecs","JobControl")
ELSE IF UNASSIGNED(prm_BaseIds) OR prm_BaseIds=""
THEN Call DSLogFatal(var_tmp:"prm_BaseIds","JobControl")
ELSE IF UNASSIGNED(prm_TrailerIds) OR prm_TrailerIds=""
THEN Call DSLogFatal(var_tmp:"prm_TrailerIds","JobControl")
ELSE IF UNASSIGNED(prm_TrailerLens) OR prm_TrailerLens=""
THEN Call DSLogFatal(var_tmp:"prm_TrailerLens","JobControl")
ELSE IF UNASSIGNED(prm_TrailLenByteLen) OR prm_TrailLenByteLen=""
THEN Call DSLogFatal(var_tmp:"prm_TrailLenByteLen","JobControl")
ELSE IF UNASSIGNED(prm_RecLen)
THEN prm_RecLen=0;
***If prm_RecLen is zero then use Record Descriptor Word(RDW) else process as fixed width file.
var_RDW = (prm_RecLen=0)
***Add CR/LF (Windows) or LF (UNIX) length for fixed length format file.
var_PhyRecLen=prm_RecLen + IF (System(91)) THEN 2 ELSE 1
***Open Input File
var_FileName=prm_project_path:prm_path_category:prm_source
OPENSEQ var_FileName TO var_InFile ELSE Call DSLogFatal(var_FileName:" OPEN ERROR","JobControl")
***Skip Input File Records (prm_SkipRecs)
var_EOF=0
var_ReadCNT=0
LOOP WHILE var_ReadCNT < prm_SkipRecs AND NOT(var_EOF)
****First two bytes for RDW are the RDW (length) in binary
IF var_RDW THEN
GET RDW,2 FROM var_InFile ELSE var_EOF=1
*****Binary conversion
var_RecLen = (SEQ(RDW[1,1]) * 256) + SEQ(RDW[2,1])
*****Second two bytes could store extra data for the file
GET var_ExtraData,2 FROM var_InFile ELSE var_EOF=1
*****Subtract 4 for the RDW on VB files
var_RecLen = var_RecLen - 4
END
ELSE
var_RecLen = var_PhyRecLen
END
GET INRECORD,var_RecLen FROM var_InFile ELSE var_EOF=1
var_ReadCNT += 1
REPEAT
var_SkipCNT = var_ReadCNT
***Determine File Info Array Size
***Base file info is in array dimension 1 and trailer info is in 2
DIM var_FileCNT(2)
var_MaxCNT=0
FOR I = 1 TO 2
var_FileCNT(I) = IF I=1 THEN DCOUNT(prm_BaseIds,';') ELSE DCOUNT(prm_TrailerIds,';')
IF var_FileCNT(I)=0 THEN var_FileCNT(I)=1
IF var_FileCNT(I) > var_MaxCNT THEN var_MaxCNT = var_FileCNT(I)
NEXT I
***Dimension File Handle & Count Arrays
DIM var_FileHandle(2,var_MaxCNT), var_RecCNT(2,var_MaxCNT)
MAT var_RecCNT=0
***Establish File Names & Handles Values
var_FileName = prm_project_path:prm_path_category:prm_source
FOR I = 1 TO 2
var_FNPrefix = FIELD(var_FileName,'.',1,1):IF I=1 THEN "_Base" ELSE "_Trail"
var_FNSuffix = FIELD(var_FileName,'.',2,1)
var_FileIds = IF I=1 THEN prm_BaseIds ELSE prm_TrailerIds
FOR J=1 TO var_FileCNT(I)
var_FNFull = var_FNPrefix:FIELD(FIELD(var_FileIds,';',J,1),',',1,1):'.':var_FNSuffix
Call DSLogInfo("File Name(":I:",":J:") is ":var_FNFull,"JobControl")
OPENSEQ var_FNFull TO var_FileHandle(I,J) THEN NAP 2
NEXT J
NEXT I
***Create Array to work with FIND statement
var_BaseIds = CONVERT(';,',@FM:@VM,prm_BaseIds)
var_TrailIds = CONVERT(';,',@FM:@VM,prm_TrailerIds)
***Loop through Input File
var_EOF=0
LOOP
IF var_RDW THEN
****First two bytes for RDW are the RDW (length) in binary
GET RDW,2 FROM var_InFile ELSE var_EOF=1
*****Binary conversion
var_RecLen = (SEQ(RDW[1,1]) * 256) + SEQ(RDW[2,1])
*****Second two bytes could store extra data for the file
GET var_ExtraData,2 FROM var_InFile ELSE var_EOF=1
*****Subtract 4 for the RDW on VB files
var_RecLen= var_RecLen - 4
END
ELSE
var_RecLen= var_PhyRecLen
END
GET var_InRec,var_RecLen FROM var_InFile ELSE var_EOF=1
UNTIL var_EOF
IF Not(var_RDW) THEN var_RecLen=prm_RecLen
***Capture Base Information
var_ReadCNT += 1
var_BaseRec = var_InRec[1, prm_BaseLen]
var_BaseKey = var_InRec[prm_KeyStart, prm_KeyLen]
var_BaseNum = 1
IF prm_BaseIdStart # 0 THEN
var_BaseId = ASCII(var_InRec[prm_BaseIdStart,prm_BaseIdLen])
FIND var_BaseId IN var_BaseIds SETTING var_BaseNum
ELSE var_BaseNum=var_FileCNT(1)
END
WRITESEQ var_BaseRec TO var_FileHandle(1,var_BaseNum)
ELSE Call DSLogFatal("Write Base ":var_BaseId:" Failed","JobControl")
var_RecCNT(1,var_BaseNum) += 1
!For Debugging
!var_EOF = (var_RecCNT(1,var_BaseNum) > 9)
var_CurPos = prm_BaseLen + 1
***Loop through Trailers
LOOP WHILE var_CurPos < var_RecLen
var_TrlLen = 0
FOR I=0 TO (prm_TrailLenByteLen - 1)
var_TrlLen += SEQ(var_InRec[var_CurPos + I, 1])
NEXT I
BEGIN CASE
CASE var_TrlLen > 0 AND ((var_CurPos + var_TrlLen - 1) <= var_RecLen)
***In case of a valid Trailer
var_TrailRec = var_InRec[var_CurPos + prm_TrailLenByteLen, var_TrlLen - prm_TrailLenByteLen]
IF var_TrailRec # EBCDIC(STR(' ',var_TrlLen-prm_TrailLenByteLen)) THEN
var_TrailId = ASCII(var_TrailRec[1,1])
FIND var_TrailId IN var_TrailIds SETTING var_TrailNum
ELSE
var_TrailId = ASCII(var_TrailRec[1,2])
FIND var_TrailId IN var_TrailIds SETTING var_TrailNum
ELSE
var_TrailNum=var_FileCNT(2)
END
END
var_TrailRec = var_BaseKey:var_TrailRec
var_PadLen = FIELD(prm_TrailerLens,';',var_TrailNum,1) - LEN(var_TrailRec)
var_TrailRec := EBCDIC(STR(' ',var_PadLen))
WRITESEQ var_TrailRec TO var_FileHandle(2,var_TrailNum)
ELSE Call DSLogFatal("Write Trailer ":var_TrailId:" Failed","JobControl")
var_RecCNT(2,var_TrailNum) += 1
END
var_CurPos += var_TrlLen
CASE 1
var_CurPos = var_RecLen
END CASE
REPEAT
REPEAT
***Shutdown Process & Display Information
Call DSLogInfo("Processing Complete ":TIMEDATE(),"JobControl")
Call DSLogInfo("Read Count is ":var_ReadCNT,"JobControl")
Call DSLogInfo("Skip Count is ":var_SkipCNT,"JobControl")
FOR I=1 TO 2
FOR J=1 TO var_FileCNT(I)
CLOSESEQ var_FileHandle(I,J)
var_Msg = IF I=1 THEN "Base" ELSE "Trailer":" "
var_Msg := FIELD(FIELD(IF I=1 THEN var_BaseIds ELSE var_TrailIds,@FM,J,1),@VM,1,1)
var_Msg := " Count is ":var_RecCNT(I,J)
Call DSLogInfo(var_Msg,"JobControl")
NEXT J
NEXT I
CLOSESEQ var_InFile
ErrorCode = 0 ;* set this to non-zero to stop the stage/job
Appreciate your time..
Thanks