CloseSeq not releasing a file

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
johnno
Participant
Posts: 50
Joined: Wed Mar 05, 2003 5:33 am

CloseSeq not releasing a file

Post by johnno »

Hi all,

I have been trawling through the forum looking for a possible solution to my problem and am not sure whether or not I have found it. I have a DS Basic job that reads a sequential parameter file and then submits a sequence using these parameters. Immediately after I finish reading the file I do a CloseSeq on the file so other jobs can use it. We have experienced a number of instances where a process that takes some time to run seems to hold the parameter file thereby stopping other processes from using it.

The manual says this statement resets the update lock applied by the OpenSeq statement, so I assume that this means as soon as the CloseSeq statement has been executed other jobs should be able to access the file.

After looking at a number of posts, I notice that me Open/Read/Close structure is not what seems to be standard practice, but will take that in my stride if anyone can shine light on the problem (maybe this is the problem?!). For convenience I have supplied a copy of the code below (sorry about the format, hope it is readable):

$INCLUDE DSINCLUDE JOBCONTROL.H

Ans = 0

* Set default UserStatus to 'Fail' (9). This will be set to 'Success' (0) at the end of this job

Call DSSetUserStatus(9)

* Get the jobname into a variable to be used when writing to the job log. This saves time by not having to change
* each occurrence of the jobname in this code.

svJobName = DSGetJobInfo(DSJ.ME, DSJ.JOBNAME)

* Set job variables to check whether all necessary parameters have been passed

ReadyParmPassed = @False
ProcessedParmPassed = @False
QuarantineParmPassed = @False
ReportParmPassed = @False
TempParmPassed = @False
DSNParmPassed = @False
EmailRecipientParmPassed = @False
EmailSenderParmPassed = @False
EmailServerParmPassed = @False

* Open sequential file containing parameters for this process

Call DSLogInfo("Reading parameters from " : pParamFile, svJobName)

OpenSeq pParamFile To ParmFile Then
Call DSLogInfo("Reading parameters from " : pParamFile, svJobName)
End Else
* File cannot be accessed or does not exist
Call DSLogInfo("*** Unable to open file: " : pParamFile, svJobName)
GoTo EXITJOB
End

* Create loop to read each row of the parameter file

Loop

* Read a row from the parameter file

ReadSeq ParmRow From ParmFile
On Error
Call DSLogInfo("*** Unable to read parameter file: " : pParamFile, svJobName)
GoTo EXITJOB
End Then

* Split out parameter values

Format = Trim(Field(ParmRow, "|", 1))
ParmName = Trim(Field(ParmRow, "|", 2))
ParmValue = Trim(Field(ParmRow, "|", 3))

If Format = "ALL" Then

Begin Case

Case ParmName = "pInputPath"
pInputPath = ParmValue : "Selector\Ready\"
ReadyParmPassed = @True

Case ParmName = "pProcessedPath"
pProcessedPath = ParmValue : "Selector\Processed\"
ProcessedParmPassed = @True

Case ParmName = "pQuarantinePath"
pQuarantinePath = ParmValue : "Selector\Quarantine\"
QuarantineParmPassed = @True

Case ParmName = "pReportPath"
pReportPath = ParmValue : "Selector\Reports\"
ReportParmPassed = @True

Case ParmName = "pTempPath"
pTempPath = ParmValue : "Selector\Temp\"
TempParmPassed = @True

Case ParmName = "pDSN"
pDSN = ParmValue
DSNParmPassed = @True

Case ParmName = "pEmailRecipient"
pEmailRecipient = ParmValue
EmailRecipientParmPassed = @True

Case ParmName = "pEmailSender"
pEmailSender = ParmValue
EmailSenderParmPassed = @True

Case ParmName = "pEmailServer"
pEmailServer = ParmValue
EmailServerParmPassed = @True

Case ParmName = "pWildCard"
pWildCard = ParmValue
WildCardParmPassed = @True

End Case

End

End Else

Exit ; * end of file

End

Repeat

* Close parm file so that it is not held by Selector and stops other jobs from running

CloseSeq ParmFile

* Prepare to call the first sequence for this format

* Populate static parameters particular to Selector:

pListFile = "SELFilesToProcess.txt"

* Give the sequence a handle and check that the Attach worked OK

If ReadyParmPassed AND ProcessedParmPassed AND QuarantineParmPassed AND ReportParmPassed AND TempParmPassed AND DSNParmPassed AND EmailRecipientParmPassed AND EmailSenderParmPassed AND EmailServerParmPassed AND WildCardParmPassed Then

hStartProcess = DSAttachJob("SELSequence010", DSJ.ERRFATAL)

If NOT(hStartProcess) Then
Call DSLogInfo("*** Unable to attach SELSequence010", svJobName)
GoTo EXITJOB
End Else

* Set up the parameters

ErrCode = DSSetParam(hStartProcess, "pInputPath", pInputPath)
ErrCode = DSSetParam(hStartProcess, "pWildCard", pWildCard)
ErrCode = DSSetParam(hStartProcess, "pTempPath", pTempPath)
ErrCode = DSSetParam(hStartProcess, "pListFile", pListFile)
ErrCode = DSSetParam(hStartProcess, "pReportPath", pReportPath)
ErrCode = DSSetParam(hStartProcess, "pProcessedPath", pProcessedPath)
ErrCode = DSSetParam(hStartProcess, "pQuarantinePath", pQuarantinePath)
ErrCode = DSSetParam(hStartProcess, "pEmailRecipient", pEmailRecipient)
ErrCode = DSSetParam(hStartProcess, "pEmailSender", pEmailSender)
ErrCode = DSSetParam(hStartProcess, "pEmailServer", pEmailServer)
ErrCode = DSSetParam(hStartProcess, "pDSN", pDSN)
ErrCode = DSSetParam(hStartProcess, "pUserID", pUserID)
ErrCode = DSSetParam(hStartProcess, "pPassword", pPassword)

* Run the job, wait for it to finish and then get it's status

ErrCode = DSRunJob(hStartProcess, DSJ.RUNNORMAL)
ErrCode = DSWaitForJob(hStartProcess)
svJobStatus = DSGetJobInfo(hStartProcess, DSJ.JOBSTATUS)
svUserStatus = DSGetJobInfo(hStartProcess, DSJ.USERSTATUS)

* Use the UserStatus from the above sequence to determine the succsess or failure of the process

Begin Case

Case svUserStatus = 0 OR svUserStatus = "0"
Call DSLogInfo("Selector process completed successfully", svJobName)
CALL DSSetUserStatus(0)
Ans = 0

Case svUserStatus = 2 OR svUserStatus = "2"
svFailedFile = @TRUE
CALL DSSetUserStatus(2)
Call DSLogInfo("*** One or many files may have had reports generated - Buiness area to investigate", svJobName)
Ans = 2

Case svUserStatus = 9 OR svUserStatus = "9"
CALL DSSetUserStatus(9)
Call DSLogInfo("*** Fatal error in lower level sequence", svJobName)
Ans = 9

Case @True
CALL DSSetUserStatus(9)
Ans = 9
Call DSLogInfo("*** Invalid Userstatus from SELSequence010 - " : svUserStatus, svJobName)

End Case

End

End Else
Call DSLogInfo("*** Selector - Fatal Error - Mandatory parameters not retrieved from the parameter file ", svJobName)
CALL DSSetUserStatus(9)
Ans = 9
End

EXITJOB:
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

Consider using a DSExecute statement to "cat" or "type" a file into the screenoutput variable. It's a whole lot easier than reading a file, plus there's no pesky locks involved.

UNIXcmd = "cat ":YourFilePath:"/yourfile.txt"
CALL DSExecute("UNIX", UNIXcmd, ScreenOutput, ReturnCode)

Now, the screen output is in a dynamic array variable ScreenOutput. So, do a DCOUNT(ScreenOutput, @AM) to get the number of lines (rows). Setup a loop and scan the data and do what you will.

No locks, no mess.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

The lock is set even if the ELSE clause is taken in OpenSeq, as you may be opening a non-existent file for writing. Your GoTo EXITJOB statement bypasses your CloseSeq statement, so the lock would not be released in this case.

You should also code ON ERROR and LOCKED clauses for your OpenSeq statements.

Even better is to use the OpenSeqFile function I posted some time ago to the downloads area at ADN; then you don't have to repeat the detailed coding for every routine you write that works with text files.

Ken's suggestion about capturing the output of cat or type is OK, or course the command in your case (on Windows) is typeand the shell argument is "DOS". Also, I never count the lines in the output (it's an unnecessary pass through the data); I use REMOVE to process them, or know the position in advance.

Code: Select all

LineCount = 0
Loop
Remove Line From Output Setting MoreLines
   LineCount += 1
   GoSub ProcessLine
While MoreLines
Repeat
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
johnno
Participant
Posts: 50
Joined: Wed Mar 05, 2003 5:33 am

Post by johnno »

Sorry for not getting back to you guys earlier. Thanks for the replies but unforunately, we have tracked down the problem to new versions of routines not overwriting existing versions in our production environment.

I have posted another query re this issue.

Thanks again for your help

Cheers
Johnno
Post Reply