Datastage returns fake return code
Moderators: chulett, rschirm, roy
Datastage returns fake return code
Hi,
I am doing a project which requires me to process a number of files. However, this processing requires me to delete the input file one by one once it is processed. e.g.
I have input files: input1, input2, input3
After I process input1 using DS Job -wait -jobstatus
I delete input1 before processing input2 and so on
However, eventhough I have used "-wait", it seems sometimes the program delete the input1 even before DS Job starts running properly, thus will return an error code. The funny thing is, sometimes the error code is 2, sometimes it's 1 (which is successful). Thus, the next processing which is loading the data will fail because it can't find the output from the ds job. Anybody has experienced that problem before? This problem is quite intermitten, I tried to simulate it by writing down the steps, but it just does not work the same way everytime. Thanks a lot.
I am doing a project which requires me to process a number of files. However, this processing requires me to delete the input file one by one once it is processed. e.g.
I have input files: input1, input2, input3
After I process input1 using DS Job -wait -jobstatus
I delete input1 before processing input2 and so on
However, eventhough I have used "-wait", it seems sometimes the program delete the input1 even before DS Job starts running properly, thus will return an error code. The funny thing is, sometimes the error code is 2, sometimes it's 1 (which is successful). Thus, the next processing which is loading the data will fail because it can't find the output from the ds job. Anybody has experienced that problem before? This problem is quite intermitten, I tried to simulate it by writing down the steps, but it just does not work the same way everytime. Thanks a lot.
Hi,
You need to supply more info regarding your logic.
posting the batch file might help.
Also a bit more detailed info on the logic around/in the jobs you run.
You need to supply more info regarding your logic.
posting the batch file might help.
Also a bit more detailed info on the logic around/in the jobs you run.
Roy R.
Time is money but when you don't have money time is all you can afford.
Search before posting:)
Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Time is money but when you don't have money time is all you can afford.
Search before posting:)
Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Thanks...........
Code: Select all
Dim DSPROJ As String = "TEST_PROJECT"
Dim DSPWD As String = solutionPath + "ds_setting\ds_password.txt"
Dim DSSERVER_NAME As String = File.OpenText(solutionPath + "ds_setting\ds_server_name.txt").ReadLine()
'DSJob 's name
Dim JOBDS As String = groupType + recType + version
'DSJob 's command
Dim dsExec As String = ds_path
'Main filtering job
Dim dsParams As String = " -file " + DSPWD + " " + DSSERVER_NAME + " -run -jobstatus -wait -param INPUTFILE=" + fullPathInputFileName + " -param OUTPUTFILE=" + fullPathOutputFileName + filterConditions + " " + DSPROJ + " " + JOBDS
Dim dscommand As String = dsExec + dsParams
Console.WriteLine(dscommand)
Dim processInfo As New ProcessStartInfo
processInfo.FileName = dsExec
processInfo.Arguments = dsParams
processInfo.WindowStyle = ProcessWindowStyle.Hidden
Dim pr1 As Process = Process.Start(processInfo)
pr1.WaitForExit()
If pr1.ExitCode <> 1 Then
Dim processRestartInfo As New ProcessStartInfo
processRestartInfo.FileName = dsExec
processRestartInfo.Arguments = " -file " + DSPWD + " " + DSSERVER_NAME + " -run -mode RESET " + DSPROJ + " " + JOBDS
processRestartInfo.WindowStyle = ProcessWindowStyle.Hidden
Dim pr2 As Process = Process.Start(processRestartInfo)
pr2.WaitForExit()
fileInStream.Close()
Throw New Exception("DS Job failed, exit code = " + pr1.ExitCode.ToString)
Else
Thread.Sleep(60000)
File.Delete(fullPathInputFileName)
fileName = fileInStream.ReadLine()
End If
Last edited by cheerfu1 on Thu Aug 04, 2005 9:04 pm, edited 1 time in total.
Please do us all a favor and edit your post - there's a button to allow you to do that. I'm assuming your original code is properly indented - wrap all of it in Code tags so that it displays properly here as well.
Just makes it simpler all around.
Also, are there job parameters or arguments involved here - like, for example 'workid'. What sets that?
Just makes it simpler all around.
Also, are there job parameters or arguments involved here - like, for example 'workid'. What sets that?
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Hi Duke,kduke wrote:There is no -jobstatus in code posted. I also think that DataStage needs time to clean up between jobs. You need to sleep a few seconds between jobs.After I process input1 using DS Job -wait -jobstatus
I have let it sleep for a minute however the problem persists especially when processing bigger files.
jobstatus was intentionally removed. However, I have updated the code which includes the jobstatus with the same problem. Thanks for your replies. Please do advice me.
A couple of things to keep in mind...cheerfu1 wrote:jobstatus was intentionally removed. However, I have updated the code which includes the jobstatus with the same problem.
The -wait and -jobstatus commands are mutually exclusive as both tell it to wait before moving to the next step. You should use one of the other, not sure which takes precedence when you use both as they handle return codes differently.
If you are expecting a return code of 1 to indicate success, then you must use the -jobstatus option. When using -wait a zero return status indicates success and a non-zero return code indicates failure.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Thank you for your reply. I still can't solve the problem. However, I do a workaround by busy-waiting the output file, checking if the file is in use by another process (which is DS process). If it is, then I will keep looping. So far it works. But I do want a better solution as I am not sure whether this method will work for all occasions.chulett wrote:A couple of things to keep in mind...cheerfu1 wrote:jobstatus was intentionally removed. However, I have updated the code which includes the jobstatus with the same problem.
The -wait and -jobstatus commands are mutually exclusive as both tell it to wait before moving to the next step. You should use one of the other, not sure which takes precedence when you use both as they handle return codes differently.
If you are expecting a return code of 1 to indicate success, then you must use the -jobstatus option. When using -wait a zero return status indicates success and a non-zero return code indicates failure.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
it seems that the -wait parameter passed to the dsjob executable does not work properly arbitrarily. I am supposed to process the input file, then delete the input file, after the DS JOB completes. However, it deletes the input file first before the DSJOB starts properly, eventhough I have used -wait.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Don't know. As a workaround, wrap your job in a job sequence or in job control code, and invoke that from dsjob. The wait does work reliably there. You can manage the deletion from the same job sequence.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: