Hi,
Job design includes perl program which calls two jobs in serial... First job reads couple of seq. files and Redbrick database tables ..creates output sequential files. These files are used as a input by second job. Second job reads these files and loads redbrick table. Perl program uses dsjob command to execute the job. Completion code is used to check the job execution.
my ($DsCommand) = "$DsPath/$DsJob -server aaa -user $DsUser -password $DsPassword -run -mode NORMAL -wait -jobstatus $ParamList $DsProject $JobName > $DwLog/$JobName.log";
system($DsCommand);
$RetValue = $? >> 8;
if ($RetValue == 1 ) {
return $RetValue;
}
else {
$Subject = "Job Failed.";
$Content = "return code $RetValue.\n";
$Content .= "Please check the logfile for further details \n";
SendMail ("pageoper","$Subject","$Content");
return $RetValue;
}
-----------------------
Jobstatus problem1:
Perl program starts Job1 execution and waits for the return code (job status). Job1 status is 'Running' in the director.... Output Files are also not populated completely. But, sometimes, perl routine returns jobstatus as 'Job Finished' and Job2 starts processing.... Job2 loads redbrick table with incorrect data because files are not populated completely by job1.
As observed, pattern is - This situation arises if DataStage server has been restarted before running the perl script...
Has anyone come across this kinda situation ? Is there anything wrong in the return code check routine ?
Problem 2:
Job log file shows fatal error and then 'job aborted' line but director job view page shows job status as 'Finished' instead of 'Aborted'.
Incorrect Job Status
Moderators: chulett, rschirm, roy
I'm a little lost here, but that's never stopped me before. 
You are saying that sometimes the second job is started before the first job actually finishes? Not sure how that would happen unless it is not truly waiting for the job to complete. For fun, what O/S are you on? Exact version of DataStage?
I can't check this right now, but I believe that you need to use either -wait or -jobstatus, not both. The former waits and returns 'normal' UNIX style return codes and only let's you know if it was able to start the job, while the later returns the actual DS job status codes you are expecting. It may be getting confused with both on the command line, I'd suggest trying just the -jobstatus option.
No clue on your second problem. Have you contacted Support and reported the issue? Perhaps it is a known issue or something that can be solved with an upgrade. (which you really should do, IMHO. You will love the improvements as you go from 5 to 6 to 7 - I guarantee!)
You are saying that sometimes the second job is started before the first job actually finishes? Not sure how that would happen unless it is not truly waiting for the job to complete. For fun, what O/S are you on? Exact version of DataStage?
I can't check this right now, but I believe that you need to use either -wait or -jobstatus, not both. The former waits and returns 'normal' UNIX style return codes and only let's you know if it was able to start the job, while the later returns the actual DS job status codes you are expecting. It may be getting confused with both on the command line, I'd suggest trying just the -jobstatus option.
I'm not sure what this means or how it would be a problem. Are you saying that you have problems during the first run after the Server has been restarted, but subsequent runs are ok?ANSHULA wrote:As observed, pattern is - This situation arises if DataStage server has been restarted before running the perl script...
No clue on your second problem. Have you contacted Support and reported the issue? Perhaps it is a known issue or something that can be solved with an upgrade. (which you really should do, IMHO. You will love the improvements as you go from 5 to 6 to 7 - I guarantee!)
There's not nearly enough information here to help you, nageshn. Please start a new thread so we can have your specifications (DS version, O/S, etc) and add as many details as you can about your job design and the issues you are having. Then we'll be able to help.nageshn wrote:We too face the similar problem, the job does not populate data into our database.(Oracle) though the job status is 'finished'.
When we try to fire the same query from SQL the data gets loaded.
Help please.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Hi,
my guess is your job crashed rather then finished ok.
(check your &PH& for crashed jobs)
I'd like to remind you of the 96 status - job crashed status that might seem as a normal ending to your job but is actually far from it.
this status was introduced in the 5 release and needs "attention".
so use DS Basic to get the actual job status after it finishes.
you can do 1 of the following:
1. rewrite the perl logic of checking DS job success and get the status via DS.
2. compleetly rewrite perl logic to basic control job.
if you must have perl script for starting DS then ony use it for that perpose.
(looking for that spell checker lol)
that's my 0.02 worth.
IHTH,
my guess is your job crashed rather then finished ok.
(check your &PH& for crashed jobs)
I'd like to remind you of the 96 status - job crashed status that might seem as a normal ending to your job but is actually far from it.
this status was introduced in the 5 release and needs "attention".
so use DS Basic to get the actual job status after it finishes.
you can do 1 of the following:
1. rewrite the perl logic of checking DS job success and get the status via DS.
2. compleetly rewrite perl logic to basic control job.
if you must have perl script for starting DS then ony use it for that perpose.
(looking for that spell checker lol)
that's my 0.02 worth.
IHTH,
Roy R.
Time is money but when you don't have money time is all you can afford.
Search before posting:)
Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Time is money but when you don't have money time is all you can afford.
Search before posting:)
Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
I have used Redbrick before and it does some tricky things. I am doing this from memory and no longer have access to Redbrick so here goes. I think you can tell Redbrick to fail if one record is bad or to ignore bad records. You need to examine the tmu file for the load.
On the load of job1 how do you know it failed? It sounds like you need a reject link and trap bad records. The next job may not run if you reject records which sounds like what you want to happen.
Just because all the records do not load does not mean the job failed. DataStage treats these as warnings in same cases. If a field will not accept nulls then DataStage will not load it. It will send it down the reject link if told to. DataStage is successfully processing everything you told it to and ignoring everything you told it to ignore. Does that make sense?
On the load of job1 how do you know it failed? It sounds like you need a reject link and trap bad records. The next job may not run if you reject records which sounds like what you want to happen.
Just because all the records do not load does not mean the job failed. DataStage treats these as warnings in same cases. If a field will not accept nulls then DataStage will not load it. It will send it down the reject link if told to. DataStage is successfully processing everything you told it to and ignoring everything you told it to ignore. Does that make sense?
Mamu Kim