Aborting a Job based on number of rejects.

[thebird]

Hi,

I need to abort my job as soon the number of rejected records equals a threshold value which is passed as a parameter to the job. Can this be done?

There are different levels of validations that need to be performed and at each validation stage there can be rejects which are collected in a reject file.

Suppose the threshold limit is set to 5, then the job should aboort as soon as an error is encountered in the 6th error record. The 6th error shouldn't come down the reject link.

How can the job be aborted at this stage?

Any help regarding this would be appreciated. Please note that this is an urgent requirement.

Thanks in advance.

Regards,

Thebird.

[ArndW]

thebird,

if you look at the constraints of your transform stage you will see a column informatively labeled "abort after rows" where you can specify a value. The default is 0.

You can also use the internal variables @INROWNUM and @OUTROWNUM to put in your own constraints based on number of rows per link.

[kris]

I need to abort my job as soon the number of rejected records equals a threshold value which is passed as a parameter to the job. Can this be done?

Bird,

There are several ways you can do this. If the threshold is definite value for all the jobs then you can pass it as a parameter to job and use in constraint or hardcode in your job.

But If you have to calculate your treshold (example 5% of source records) at run time during the process and it can change to each run, My approach to this:

1. Calculate the threshold upfront and pass it to the job.(Ex: If it is a batch job and the source is sequential file, write a routine to get the number of rows and calculate the threshold and assign it to a variable. Now you can pass this variable as a parameter to the job and use it in the code to abort the job.

2. If the source is table, routine would connect to database and get the count(*) of your selection criateria and calculates the threshold and pass it to the job.

3. If you are running your jobs thru shell scripts, then you have a better hold of what you need in the shell, you can define userdefined environment variable in the job and calculate the threshold in your script and store it in a local shell variable and pass it thru env variable to the job.

If you want to know how to use environment variable, serach this form...there is plenty of info.

Thanks,
Kris~

[thebird]

Hi,

What I would really like to know is whether there is a function call like the DSJobLogFatal in server edition, here.

The threshold value is to be passed through a parameter, but how is the job made to abort?

Regards,

Thebird.

[ArndW]

TheBird,

have you tried using a constraint with a condition along the lines of @OUTROWNUM<ThreshholdParameter?

(fixed #<Parameter># error noted by Craig)

[chulett]

Actually, just to nit pick:

Constraints are one of the odd places where you don't wrapper a job parameter with pound signs. So, the constraint would be

Code: Select all

@OUTROWNUM < ThresholdParameter

And that won't abort anything, just stop any further rows from going down the link.

So I take it that the PX version of the Transformer doesn't have the 'Abort after X rows' constraint functionality?

[ArndW]

Craig,

don't worry - the Px transformer has the same output constraint with number of rows as does the Server.

[thebird]

Hi,

Actuallly, the job has validations at different levels ie, :

1. A validation before a column import stage.( the source is in header-detail-trailer format.)

I am reading the data as one string and then splitting the columns in the column import stage.

2. Another validation after the columns have been formed, ie, after the column import stage.

3. A set of look-ups after the above validations, which can also send out rejects.

At all these levels there are rejects.

Suppose the threshold parameter is set to 5, I want the job to abort, when the sixth error is encountered, ie, before it is written to the reject file.

How can this be done, as the rejects are being caught in separate reject files and are then collected into a single one?

I guess I would have to keep a reject count starting from the first level of validation and which is then passed over to all other validations and look ups.

Am I right? If so, how can the count of rejects be taken at each level of validation?

Thanks.

Regards,

Aneesh Menon.

[ray.wurlod]

Even setting a threshhold of 5 will surely apply separately to each processing node?

[get2bhavesh]

there is routine available in the Routines/sdk/JobAbortUtility. Try out using that routine in one of the contraints in the transformer.