Issues with sequential file stages

sweetleaf · Post by **sweetleaf** » Mon May 26, 2003 8:57 pm

Hi,

I have a DS problem I am hoping someone might be able to shed some light on.

BACKGROUND:
I have a DS job (A) which calls 2 other DS jobs (B,C). B and C
are jobs which read the contents of 2 designated (and seperate) folders (folders "1" and "2") on the network. Both jobs include a sequential file stage, one transform and 1 output link to an oracle table. The contents of folder "1" are comma-delimited text files with 5 columns, whereas the contents of folder "2" are comma-delimited text files and csv's with 8 columns.

The PROBLEM:
The problem I'm having is that when DS encounters a text file in folder "1" which has too many columns, or trailing commas or some other imperfection, the entire Job "A" aborts -without continuing to process the rest of the files in folder "1" or in folder "2".

Or even when DS encounters duplicates (ie. records in that text file which match records already existing in the oracle table being outputted to), Job "A" does not abort but instead of continuing to process all the text files in folder "1", and then all the files in folder "2", it just stops.

QUESTION:
How can I get DS to continue processing other files after it encounters a bad one and rejects it. Similarly how can I get DS to continue processing other files after it encounters duplicates?

Any help is greatly appreciated!

ray.wurlod · Post by **ray.wurlod** » Tue May 27, 2003 4:22 am

Change the calls to DSAttachJob so that the second argument is DSJ.ERRWARN or DSJ.ERRNONE rather than DSJ.ERRFATAL

(Read the on-line help about DSAttachJob to find out why.)

Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518

sweetleaf · Post by **sweetleaf** » Tue May 27, 2003 9:08 am

Thanks so much Ray!

It's a relief that there is a way to correct this..

I've checked the online DS docs and what is available on DSattachJob does not really cover DSJ.ERRNONE or DSJ.ERRWARN - but i will continue searching..

thanks again

ray.wurlod · Post by **ray.wurlod** » Wed May 28, 2003 3:25 am

Here's the relevant lines from the Help file. It seems pretty clear to me, particularly the line about the purpose of the ErrorMode argument. ErrorMode specifies what the parent job does if the child job terminates abnormally.

==================================================

JobHandle = DSAttachJob (JobName, ErrorMode)

JobHandle is the name of a variable to hold the return value which is subsequently used by any other function or routine when referring to the job. Do not assume that this value is an integer.

JobName is a string giving the name of the job to be attached to.

ErrorMode is a value specifying how other routines using the handle should report errors. It is one of:

DSJ.ERRFATAL Log a fatal message and abort the controlling job (default).

DSJ.ERRWARN Log a warning message but carry on.

DSJ.ERRNONE No message logged - caller takes full responsibility (failure of DSAttachJob itself will be logged, however).