Handling rejected Records

Srilakshmi · Post by **Srilakshmi** » Thu May 25, 2006 5:36 am

Hi ,
I have got a query. Iam new to parallel extenders. The source and target is oracle . How do I handle the rejected records from the source and how do I reprocess the rejected records.

ArndW · Post by **ArndW** » Thu May 25, 2006 5:38 am

The source won't have any rejected records, but when writing to the target in PX you can (in most cases) put a link coming out of the target stage that will contain the rejects. This is documented in the Parallel Job Developer's Guide in the section for the particular database stage.

rony_daniel · Post by **rony_daniel** » Thu May 25, 2006 6:32 am

ArndW wrote:The source won't have any rejected records, but when writing to the target in PX you can (in most cases) put a link coming out of the target stage that will contain the rejects. This is documented in the Parallel Job Developer's Guide in the section for the particular database stage.

Hi ArndW,

If my source stage is a sequential file stage and suppose I have a date datatype filed in it. Some of the records in the source has invalid date value (say "2006-02-31"), then those records are droped at source stage itself. Can we not call these records rejects at the source?

ray.wurlod · Post by **ray.wurlod** » Thu May 25, 2006 3:26 pm

That's a different question. The OP's source was Oracle - you don't get any rejects from a SELECT statement.

With a Sequential File stage you can have a rejects output link which captures, in raw format, any row that does not satisfy the metadata of the primary output link. Without a rejects output link these rows are, as you noted, dropped (or you can abort the job). Use the Reject Mode property (in the Options group) to determine disposal of such rows.

Srilakshmi · Post by **Srilakshmi** » Fri May 26, 2006 12:20 am

Hi Arnd,
My first question is answered. my second question is how are the rejected records reprocessed.
Thanks

sanjay · Post by **sanjay** » Fri May 26, 2006 1:00 am

Hi
If record is rejected due to date invalid. wht is the process u have . I mean whether u will correct that record.

Sanjay

Srilakshmi wrote:Hi Arnd,
My first question is answered. my second question is how are the rejected records reprocessed.
Thanks

vmcburney · Post by **vmcburney** » Fri May 26, 2006 3:55 am

You've got to assume that if a record is rejected that reprocessing it will have the same result, since the data has not been changed. If you are changing data then you are manually altering it to differ to what is in source systems which is a dangerous path to take.

Have a look at my data quality firewall blog on how to get some meaningful metrics out of your reject links and how to build rules to keep or drop invalid rows.

The best fix is to go back and fix the problem in the source system and pick up the change through normal extracts or change capture.

ray.wurlod · Post by **ray.wurlod** » Fri May 26, 2006 3:58 pm

DataStage does not process rejected records automatically in any way. It delivers them onto the rejects-handling output link of the stage in question. For Sequential File stage at least, the record is delivered as Raw - you can further process these rows by converting from Raw to something else, but remember that the row is there in the first place because it did not match the given metadata. You can certainly convert to string, and write to another text file.