Page 1 of 2

How to capture reject records in ETL process?

Posted: Mon Dec 11, 2006 11:07 pm
by swathi.mekala
I have 174 records in input file but i am getting only 173 records as output and 1 record is rejected.I want to capture the reject record or display the reject record.
reject record is displayed as one column as {1a} but i want the whole record to be displayed.I am capturing the reject record in sequential file
Input,ouptut file is Sequential file

Thanks
swathi

Re: How to capture reject records in ETL process?

Posted: Mon Dec 11, 2006 11:21 pm
by narasimha
Welcome Aboard :D

Swathi,

Your question is a little vague.
swathi.mekala wrote:How to capture reject records in Datastage(ETL process)
We have a search option, make use of it.
There are many topics relating to capturing rejects.
If you have any specific question (not found by a search) get back, there are many people here to help you out.

Posted: Mon Dec 11, 2006 11:49 pm
by ray.wurlod
For example, and especially in parallel jobs, you need to define what you mean by the term. Most stages support a reject-handling link, but what it means is different in different stage types.

Posted: Tue Dec 12, 2006 3:26 pm
by samsuf2002
what are the stages you are using in your job and at what point you want to reject the records?

Posted: Tue Dec 12, 2006 3:42 pm
by ray.wurlod
Who is "u" and what is "r"? Is "wantt" related to "Gantt"?

Please use English on this forum.

Posted: Wed Dec 13, 2006 11:43 am
by Nageshsunkoji
Hi Swathi,

There are so many stages in Parallel to support reject process.

Lookup ----- Lookup fails (capture the driving link)

Merge-------- join fails (Capture the refrence/updated records)

Tranformer -------- caputure the records which are not satifying the requirements (Like null handling)

Sequential file : Reject the records if data is not satisfying the DataTypes.

Like some other stages are there........

You have to clear about your requirements........

If Its a Interview Question.......... DSXchange will not help you......

Posted: Wed Dec 27, 2006 12:33 am
by swathi.mekala
Hi all;

I am sorry ,i posted it uncorrectly. what i intention is to capture the rejected records which are of incorrect format or datatypes. for example we are using the sequential file in our job to import the data . and if the data is of in correct format then we are capturing the records in to another sequential file by using the "reject mode" property as "output" and then when the job is run then we are getting the reject records into the sequential file . but we cant able to see the reject record exactly as it is. please help me regarding this query,its urgent.

Cheers;
Swathi.

Posted: Wed Dec 27, 2006 1:01 am
by ray.wurlod
We don't do "urgent" - we are all volunteers who post as and when we can. If you want "urgent" sign up with your support provider for premium services, and learn the true cost of urgent.

The column sent to the reject link by the Sequential File stage is in Raw format. You can feed it to a Peek stage, and see it in the log entry produced You can write it to a sequential file, and read that with any utility external to DataStage.

Or, if you know its layout, you can parse the raw (or VarBinary) string into columns using a Column Import stage.

Posted: Tue Mar 06, 2007 2:46 am
by battaliou
Try rejecting your data into a dataset instead of a sequential file. Sounds like your delimiter settings in your sequential file are not initialised properly. Also be aware of nulls and use an explicit NullToValue conversion on nullable fields.

Posted: Tue Mar 06, 2007 4:58 am
by kumar_s
As noted, you can try writing into dataset, and later you can convert Varbinary into varchar in Sequential file.

Posted: Thu Mar 08, 2007 5:29 am
by Karine
Hi Kumar,
I have a similar requirement where I need to identify the rejected rows with its column values so that the data steward can analyze the error. Can you please explain how I can convert varbinary into varchar or its columns ?

Karine

Posted: Thu Mar 08, 2007 5:50 am
by kumar_s
I think Swathi might have tried to view data from the Sequential stage.
Though it is mentioned as Varbinary datatype, the reject records are stored fully. Try to check the data of the rejected file from unix using cat.

Posted: Thu Mar 08, 2007 6:03 am
by Karine
I'm also want to "deconstruct" the varbinary field into its columns, but when I tried to link the rejected seq file or dataset (from a reject link) to a transformer stage, I'm getting the error message - The source stage does not support non-reject output links whilst it has input links.
Am I missing something????

Posted: Thu Mar 08, 2007 6:08 am
by kumar_s
Perhaps, you can process the reject file in separate job. Read it as varchar and "deconstruct" as you wish. But still the records which got reject might be due to metadata mismatch, hence you may not parse correctly.

Posted: Thu Mar 08, 2007 8:29 am
by ray.wurlod
Karine wrote:I'm also want to "deconstruct" the varbinary field into its columns, ...
Am I missing something????
The Column Import stage can deconstruct a raw (VarBinary) string.