How to capture reject records in ETL process?
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 3
- Joined: Mon Dec 04, 2006 10:30 pm
- Contact:
How to capture reject records in ETL process?
I have 174 records in input file but i am getting only 173 records as output and 1 record is rejected.I want to capture the reject record or display the reject record.
reject record is displayed as one column as {1a} but i want the whole record to be displayed.I am capturing the reject record in sequential file
Input,ouptut file is Sequential file
Thanks
swathi
reject record is displayed as one column as {1a} but i want the whole record to be displayed.I am capturing the reject record in sequential file
Input,ouptut file is Sequential file
Thanks
swathi
Last edited by swathi.mekala on Tue Dec 26, 2006 6:34 am, edited 1 time in total.
Re: How to capture reject records in ETL process?
Welcome Aboard :D
Swathi,
Your question is a little vague.
There are many topics relating to capturing rejects.
If you have any specific question (not found by a search) get back, there are many people here to help you out.
Swathi,
Your question is a little vague.
We have a search option, make use of it.swathi.mekala wrote:How to capture reject records in Datastage(ETL process)
There are many topics relating to capturing rejects.
If you have any specific question (not found by a search) get back, there are many people here to help you out.
Narasimha Kade
Finding answers is simple, all you need to do is come up with the correct questions.
Finding answers is simple, all you need to do is come up with the correct questions.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
For example, and especially in parallel jobs, you need to define what you mean by the term. Most stages support a reject-handling link, but what it means is different in different stage types.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 397
- Joined: Wed Apr 12, 2006 2:28 pm
- Location: Tennesse
what are the stages you are using in your job and at what point you want to reject the records?
Last edited by samsuf2002 on Tue Dec 12, 2006 4:23 pm, edited 1 time in total.
hi sam here
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 222
- Joined: Tue Aug 30, 2005 2:07 am
- Location: pune
- Contact:
Hi Swathi,
There are so many stages in Parallel to support reject process.
Lookup ----- Lookup fails (capture the driving link)
Merge-------- join fails (Capture the refrence/updated records)
Tranformer -------- caputure the records which are not satifying the requirements (Like null handling)
Sequential file : Reject the records if data is not satisfying the DataTypes.
Like some other stages are there........
You have to clear about your requirements........
If Its a Interview Question.......... DSXchange will not help you......
There are so many stages in Parallel to support reject process.
Lookup ----- Lookup fails (capture the driving link)
Merge-------- join fails (Capture the refrence/updated records)
Tranformer -------- caputure the records which are not satifying the requirements (Like null handling)
Sequential file : Reject the records if data is not satisfying the DataTypes.
Like some other stages are there........
You have to clear about your requirements........
If Its a Interview Question.......... DSXchange will not help you......
NageshSunkoji
If you know anything SHARE it.............
If you Don't know anything LEARN it...............
If you know anything SHARE it.............
If you Don't know anything LEARN it...............
-
- Participant
- Posts: 3
- Joined: Mon Dec 04, 2006 10:30 pm
- Contact:
Hi all;
I am sorry ,i posted it uncorrectly. what i intention is to capture the rejected records which are of incorrect format or datatypes. for example we are using the sequential file in our job to import the data . and if the data is of in correct format then we are capturing the records in to another sequential file by using the "reject mode" property as "output" and then when the job is run then we are getting the reject records into the sequential file . but we cant able to see the reject record exactly as it is. please help me regarding this query,its urgent.
Cheers;
Swathi.
I am sorry ,i posted it uncorrectly. what i intention is to capture the rejected records which are of incorrect format or datatypes. for example we are using the sequential file in our job to import the data . and if the data is of in correct format then we are capturing the records in to another sequential file by using the "reject mode" property as "output" and then when the job is run then we are getting the reject records into the sequential file . but we cant able to see the reject record exactly as it is. please help me regarding this query,its urgent.
Cheers;
Swathi.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
We don't do "urgent" - we are all volunteers who post as and when we can. If you want "urgent" sign up with your support provider for premium services, and learn the true cost of urgent.
The column sent to the reject link by the Sequential File stage is in Raw format. You can feed it to a Peek stage, and see it in the log entry produced You can write it to a sequential file, and read that with any utility external to DataStage.
Or, if you know its layout, you can parse the raw (or VarBinary) string into columns using a Column Import stage.
The column sent to the reject link by the Sequential File stage is in Raw format. You can feed it to a Peek stage, and see it in the log entry produced You can write it to a sequential file, and read that with any utility external to DataStage.
Or, if you know its layout, you can parse the raw (or VarBinary) string into columns using a Column Import stage.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Try rejecting your data into a dataset instead of a sequential file. Sounds like your delimiter settings in your sequential file are not initialised properly. Also be aware of nulls and use an explicit NullToValue conversion on nullable fields.
3NF: Every non-key attribute must provide a fact about the key, the whole key, and nothing but the key. So help me Codd.
I think Swathi might have tried to view data from the Sequential stage.
Though it is mentioned as Varbinary datatype, the reject records are stored fully. Try to check the data of the rejected file from unix using cat.
Though it is mentioned as Varbinary datatype, the reject records are stored fully. Try to check the data of the rejected file from unix using cat.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
I'm also want to "deconstruct" the varbinary field into its columns, but when I tried to link the rejected seq file or dataset (from a reject link) to a transformer stage, I'm getting the error message - The source stage does not support non-reject output links whilst it has input links.
Am I missing something????
Am I missing something????
Perhaps, you can process the reject file in separate job. Read it as varchar and "deconstruct" as you wish. But still the records which got reject might be due to metadata mismatch, hence you may not parse correctly.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
The Column Import stage can deconstruct a raw (VarBinary) string.Karine wrote:I'm also want to "deconstruct" the varbinary field into its columns, ...
Am I missing something????
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.