Rejecting bad records from sequential file
Posted: Tue Sep 29, 2015 10:57 pm
Hi,
I have a sequential file stage which reads data from a file and propagates it further to the next stage. The problem is that my job fails whenever there's a bad record in the file (like metadata mismatch, different record type in file from the metadata). Is there a way I can reject bad records and let the good records flow to the next stage. I don't want to handle these bad records in next transformer stage, just want to reject these records in the sequential file stage itself (while reading the records).
I have a sequential file stage which reads data from a file and propagates it further to the next stage. The problem is that my job fails whenever there's a bad record in the file (like metadata mismatch, different record type in file from the metadata). Is there a way I can reject bad records and let the good records flow to the next stage. I don't want to handle these bad records in next transformer stage, just want to reject these records in the sequential file stage itself (while reading the records).