Read Rejected Rows and Format the Rows

[anil411]

I am reading Sequential File of 27000 Records of which 25000 Records MetaData is matching and processing the 25000 records.

While Reading Sequential File, We are using Reject mode option and writing Rejected records output to Sequential File.All the Rejected records (arround 2000) are written to Reject File.

I would like to Fix the 2000 records of Data.How do i format the data or Read the Reject File and try to find which column is causing the issue.

Please let me know.

[paultechm]

Read as a single varchar column(with out column length) correct each line in a transformer ,use column import to split the data into actual meta data

[chulett]

There's no "reject reason" recorded so (AFAIK) this is strictly a manual process. Compare the rejected rows to the metadata you've defined in the stage to see which fields have data that do not fit their metadata, for example a date column with an invalid date in it.

If you want them processed rather than rejected, make sure all of the fields are defined as a string and then do the data validation yourself inside the job. That way you may be able to 'correct' any issues found rather than have them simply rejected. Worst case, you can manually reject them and at the same time record the reason.

[ray.wurlod]

After capturing the rejected rows from the Sequential File stage using a reject link, re-parse them for example using a Column Import stage or Transformer stage.