Invalid/Error Data handling
Posted: Wed Nov 24, 2010 5:16 am
Hi,
I have a record which is having nearly 15 fields/columns. We have to do some data values/format/nullability checks on some 10 fields. We decided to do these checks in Transformer stage. Till here no problem.
But we have to insert records in ERRORLOG table for each field validatin failure. Suppose 3 field validation failed then 3 records in ERRORLOG and if 8 fields validation failed then 8 records in ERRORLOG table.
Please suggest me solution/design of this job and performance should not be compromised.
Regards,
Rohit
I have a record which is having nearly 15 fields/columns. We have to do some data values/format/nullability checks on some 10 fields. We decided to do these checks in Transformer stage. Till here no problem.
But we have to insert records in ERRORLOG table for each field validatin failure. Suppose 3 field validation failed then 3 records in ERRORLOG and if 8 fields validation failed then 8 records in ERRORLOG table.
Please suggest me solution/design of this job and performance should not be compromised.
Regards,
Rohit