Error logging
Posted: Thu Jul 16, 2009 10:16 am
Hi,
My requirement is to reject and load the records which fail the business rule. Currently my job design is
seq stage->transformer 1->transformer 2->dataset1 and dataset2
Seq file to read the data from text, transformer 1 is to split the trailer and data record, transformer 2 is to validate the records for business logic. There can be multiple failures for a single record; we need to capture all the failures. Records which pass all the business validation moves to DataSet1 and rejected records will be moved to dataset2. Currently I am populating the different error codes against the same record with a delimiter.
The output of dataset 1 is
Emp_no Emp_Name Emp_Sal Status
1 A 100 Validation passed
2 B 100 Validation passed
The output of dataset 2 is
Emp_no Emp_Name Emp_Sal Status Error code
3 Validation Failed C2,C3
4 B Validation Failed C3
Business wants the output as below (dataset3)
Table Emp_No Error code
EMP 3 C2
EMP 3 C3
EMP 4 C3
Can someone help me how to create the output using transformer 2 and send the detail to a new dataset 3 in the above format? Option ops out of my mind is to use dataset2 and load the table errog log table as we have comma as the delimiters between different error codes (i feel this option is not efficient)[/img]
My requirement is to reject and load the records which fail the business rule. Currently my job design is
seq stage->transformer 1->transformer 2->dataset1 and dataset2
Seq file to read the data from text, transformer 1 is to split the trailer and data record, transformer 2 is to validate the records for business logic. There can be multiple failures for a single record; we need to capture all the failures. Records which pass all the business validation moves to DataSet1 and rejected records will be moved to dataset2. Currently I am populating the different error codes against the same record with a delimiter.
The output of dataset 1 is
Emp_no Emp_Name Emp_Sal Status
1 A 100 Validation passed
2 B 100 Validation passed
The output of dataset 2 is
Emp_no Emp_Name Emp_Sal Status Error code
3 Validation Failed C2,C3
4 B Validation Failed C3
Business wants the output as below (dataset3)
Table Emp_No Error code
EMP 3 C2
EMP 3 C3
EMP 4 C3
Can someone help me how to create the output using transformer 2 and send the detail to a new dataset 3 in the above format? Option ops out of my mind is to use dataset2 and load the table errog log table as we have comma as the delimiters between different error codes (i feel this option is not efficient)[/img]