Capture reject records from data source
Posted: Tue Apr 20, 2010 8:30 pm
Hi,
I have created a parallel job that reads data from SQL Server using a SELECT statement. I am currently using "Dynamic RDBMS" stage for this purpose. The data is passed through a transformer and written to a sequential file.
There are four values with DOUBLE as the data type in the output. These values are cast to the appropriate target data type in the SELECT statement.
The DRDBMS stage throws the error "Numeric Value out of range" when the job is run.
I have gone through the forums and found that there are two solutions to this type of problem.
(a) Investigate the data to weed out/fix the offending records.
(b) Reject the data to an output file using ODBC
I have tried (a) by executing the statement in SQL Server Enterprise Manager. The output data is seemingly innocuous.
I tried (b). But I am unable to create a reject link from any of the following stages: ODBC Enterprise, ODBC Connector, DRDBMS. Since we are on Linux, SQL Server Enterprise stage is not available. I have perused the manuals and could not find a whole lot. There is mention of Reject Links in the ODBC connectivity guide, but when I create a reject link from the ODBC stage, I get a compilation error.
With ODBC connector, the error is
"The reject link is not valid with this combination of input and output links".
With ODBC enterprise, the error is
"Reject link <link_name> is not allowed with the current setting (or lack of) the "Output Reject Records" property. I cannot find this property in the Options section of the stage.
Is there a way to capture the offending records?
Thanks,
I have created a parallel job that reads data from SQL Server using a SELECT statement. I am currently using "Dynamic RDBMS" stage for this purpose. The data is passed through a transformer and written to a sequential file.
There are four values with DOUBLE as the data type in the output. These values are cast to the appropriate target data type in the SELECT statement.
The DRDBMS stage throws the error "Numeric Value out of range" when the job is run.
I have gone through the forums and found that there are two solutions to this type of problem.
(a) Investigate the data to weed out/fix the offending records.
(b) Reject the data to an output file using ODBC
I have tried (a) by executing the statement in SQL Server Enterprise Manager. The output data is seemingly innocuous.
I tried (b). But I am unable to create a reject link from any of the following stages: ODBC Enterprise, ODBC Connector, DRDBMS. Since we are on Linux, SQL Server Enterprise stage is not available. I have perused the manuals and could not find a whole lot. There is mention of Reject Links in the ODBC connectivity guide, but when I create a reject link from the ODBC stage, I get a compilation error.
With ODBC connector, the error is
"The reject link is not valid with this combination of input and output links".
With ODBC enterprise, the error is
"Reject link <link_name> is not allowed with the current setting (or lack of) the "Output Reject Records" property. I cannot find this property in the Options section of the stage.
Is there a way to capture the offending records?
Thanks,