Capture reject records from data source

[shankar_ramanath]

Hi,

I have created a parallel job that reads data from SQL Server using a SELECT statement. I am currently using "Dynamic RDBMS" stage for this purpose. The data is passed through a transformer and written to a sequential file.

There are four values with DOUBLE as the data type in the output. These values are cast to the appropriate target data type in the SELECT statement.

The DRDBMS stage throws the error "Numeric Value out of range" when the job is run.

I have gone through the forums and found that there are two solutions to this type of problem.

(a) Investigate the data to weed out/fix the offending records.
(b) Reject the data to an output file using ODBC

I have tried (a) by executing the statement in SQL Server Enterprise Manager. The output data is seemingly innocuous.

I tried (b). But I am unable to create a reject link from any of the following stages: ODBC Enterprise, ODBC Connector, DRDBMS. Since we are on Linux, SQL Server Enterprise stage is not available. I have perused the manuals and could not find a whole lot. There is mention of Reject Links in the ODBC connectivity guide, but when I create a reject link from the ODBC stage, I get a compilation error.

With ODBC connector, the error is
"The reject link is not valid with this combination of input and output links".

With ODBC enterprise, the error is
"Reject link <link_name> is not allowed with the current setting (or lack of) the "Output Reject Records" property. I cannot find this property in the Options section of the stage.

Is there a way to capture the offending records?

Thanks,

[desmondcheah]

I've tried the same and cant seem to create reject links for ODBC Stage but are able to do it for oracle stages.

Anyway from what i've encounter. When using ODBC stages and having records with errors, you should be able to get more description from the director log as compared to using native stages.

Correct me if i am wrong.. I just notice that error messages for the same problem seems to be different when it comes to different stage.

[shankar_ramanath]

Yes. Interestingly the problem disappeared once I used the ODBC Enterprise stage. I used the same scale and precision values as it was when the job failed with DRS.

I will leave this thread open as the question is still open. It would be really good if there is a way to get the rejected records. Not many Gurus have replied yet

I've tried the same and cant seem to create reject links for ODBC Stage but are able to do it for oracle stages.

Anyway from what i've encounter. When using ODBC stages and having records with errors, you should be able to get more description from the director log as compared to using native stages.

Correct me if i am wrong.. I just notice that error messages for the same problem seems to be different when it comes to different stage.