Concering Data audit

[bhargav_dd]

Dear All,
-
Recently, there is a requirement that performing data audit throught datastage job. Its functionality includes store error data in a separate file along with error message, row number and source name when detect a violation in terms of data type, length of character string, etc.
-
So, if we wanna perform that within datastage job, what is the best practice for it and what kind of stage would be referred?
If there are other good solution besides datastage, such as unix shell level, pls advise me that as well.

[ray.wurlod]

That really depends on what is meant by "reject" but, in most cases, explicit reject-handling output links from stages combined with post-job row count reconciliation ought to do most of what you require.

[kduke]

Be careful storing this information. If someone runs a job with no warning limit then every row can get an error. You could put millions of errors in an audit table.