Error Processing
Moderators: chulett, rschirm, roy
Error Processing
Hi
What is an Error Processing table and Audit table.
Plz let me know the concept behind these tables.
How these tables could be built?
What is an Error Processing table and Audit table.
Plz let me know the concept behind these tables.
How these tables could be built?
Pradeep Kumar
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Something a site chooses to implement.
Nothing to do with DataStage, though that may be the tool used to populate these tables.
My guesses would be as follows.
Nothing to do with DataStage, though that may be the tool used to populate these tables.
My guesses would be as follows.
- Error processing table records any errors generated within ETL processing.
Audit table is used to store row count reconciliations.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Hi pradkumar,
You can let us know, what is this regards to.
It can be either the either the normal naming conventions used in your site or project for the details given by Ray or it can be referred to the database internal tables.
You can let us know, what is this regards to.
It can be either the either the normal naming conventions used in your site or project for the details given by Ray or it can be referred to the database internal tables.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
As mentioned earlier, its one of the process you like to follow in you development. Error processing tables will mostly be the one which collect all the rejects with the error code. There are several approaches, based on you methodology. Its more flexible if you stored procedure.
Audit table will be the one, which will have the total row count, start time, end time of all the tables while data population (Including Error processing Tables).
Audit table will be the one, which will have the total row count, start time, end time of all the tables while data population (Including Error processing Tables).
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
You build these tables according to the specifications of your customer. What information do they want recorded? Once you have those answers, you can figure out how to capture these from DataStage and elsewhere, and create tasks for recording them into the tables.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Right now I am trying to build an audit table which consists of the following information:
1) Batch number of the daily run
2) Extraction start time
3) Extraction end time
4) Time elapsed
SO where to insert this link in the job and how to populate these columns. This table will be an ouput link from Transformer (I guess)
1) Batch number of the daily run
2) Extraction start time
3) Extraction end time
4) Time elapsed
SO where to insert this link in the job and how to populate these columns. This table will be an ouput link from Transformer (I guess)
Pradeep Kumar
Not necessarily, it can be even on Before/After SQLs. In the before SQLs of the input stage, you can insert the record with the start time as SYSDATE and current batch id as Jobparameter which is passed from JobSequence or the Main job control. And in the After job control populate you can populate the SYSDATE for End time and StartDate - EndDate as ElaspsedTime.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Premium Member
- Posts: 133
- Joined: Tue Nov 23, 2004 11:24 pm
- Location: India
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
I'd always advocate a separate job. Of course, you may need to detect and trap error information in the job itself, for such is the nature of these things, but simply redirect the error information into a file. The row counts and started/stopped times can be obtained subsequently. Use the times from active stages rather than from jobs (or both) if you're interested in discounting startup and shutdown overheads.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.