Page 1 of 1

Checking Source Data

Posted: Fri Nov 25, 2005 2:43 pm
by sainath
Hi
Every day we get data from different organizations.I wan to design ETL jobs to count the number of rows we get per month and compare with previous month .if there is drop in records it should notify.
I want to know how to design in Datatsage.please share your
valuable ideas.
THKS

Posted: Fri Nov 25, 2005 5:02 pm
by kcbland
Collect link statistics after every job run and store in a relational database. After every run, have a script verify the most recent run statistics for meaningful link counts against its average and notify if a X% drop occurred.

Posted: Sat Nov 26, 2005 12:45 am
by vmcburney
Have a look at Kim Duke's ETL Stats, amongst other things it populates an ETL_ROW_HISTORY table where you can see the history of every row for every link in a job and report or audit it with SQL.