Checking Source Data

[sainath]

Hi
Every day we get data from different organizations.I wan to design ETL jobs to count the number of rows we get per month and compare with previous month .if there is drop in records it should notify.
I want to know how to design in Datatsage.please share your
valuable ideas.
THKS

[kcbland]

Collect link statistics after every job run and store in a relational database. After every run, have a script verify the most recent run statistics for meaningful link counts against its average and notify if a X% drop occurred.

[vmcburney]

Have a look at Kim Duke's ETL Stats, amongst other things it populates an ETL_ROW_HISTORY table where you can see the history of every row for every link in a job and report or audit it with SQL.