row counts

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
times29
Premium Member
Premium Member
Posts: 202
Joined: Mon Sep 22, 2008 3:47 pm

row counts

Post by times29 »

Hi,
How can i do below in datastage 8.5

Thanks,

When extracting table information from SAP into Dataset, we should know how many records (rows) have been extracted from the SAP table and how many records have been writen to the Dataset table. Likewise, when the information is processed into the Staging tables and into the Datamart Tables. It is to be noted that in some cases, it will be impossible to confirm the row count between 2 stages (SAP-Dataset, Dataset-Staging, or Staging-Datamart) when there is ETL transformation occurring or when the original source tables are split into multiple target tables
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

There are DataStage Server routines to let you get the number of rows passing through specific links, they require a bit of programming. You could also use scripts to search through the job log and parse out the appropriate values, that might be simpler.
Which method you use depends upon what you want to do with the numbers.
vmcburney
Participant
Posts: 3593
Joined: Thu Jan 23, 2003 5:25 pm
Location: Australia, Melbourne
Contact:

Post by vmcburney »

If you were to upgrade to version 8.7 you would get the new Operations Console which comes with a repository for storing ETL metadata such as row counts for all links in a job. This is an open repository and you could run reports and audits against it such as row counts. It's possible in the link naming conventions to indicate which links are supposed to have reduced rows and which are the primary input and output of a job just by using different prefixes in the link names.
times29
Premium Member
Premium Member
Posts: 202
Joined: Mon Sep 22, 2008 3:47 pm

Post by times29 »

Hi,
I have 8.7 installed now you did mention about the new Operations Console which comes with a repository can you please tell me if i run a job now
where to go to see the row counts for that job?

Thanks
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

The DSODB database. Assuming, of course, you have set it up and configured the services associated with it, which is entirely optional.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply