creating job statistics from joblog created in datastage
Moderators: chulett, rschirm, roy
creating job statistics from joblog created in datastage
Hi all
We need to create a Dtastage job to get the job statistics from the job log created in Datastage.
1) How many source records
2) How many rejected - rejects can be in lookups, due to source metadata, error cnditions etc.
3) How many loaded/updated
Can anybody help me please....
Thanks
We need to create a Dtastage job to get the job statistics from the job log created in Datastage.
1) How many source records
2) How many rejected - rejects can be in lookups, due to source metadata, error cnditions etc.
3) How many loaded/updated
Can anybody help me please....
Thanks
skf
Do a Search on ETLStats. Bunch of goodies.
You would need to have separate links for the inserts and updates(one for insert and the other for update). (Not from the same link)
Or may be you would need to maintain the timestamp (may be within the target table or may be separately)
For this may be you would need to alter your job designs.3) How many loaded/updated
You would need to have separate links for the inserts and updates(one for insert and the other for update). (Not from the same link)
Or may be you would need to maintain the timestamp (may be within the target table or may be separately)
Success consists of getting up just one more time than you fall.
Yes, i have seperate links for insert and update. and where can i actually find required information from Kduke's site, there are lots of things available in his site. can you please guide me for same ?
[quote="loveojha2"]Do a Search on ETLStats. Bunch of goodies.
[quote]3) How many loaded/updated [/quote]
For this [b]may be [/b]you would need to alter your job designs.
You would need to have separate links for the inserts and updates(one for insert and the other for update). (Not from the same link)
Or may be you would need to maintain the timestamp (may be within the target table or may be separately)[/quote]
[quote="loveojha2"]Do a Search on ETLStats. Bunch of goodies.
[quote]3) How many loaded/updated [/quote]
For this [b]may be [/b]you would need to alter your job designs.
You would need to have separate links for the inserts and updates(one for insert and the other for update). (Not from the same link)
Or may be you would need to maintain the timestamp (may be within the target table or may be separately)[/quote]
Re: creating job statistics from joblog created in datastage
shaikf wrote:Hi all
We need to create a Dtastage job to get the job statistics from the job log created in Datastage.
1) How many source records
2) How many rejected - rejects can be in lookups, due to source metadata, error cnditions etc.
3) How many loaded/updated
Can anybody help me please....
Thanks
chk out with dsj.linkrowcount
chk in ds help for dsgetlinkinfo function
Re: creating job statistics from joblog created in datastage
You are right that we can do with this function , in my case i want output for all jobs in a project, but i dont want to explicitly specify stage name multiple times, with differnt jobs, stage name may be different. I want generic output for same.
[quote="rachitha"][quote="shaikf"]Hi all
We need to create a Dtastage job to get the job statistics from the job log created in Datastage.
1) How many source records
2) How many rejected - rejects can be in lookups, due to source metadata, error cnditions etc.
3) How many loaded/updated
Can anybody help me please....
Thanks[/quote]
chk out with dsj.linkrowcount
chk in ds help for dsgetlinkinfo function[/quote]
[quote="rachitha"][quote="shaikf"]Hi all
We need to create a Dtastage job to get the job statistics from the job log created in Datastage.
1) How many source records
2) How many rejected - rejects can be in lookups, due to source metadata, error cnditions etc.
3) How many loaded/updated
Can anybody help me please....
Thanks[/quote]
chk out with dsj.linkrowcount
chk in ds help for dsgetlinkinfo function[/quote]
EtlStats is on my tips page below my signature or the other link. When you unzip it there is a text file on how to install it. It is a bunch of jobs and source code to the jobs and routines. It also needs a few sequential files or directories to store sequential files. Instructions show you how to copy all the files to the proper directories because it unzips them in those same directories.
Import the jobs. Compile the jobs and follow instructions. Easy.
Import the jobs. Compile the jobs and follow instructions. Easy.
Mamu Kim
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
You can create nested loops that obtain a list of the stage names in a job and a list of the links connected to a stage (DSGetJobInfo and DSGetStageInfo respectively).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
consider we have one job in which 10 source stages(including hashfiles,sequential files and database) and in other job i have 15 source stages(including hashfiles,sequential files and database) , problem is that in second job if we use hash file as intermediate stage then how can we decide this is our source file or intermediate file ? possible that it may not have same design for each job !
[quote="ray.wurlod"]You can create nested loops that obtain a list of the stage names in a job and a list of the links connected to a stage (DSGetJobInfo and DSGetStageInfo respectively). ...[/quote]
[quote="ray.wurlod"]You can create nested loops that obtain a list of the stage names in a job and a list of the links connected to a stage (DSGetJobInfo and DSGetStageInfo respectively). ...[/quote]
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Challenge that requirement, for it is asinine. The right place to capture job statistics is after the job is finished. [Fundamental principle; the act of measuring invalidates the measurement while a process is executing.] This means in the job sequence or shell script that controls it. That, in turn, implies either a server routine (that can be invoked from a Routine activity in a job sequence) or some more shell scripting.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.