creating job statistics from joblog created in datastage

[shaikf]

Hi all

We need to create a Dtastage job to get the job statistics from the job log created in Datastage.
1) How many source records
2) How many rejected - rejects can be in lookups, due to source metadata, error cnditions etc.
3) How many loaded/updated

Can anybody help me please....

Thanks

[loveojha2]

Do a Search on ETLStats. Bunch of goodies.

3) How many loaded/updated

For this may be you would need to alter your job designs.
You would need to have separate links for the inserts and updates(one for insert and the other for update). (Not from the same link)

Or may be you would need to maintain the timestamp (may be within the target table or may be separately)

[hhh]

Yes, i have seperate links for insert and update. and where can i actually find required information from Kduke's site, there are lots of things available in his site. can you please guide me for same ?


[quote="loveojha2"]Do a Search on ETLStats. Bunch of goodies.

[quote]3) How many loaded/updated [/quote]

For this [b]may be [/b]you would need to alter your job designs.
You would need to have separate links for the inserts and updates(one for insert and the other for update). (Not from the same link)

Or may be you would need to maintain the timestamp (may be within the target table or may be separately)[/quote]

[rachitha]

Hi all

We need to create a Dtastage job to get the job statistics from the job log created in Datastage.
1) How many source records
2) How many rejected - rejects can be in lookups, due to source metadata, error cnditions etc.
3) How many loaded/updated

Can anybody help me please....

Thanks

chk out with dsj.linkrowcount
chk in ds help for dsgetlinkinfo function

[ArndW]

DataStage BASIC programs are case-sensitive, so in addition to translating the SMS shorthand "chk out" to "check out", remember that you need to use DSJ.LINKROWCOUNT and DSGetLinkInfo()

[hhh]

You are right that we can do with this function , in my case i want output for all jobs in a project, but i dont want to explicitly specify stage name multiple times, with differnt jobs, stage name may be different. I want generic output for same.



[quote="rachitha"][quote="shaikf"]Hi all

We need to create a Dtastage job to get the job statistics from the job log created in Datastage.
1) How many source records
2) How many rejected - rejects can be in lookups, due to source metadata, error cnditions etc.
3) How many loaded/updated

Can anybody help me please....

Thanks[/quote]


chk out with dsj.linkrowcount
chk in ds help for dsgetlinkinfo function[/quote]

[kduke]

EtlStats is on my tips page below my signature or the other link. When you unzip it there is a text file on how to install it. It is a bunch of jobs and source code to the jobs and routines. It also needs a few sequential files or directories to store sequential files. Instructions show you how to copy all the files to the proper directories because it unzips them in those same directories.

Import the jobs. Compile the jobs and follow instructions. Easy.

[ray.wurlod]

You can create nested loops that obtain a list of the stage names in a job and a list of the links connected to a stage (DSGetJobInfo and DSGetStageInfo respectively).

[hhh]

consider we have one job in which 10 source stages(including hashfiles,sequential files and database) and in other job i have 15 source stages(including hashfiles,sequential files and database) , problem is that in second job if we use hash file as intermediate stage then how can we decide this is our source file or intermediate file ? possible that it may not have same design for each job !



[quote="ray.wurlod"]You can create nested loops that obtain a list of the stage names in a job and a list of the links connected to a stage (DSGetJobInfo and DSGetStageInfo respectively). ...[/quote]

[ray.wurlod]

Decide upon and rigorously follow a stage and link naming convention.

[hhh]

Our requirement has been changed and need to generate routine in parallel job, stll i have not created parallel routine,this is different than server routine, can you pls give me idea how to implement the routine in px ?

[ray.wurlod]

Challenge that requirement, for it is asinine. The right place to capture job statistics is after the job is finished. [Fundamental principle; the act of measuring invalidates the measurement while a process is executing.] This means in the job sequence or shell script that controls it. That, in turn, implies either a server routine (that can be invoked from a Routine activity in a job sequence) or some more shell scripting.