DSXchange

Hi,

DataStage: 7.5 OS: Windows

Is there a way to find the last link of an ETL job?. My requirement is to find out the start time, end time, status & rows processed by an etl job present in a project. The project has around 5000 jobs. We have an routine to give the start time, end time and status. Now i have to get the rows processed. If am able to get the last link,i can pass that to DSGetLinkInfo macro to get that.

Thanks,
Bharathappriyan

Jobs can have multiple "end" links.


Do a search for ETLstats here on the forums.

If you use link naming conventions, for example naming an output link tablename_OUT, then your task will become a lot easier.

Hi Ray,

80% of the jobs are using naming conventions. 20% are not.

Thanks,
Selva

Fix the 20%.

Technically, it seems to me that any stage with only an input link would be a target and thus considered as a 'last link'. And I believe that a call to DSGetStageInfo() would return that when using DSJ_LINKTYPES as the Info_Type.

The job report XML has links defined as output or input. You need to figure out which is really output to you. The row counts are all attached to links and stages. The starting point is an input stage with only input rows. Same on the output stages with only output rows. If that makes sense. All stages with inputs and outputs are in the middle.

So you would need to process the input and outputs into lookups. Next process the rows. Put into tables and run SQL to find only inputs and only outputs. Compare to your job design.

A lot of work.

@chulett : If they are writing to an error/ reject target in Server jobs, it would only have an 'input' link .

(This is probably not helpful.)
The last link in a job is the one that was added last (most recently) to the job design, and is therefore the link with the highest internal link/pin number.

It is like Bill Clinton says. Depends on how you define "last".

Exactly.