how to find out howmany records were processed and rejected

[sudhakar_viswa]

Hi,

how to find out howmany records were processed and rejected.I want to store this information into database.

Thanks,
sudhakar

[hhh]

do you want to capture job log info for rejected records or actual rejected records ?

Hi,

how to find out howmany records were processed and rejected.I want to store this information into database.

Thanks,
sudhakar

[sudhakar_viswa]

Hi,

after the completing the job execution i need to store the howmany records are processed and howmany records are regected into database.

is there any way to capture the information in log files

Thanks,
sudhakar

[rwierdsm]

Hi sudhakar,

Here's a couple of threads that will help you.

viewtopic.php?t=100831&highlight=record+counts

viewtopic.php?t=96468&highlight=record+counts

viewtopic.php?t=91135&highlight=record+counts

Rob Wierdsma

[hhh]

You can write after job subroutine with the help of functions like DSGetLogSummary and DSGetLogSummary,this may useful for what u want to capture.

Hi,

after the completing the job execution i need to store the howmany records are processed and howmany records are regected into database.

is there any way to capture the information in log files

Thanks,
sudhakar

[DSguru2B]

You can use @INROWNUM and @OUTROWNUM to get that info from within the job. If you want to get that info from outside a job, i suggest you look at DSGetLogInfo() functions, design the job to handle rejects appropriately so that a distinct count can be done on those links.
Regards,

[sudhakar_viswa]

Hi DSguru,

How to acheive this.Give me some idea


Thanks,
sudhakar

[DSguru2B]

Recognize the type of rejects and handle them in the transformer itself. Make it go through a link named say 'rejected' to a reject file. So the number of records going through the rejected link will be your reject count and the number of records going through your other link will be your processed count.
How you can handle that?
Well the answer to that question depends upon what you are handling. If its failed key values by lookups you can do inlink.NOTFOUND property. If its data validation process, you can use stage variables. It all depends upon what you are trying to reject.

[sudhakar_viswa]

Hi DSguru,

I don't want to use the aggregator.Thats why how can i achieve

Thanks,
sudhakar

[DSguru2B]

Why do you want to use the aggregator if you can get that info by @OUTROWNUM. If you have millions of records, the aggregator is going to take a toll on your performance.
But if you insist, have the links go into the aggregator, do a count on the key on those links. That will get you the count.

[kris007]

Why do you want to use the aggregator if you can get that info by @OUTROWNUM.

OP didn't want to use the Aggregator stage

I looked through the links provided by Rob and they pretty much cover the areas you need, to accomplish your task.

[sudhakar_viswa]

Hi DSGuru,

For each record it will give the @OUTROWNUM.How can i get the last record @OUTROWNUM.If you use the stage variable you can store the @OUTROWNUM.How can i store this information in database

Thanks,
sudhakar

[DSguru2B]

Well, i think you will have to use the aggregator to get the max of @OUTROWNUM to get the link count, if you want to do it within that job. Or the other way is to do a wc -l on the file and get the value.

[sudhakar_viswa]

Hi DSGuru,

Thanks for your replying.My reqirement is to do everything in one job.And one more thing is i have to update the database.How can i apply wc -l in the same job.

Thanks,
regards,

[DSguru2B]

The sequential file where you are storing the rejects. Have an output link coming out of it. In the properties, on the stage tab click on stage uses filter commands. In the path provide the path of the file, on the right in the filter command give the command wc -l.
In the output link, specify only one column say count. That will get you the count.