how to find out howmany records were processed and rejected
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 85
- Joined: Fri Nov 18, 2005 5:35 am
how to find out howmany records were processed and rejected
Hi,
how to find out howmany records were processed and rejected.I want to store this information into database.
Thanks,
sudhakar
how to find out howmany records were processed and rejected.I want to store this information into database.
Thanks,
sudhakar
i need to know datastage
Re: how to find out howmany records were processed and rejec
do you want to capture job log info for rejected records or actual rejected records ?
sudhakar_viswa wrote:Hi,
how to find out howmany records were processed and rejected.I want to store this information into database.
Thanks,
sudhakar
-
- Participant
- Posts: 85
- Joined: Fri Nov 18, 2005 5:35 am
-
- Premium Member
- Posts: 209
- Joined: Fri Jan 09, 2004 1:14 pm
- Location: Toronto, Canada
- Contact:
Hi sudhakar,
Here's a couple of threads that will help you.
viewtopic.php?t=100831&highlight=record+counts
viewtopic.php?t=96468&highlight=record+counts
viewtopic.php?t=91135&highlight=record+counts
Rob Wierdsma
Here's a couple of threads that will help you.
viewtopic.php?t=100831&highlight=record+counts
viewtopic.php?t=96468&highlight=record+counts
viewtopic.php?t=91135&highlight=record+counts
Rob Wierdsma
Rob Wierdsma
Toronto, Canada
bartonbishop.com
Toronto, Canada
bartonbishop.com
You can write after job subroutine with the help of functions like DSGetLogSummary and DSGetLogSummary,this may useful for what u want to capture.
sudhakar_viswa wrote:Hi,
after the completing the job execution i need to store the howmany records are processed and howmany records are regected into database.
is there any way to capture the information in log files
Thanks,
sudhakar
You can use @INROWNUM and @OUTROWNUM to get that info from within the job. If you want to get that info from outside a job, i suggest you look at DSGetLogInfo() functions, design the job to handle rejects appropriately so that a distinct count can be done on those links.
Regards,
Regards,
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
-
- Participant
- Posts: 85
- Joined: Fri Nov 18, 2005 5:35 am
Recognize the type of rejects and handle them in the transformer itself. Make it go through a link named say 'rejected' to a reject file. So the number of records going through the rejected link will be your reject count and the number of records going through your other link will be your processed count.
How you can handle that?
Well the answer to that question depends upon what you are handling. If its failed key values by lookups you can do inlink.NOTFOUND property. If its data validation process, you can use stage variables. It all depends upon what you are trying to reject.
How you can handle that?
Well the answer to that question depends upon what you are handling. If its failed key values by lookups you can do inlink.NOTFOUND property. If its data validation process, you can use stage variables. It all depends upon what you are trying to reject.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
-
- Participant
- Posts: 85
- Joined: Fri Nov 18, 2005 5:35 am
Why do you want to use the aggregator if you can get that info by @OUTROWNUM. If you have millions of records, the aggregator is going to take a toll on your performance.
But if you insist, have the links go into the aggregator, do a count on the key on those links. That will get you the count.
But if you insist, have the links go into the aggregator, do a count on the key on those links. That will get you the count.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
-
- Participant
- Posts: 85
- Joined: Fri Nov 18, 2005 5:35 am
-
- Participant
- Posts: 85
- Joined: Fri Nov 18, 2005 5:35 am
The sequential file where you are storing the rejects. Have an output link coming out of it. In the properties, on the stage tab click on stage uses filter commands. In the path provide the path of the file, on the right in the filter command give the command wc -l.
In the output link, specify only one column say count. That will get you the count.
In the output link, specify only one column say count. That will get you the count.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.