Get count of rejected records without outputting them?

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Ragunathan Gunasekaran
Participant
Posts: 247
Joined: Mon Jan 22, 2007 11:33 pm

Get count of rejected records without outputting them?

Post by Ragunathan Gunasekaran »

Hi ,


I am validating a record using lookup stage and if there is a lookup failure i need to reject that record but the rejected record need not be posted to a log(Datastage logs/log files).

I am wondering is there a Do nothing functionality in any of the stages in Datastage 8.1 so that i could use that to stop the flow of the rejected/discarded data..?( I am aware that in a parallel job data lands finally to passive stages/peek stage .. but in my scenario, i shouldn't be using any passive stages to capture that rejected data )

Any clue please ..?
Regards
Ragu
ThilSe
Participant
Posts: 80
Joined: Thu Jun 09, 2005 7:45 am

Post by ThilSe »

If your intention is to count the number of records in reject link, you can use a copy stage.

Regards,
Senthil
Ragunathan Gunasekaran
Participant
Posts: 247
Joined: Mon Jan 22, 2007 11:33 pm

Post by Ragunathan Gunasekaran »

Hi,

The count and the reconcillation stuff happens in a After job subroutine

I would need records to be flowing out of the reject link so that i gets counted by the After job subroutine but rejected records should just get ignored from the rest of the processing( neither stored nor logged any where !)

I believe if i connect copy stage to a reject link and leave it with out a output link (to throw away the data from further processing) then i will get a compilation error( As a pre requisite the copy operator needs an output).
Regards
Ragu
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Ragunathan Gunasekaran wrote:I believe if i connect copy stage to a reject link and leave it with out a output link (to throw away the data from further processing) then i will get a compilation error( As a pre requisite the copy operator needs an output).
Try it and let us know.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Ragunathan Gunasekaran
Participant
Posts: 247
Joined: Mon Jan 22, 2007 11:33 pm

Post by Ragunathan Gunasekaran »

Well said .. The solution should be tried first ....

Copy stage works to throw away data from further processing with ease...I have connected the copy stage to the reject link of the lookup stage and left the copy stage with out any output links

With the above design the job compiled and ran sucessfully maintaining the count on the reject link and discarded data from further processing.... Thanks for the suggestion.
Regards
Ragu
Post Reply