Link count for copy stage
Posted: Thu Dec 27, 2012 1:41 am
Hi,
i have a job which loads from a source dataset to a target dataset ( + alookup before loading to target) with some transformation rules involved which is handled through a transformer. There is a reject link configured for a lookup failure. The reject link has the following stages: copy stage and a peek stage. Link count is retrieved through peek stage name and peek input link name. Instead of logging the info through peek, we replaced peek by another copy stage (just to check) and hence now the reject link has 2 copy stages like copy1->copy2
we randomly get issues in the above mentioned flow where the job monitor log suggest that copy1 output link shows 1000 records whereas copy2 input link shows 0 [force property in both copy is set to TRUE].
i have a job which loads from a source dataset to a target dataset ( + alookup before loading to target) with some transformation rules involved which is handled through a transformer. There is a reject link configured for a lookup failure. The reject link has the following stages: copy stage and a peek stage. Link count is retrieved through peek stage name and peek input link name. Instead of logging the info through peek, we replaced peek by another copy stage (just to check) and hence now the reject link has 2 copy stages like copy1->copy2
we randomly get issues in the above mentioned flow where the job monitor log suggest that copy1 output link shows 1000 records whereas copy2 input link shows 0 [force property in both copy is set to TRUE].