Challenging Task

dganeshm · Post by **dganeshm** » Wed Jan 27, 2010 2:26 pm

Hi ,

http://picasaweb.google.com/dganeshm/DS ... 6817687890

What I need to do is, I need to get the total records that is originating from ODBC_0 , and the number of rejected records from Transformer_27. But have to insert these numbers in the same row in a table. When I tried to do it , I was able to insert but not update , since the insert hadnt still been commited, so was unable to update, can someone help me with this one..

ray.wurlod · Post by **ray.wurlod** » Wed Jan 27, 2010 3:19 pm

Consider a separate job to read the link row counts from the completed job and put the row into the table.

Create a generic routine to read a link row count, given the job name, active stage name and link name.

dganeshm · Post by **dganeshm** » Wed Jan 27, 2010 3:26 pm

ray.wurlod wrote:Consider a separate job to read the link row counts from the completed job and put the row into the table.

Create a generic routine to read a link row count, given the job name, active stage name and link name.

Is it possible to get the total rows passing through in a xformer using stage variable?? I think I can avoid using the aggregator..which has its own limitations..

ray.wurlod · Post by **ray.wurlod** » Wed Jan 27, 2010 3:35 pm

Of course it is, but what the stage variable - or even the system variable - will have is the total number of rows so far. So you would need an Aggregator to trap just the final value. And then you have the issue of how to get both values into the same row in the table without causing locking problems. That, too, is do-able but I think the approach I suggested earlier is cleaner.

Kryt0n · Post by **Kryt0n** » Wed Jan 27, 2010 4:04 pm

Make the outputs from the aggregators in to reference files to a lookup joining back on to the source rows... you may have to push all links to passive stages prior to doing a lookup.

Or separate jobs as previously suggested...

Question is why you need to log an aggregated value against all source rows...

dganeshm · Post by **dganeshm** » Wed Jan 27, 2010 4:19 pm

Kryt0n wrote:Make the outputs from the aggregators in to reference files to a lookup joining back on to the source rows... you may have to push all links to passive stages prior to doing a lookup.

Or separate jobs as previously suggested...

Question is why you need to log an aggregated value against all source rows...

The reason is to reocrd total source and the number od rows rejected by the web service.