Issue with link count in Modify stage

xch2005 · Post by **xch2005** » Fri Feb 19, 2010 5:10 am

Hi,

The modify stage is used to format the rejects from odbc stage and capture the records in a flat file. After execution of the job, when looked at the monitor to view the no. of records in each link, the primary and output links for the modify stage shows as 0 though the records are present in both the links. This scenario happens when the execution mode of the modify stage is "Parallel", also the status is shown as "unknown".

When the execution mode is changed in the modify stage to "Sequential" then the link shows the count and status is "Finished". Simlarly when the primary link is normal link the count is displayed regardless of the execution mode.

The problem occurs when the source link is a reject link.

Any thoughts on this?

Thanks in advance.

rohithmuthyala · Post by **rohithmuthyala** » Mon Feb 22, 2010 2:43 am

I think this is a common problem while using modify stage....Similar kind of problem i've faced while using modify stage...the counts won't be shown but the records do flow and i got the desired output.

ray.wurlod · Post by **ray.wurlod** » Mon Feb 22, 2010 3:03 am

You can sometimes see the same thing with the Copy stage. The answer is in the score - the stage is actually taken out by the optimizer. In the case of a Modify stage, it can be made a property of the input link of its downstream stage.

xch2005 · Post by **xch2005** » Thu Mar 04, 2010 1:01 am

when viewed in Director (Monitor) the status for Modify stage/link shows "Unknown", is there any settings change required?

If the execution is seq. then the stauts shows as "Finished" as expected.

xch2005 · Post by **xch2005** » Thu Mar 04, 2010 1:02 am

when viewed in Director (Monitor) the status for Modify stage/link shows "Unknown", is there any settings change required?

If the execution is seq. then the stauts shows as "Finished" as expected.

ray.wurlod · Post by **ray.wurlod** » Thu Mar 04, 2010 1:22 am

The reason for "unknown" and no row count is that there is no Modify stage in the executed job.

Please re-read my previous post.

xch2005 · Post by **xch2005** » Thu Mar 04, 2010 4:42 am

Thanks Ray.

Actually we are capturing the rowcount and the link names and display the stats on a front end. Though there are records from the link, it shows as 0 (for input & output link) in the modify stage which is incorrect.

Basically we are dropping few columns and capturing the reject records.

Processing is done with two nodes and xml also shows rowcount as 0.

Any suggestions on getting the correct rowcount please

ray.wurlod · Post by **ray.wurlod** » Thu Mar 04, 2010 2:56 pm

Examine the score. Satisfy yourself that there is no Modify stage (operator). Try using the stage on the other end of the link as the stage name argument to DSGetLinkInfo().

xch2005 · Post by **xch2005** » Mon Apr 26, 2010 6:24 am

Thanks Ray.

As an interim solution using the execution mode as sequential to get the count.