Getting linkcount from previous stage

[abc123]

I have a job where data is going from a join stage to a transformer. The link between the join stage and the transformer stage is called Link1. In the transformer, in a stage variable, I need the count of rows coming into Link1 for a calculation. I don't want to use DSGetJobInfo for every row which will slow the system down enormously. How do I get around this problem?

[chulett]

What kind of 'count'? A running count can be had using the @INROWNUM system variable. The total count that is going to be coming? I'm afraid you won't know that until you've gotten them all.

If what you need is the latter, that typically involves two-pass processing or at least some kind of pre-processing so you know how many records you are going to be dealing with. Perhaps someone out there knows the best way to go about something like that in a PX job...

[abc123]

craig, thank you for your response. Yes, I need the final count. I have decided to create another job after this to handle this. I wanted to do this in one job but apparently this is not possible without evaluating it for every row.

[ray.wurlod]

You can do it in one job - in an after-job subroutine - but I think after a while you'll see the benefits of the more modular approach you've taken.

[Marley777]

Hello ABC123, how did you end up obtaining the final count in a seperate job? Thanks!!!

[abc123]

Yes RStone.

[Marley777]

I should have asked; What did you do in the seperate job to obtain the count?


Thanks!!