Finding max record number

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
mydsworld
Participant
Posts: 321
Joined: Thu Sep 07, 2006 3:55 am

Finding max record number

Post by mydsworld »

Can anyone plz let me know the method of finding the max input records number using Stage variable only (not using INROWNUM or other ).Please let me know what will be the intial declaration & derivations for stage variables.
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Intial Declaration for a stage variable is

Code: Select all

vCounter = @PartitionNum - @NumPartitions
And the Derivation should be

Code: Select all

vCounter = vCounter + @NumPartitions
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
mydsworld
Participant
Posts: 321
Joined: Thu Sep 07, 2006 3:55 am

Post by mydsworld »

This gives me record numbers for each record. I want the max record number
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Sorry... not sure why I give the previous input. :?
You cant find the max record count using transformer. Atleast using Transformer alone.
You might need to use a Aggregator/Sort/RemoveDuplicate stages to find it if coupled with Transformer, or Count with Aggregator stage.
You can possibly call the LinkRowNum function. But it wont give you the right result, if you call the function, within the same job.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

You can find the maximum on each partition. This example is for integer.
Initialize stage variable svMaxValue to -2147483648 (for VarChar initialize to ""). Then assign its value as

Code: Select all

If InLink.TheColumn > svMaxValue Then InLink.TheColumn Else svMaxValue
Downstream in the job, you bring these partitions together using a collector of some kind and repeat the process. Now, because there is only one stream, you have the maximum over all partitions.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

ray.wurlod wrote:You can find the maximum on each partition. This example is for integer.
Initialize stage variable svMaxValue to -2147483648 (for VarChar initialize to ""). Then assign its value as

Code: Select all

If InLink.TheColumn > svMaxValue Then InLink.TheColumn Else svMaxValue
Downstream in the job, you bring these partitions together using a collector of some kind and repeat the process. Now, because there is only one stream, you have the maximum over all partitions.
I guess this is for Max Value. And It could be very similar for Max record count as asked for. Still it would required another stage to find the Max Or should grep for last record.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

You know, I started to reply in terms of record count, and somehow switched to maximum value. There should be no difficulty initializing a stage variable to 0 and adding 1 to it for each row processed, then summing and/or maxing these downstream after collecting onto a single stream.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
John Smith
Charter Member
Charter Member
Posts: 193
Joined: Tue Sep 05, 2006 8:01 pm
Location: Australia

Re: Finding max record number

Post by John Smith »

mydsworld wrote:Can anyone plz let me know the method of finding the max input records number using Stage variable only (not using INROWNUM or other ).Please let me know what will be the intial declaration & derivations for stage variables.
hi,
what is your input ? is that a table or a file or a dataset? perhaps there's an easier way to do the count to get max input records before the job? then pass that value as a job parameter that can be used within the job.
sorry if i am way off the mark not knowing your exact situation. just a thot.
rgds
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

A "thot" ? Rhymes with "zot"?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply