standard deviation

Archive of postings to DataStageUsers@Oliver.com. This forum intended only as a reference and cannot be posted to.

Moderators: chulett, rschirm

Locked
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

standard deviation

Post by admin »

Does anyone know of a function to calculate standard deviation within Datastage?

Thanks,

Mario Dudley
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

None exists, but its not difficult to write one yourself PROVIDED that you can deliver the count of values, the sum of values and the sum of squared values to the function. This is tricky to do in a streaming environment, as you need to capture "breakpoints" - that is, where the value of the categorising column changes. This can be done with, for example, stage variables.

The formula for standard deviation of X is
Sqrt(((N * Sum(X**2) - Sum(X) * Sum(X)) / (N * (N - 1)))
where N is the count of values of X, Sum(X) is the sum of values of X, and
Sum(X**2) is the sum of squared values of X. Clearly, since theres an (N -
1) in the denominator, this function can not be used on a detail line (where N = 1), since this would cause a divide by zero error.

-----Original Message-----
From: Dudley, Mario M [mailto:Mario.Dudley@PSS.Boeing.com]
Sent: Friday, 17 August 2001 05:16
To: datastage-users@oliver.com
Subject: standard deviation


Does anyone know of a function to calculate standard deviation within Datastage?

Thanks,

Mario Dudley
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Mario,
There is no function, but you may be able to accomplish it by using the
aggregator stage.

PM


----Original Message Follows----
From: "Dudley, Mario M"
Reply-To:
To: "datastage-users@oliver.com"
Subject: standard deviation
Date: Thu, 16 Aug 2001 14:15:33 -0700

Does anyone know of a function to calculate standard deviation within
Datastage?

Thanks,

Mario Dudley



_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
Locked