librarylibs/Statistics/src/Statistics.xtl

Statistics: statistics -- median, quantiles, five-number summaries, z-scores, covariance, correlation, a least-squares line, binned counts; builds on the standard Stats. Import it with an alias of your choice: "sx:" u_se< "Statistics". Put libs/Statistics/src on XETAL_PATH ("just path"); the reference is libs/Statistics/docs. Names with l: are exported; those under h: are private to this file.

X_eTaL's standard Stats has the mean, variance (of the population), standard deviation and range: import it beside this one ("s:" u_se< "Stats"). Results here are Floats.

source · imports s: std/Stats.xtl

Summaries

ʰq̲1 : Float -> Float -> Float

function (private) · line 18

The p quantile of sorted Floats x (0 <= p <= 1), interpolating between the two nearest items (R's type 7, Excel's PERCENTILE).

ʰq̲1 ← { x p →
  h ← p × f̲loat (t̲ally x) − 1              ⍝ 0-origin position
  lo ← f̲loor h
  hi ← (lo + 1) m̲in (t̲ally x) − 1
  a ← f̲irst (1 + lo) s̲elect x
  b ← f̲irst (1 + hi) s̲elect x
  a + (h − f̲loat lo) × b − a
}

Used in: ˡq̲uantile

ˡq̲uantile : (Num a, Num b) => a -> b -> Float

function · line 28

p q_uantile v: the p quantile of v for each p (0.5 is the median).

ˡq̲uantile ← { p v →
  0 = t̲ally v ? @ p̲anic< "the list is empty: a median or quantile needs at least one value"
  '∨ r̲/ r̲avel (p < 0) ∨ p > 1 ? @ p̲anic< "a quantile is from 0 to 1, not {p}"
  x ← s̲ort f̲loat v
  '{ q → x ʰq̲1 q } e̲ach f̲loat p
}

p̲anic< expands to

(⎕P̲ANIC ("the list is empty: a median or quantile needs at least one value"))

p̲anic< expands to

(⎕P̲ANIC ("a quantile is from 0 to 1, not " c̲at (f̲ormat (p))))

Used in: ˡm̲edian, ˡf̲ive

ˡm̲edian : Num a => a -> Float

function · line 37

m_edian v: the middle value (the mean of the two middle ones for an even count).

ˡm̲edian ← { v → f̲irst 0.5 ˡq̲uantile v }

Used in: libs/Csv/demos/cities.xtl:21

ˡf̲ive : Num a => a -> Float

function · line 41

f_ive v: the five-number summary: least, lower quartile, median, upper quartile, greatest.

ˡf̲ive ← { v → 0.0 0.25 0.5 0.75 1.0 ˡq̲uantile v }

Used in: libs/Statistics/demos/heights.xtl:11

ˡz̲scores : Num a => a -> Float

function · line 44

z_scores v: how many standard deviations each item is from the mean.

ˡz̲scores ← { v →
  0.0 = ˢs̲d v ? @ p̲anic< "the values do not vary (their standard deviation is 0): z-scores are undefined"
  ((f̲loat v) − ˢm̲ean v) ÷ ˢs̲d v
}

p̲anic< expands to

(⎕P̲ANIC ("the values do not vary (their standard deviation is 0): z-scores are undefined"))

Used in: z

Relationships

ˡc̲ovariance : (Num a, Num b) => a -> b -> Float

function · line 52

a c_ovariance b: the mean product of the deviations (population).

ˡc̲ovariance ← { a b → ˢm̲ean ((f̲loat a) − ˢm̲ean a) × (f̲loat b) − ˢm̲ean b }

Used in: ˡc̲orrelation, ˡf̲it

ˡc̲orrelation : (Num a, Num b) => a -> b -> Float

function · line 55

a c_orrelation b: Pearson's correlation, from -1 to 1.

ˡc̲orrelation ← { a b →
  (0.0 = ˢs̲d a) ∨ 0.0 = ˢs̲d b ? @ p̲anic< "a list that does not vary (standard deviation 0) has no correlation"
  (a ˡc̲ovariance b) ÷ (ˢs̲d a) × ˢs̲d b
}

p̲anic< expands to

(⎕P̲ANIC ("a list that does not vary (standard deviation 0) has no correlation"))

Used in: libs/Csv/demos/cities.xtl:22, libs/Statistics/demos/heights.xtl:13

ˡf̲it : (Num a, Num b) => a -> b -> Float

function · line 62

x f_it y: the least-squares line through the points, intercept and slope (y is about intercept + slope * x).

ˡf̲it ← { x y →
  slope ← (x ˡc̲ovariance y) ÷ ˢv̲ariance x
  ((ˢm̲ean y) − slope × ˢm̲ean x) c̲at slope
}

Used in: f

Binning

ˡb̲ins : Num a => Int -> a -> Int

function · line 71

n b_ins v: how many items of v fall in each of n bins of equal width from its least to its greatest (the greatest in the last bin).

ˡb̲ins ← { n v →
  lo ← 'm̲in r̲/ f̲loat v
  step ← 0.000000000001 m̲ax (('m̲ax r̲/ f̲loat v) − lo) ÷ f̲loat n
  bin ← 1 + (n − 1) m̲in f̲loor ((f̲loat v) − lo) ÷ step
  '{ t̲ally w̲here bin = ⍵ } e̲ach r̲ange n
}