librarylibs/Kit/src/Kit.xtl

Kit: the experiment kit's data sets, each with a known structure and made from its own seed: flat (Gaussian clusters), directional (classes that differ by direction, not length) and hierarchical (a tree, its distances along the tree, a class per branch, and a plain layout of it in the plane); and the seeded numbers they are made of. Import it with an alias of your choice: "kt:" u_se< "Kit". Put libs/Kit/src on XETAL_PATH ("just path"); the reference is libs/Kit/docs. Names with l: are exported; those under h: are private to this file.

A data set is (X, y): X a matrix, one point per row; y its labels, 1 to k. Every set is a function of its seed and its sizes alone (not of the run's seed, nor of what was drawn before it), so two demos asking for the same set get the same numbers: the seed is a whole number, and the generator is Park and Miller's (1988) multiplier 16807 modulo 2^31 - 1, written out here until X_eTaL-libraries' Random takes a seed of its own (docs/xetal-libraries-asks.md, GL8). The comparisons this repository makes are between geometries on the same data: what these functions give is held equal across them.

source · imports mf: libs/Manifold/src/Manifold.xtl; gr: work/xetal-libraries/libs/Graphs/src/Graphs.xtl work/xetal-libraries/libs/Graphs/src/Graphs.xtlm

Seeded numbers

ˡu̲niform : (Int, Int) -> Float

function · line 34

u̲niform (seed, n): n Floats in [0, 1), each a function of the seed and its place: the Park-Miller sequence x -> 16807 x mod (2^31 - 1) started from 1 + |seed| mod (2^31 - 2) (so seed 0 is a seed like any other and no seed is the sequence's fixed point 0), divided by 2^31 - 1. The same seed gives the same numbers in any run.

      "kt:" u̲se< "Kit"
      ᵏᵗu̲niform (42, 3)
0.00033653387815530127 0.6561248901561484 0.4910288543864288
ˡu̲niform ← { (seed, n) →
  m ← 2147483647
  start ← 1 + (a̲bs seed) m̲od m − 1
  (v, _) ← n '{ (v, x) → y ← (16807 × x) m̲od m◆ (v c̲at y, y) } p̲ower (0 r̲eshape 0, start)
  (f̲loat v) ÷ f̲loat m
}

ˡn̲ormal : (Int, Int) -> Float

function · line 47

n̲ormal (seed, n): n Floats from the standard normal distribution (mean 0, standard deviation 1), by Box and Muller from 2n uniform numbers of the seed.

      "kt:" u̲se< "Kit"
      ᵏᵗn̲ormal (42, 2)
-0.025904525013204074 -0.2561352551643429
ˡn̲ormal ← { (seed, n) →
  u ← ˡu̲niform (seed, 2 × n)
  a ← n t̲ake u
  b ← n d̲rop u
  (s̲qrt -2.0 × l̲og 1.0 − a) × c̲os 6.283185307179586 × b
}
Used in: ʰn̲ormals

ʰn̲ormals : (Int, Int, Int) -> Float

function (private) · line 56

n̲ormals (seed, rows, cols): a rows by cols matrix of standard normal numbers of the seed.

ʰn̲ormals ← { (seed, rows, cols) → (rows c̲at cols) r̲eshape ˡn̲ormal (seed, rows × cols) }

Flat clusters

ˡf̲lat : Num a => (Int, Int, Int, Int, a) -> (Float, Int)

function · line 74

f̲lat (seed, k, n, d, spread): k Gaussian clusters of n points each in d dimensions: the k centers are standard normal numbers of the seed scaled by 4 (so clusters sit a few units apart), and each point is its center plus spread times a standard normal vector; (X, y), X of k n rows and y the cluster of each row, 1 to k, in order. The structure is flat: straight-line distance is the right one, and a point's nearest neighbors are mostly its own cluster when spread is well under the centers' spacing.

      "kt:" u̲se< "Kit"
      (X, y) ← ᵏᵗf̲lat (1, 2, 2, 2, 0.1)
      y
1 1 2 2
      s̲hape X
4 2
ˡf̲lat ← { (seed, k, n, d, spread) →
  C ← 4.0 × ʰn̲ormals (seed, k, d)
  N ← ʰn̲ormals (seed + 1, k × n, d)
  y ← n ʰe̲ach r̲ange k
  X ← ((f̲loat spread) × N) + y s̲elect C
  (X, y)
}
Used in: X, F, F5

ʰe̲ach : Int -> Int -> Int

function (private) · line 84

n e̲ach v: every item of v repeated n times, in order (a general word, written out until GL2 lands).

ʰe̲ach ← { n v → r̲avel o̲\ (n c̲at t̲ally v) r̲eshape v }

Directions

ˡd̲irections : Num a => (Int, Int, Int, a) -> (Float, Int)

function · line 103

d̲irections (seed, k, n, cone): k classes of n points in three dimensions that differ by direction and not by length: each class has a direction (a seeded standard normal vector put onto the unit sphere) and each point is that direction plus cone times a standard normal vector, put back onto the sphere, then scaled by a length drawn between 0.5 and 2 (uniform), so the lengths carry nothing about the class; (X, y). The structure is directional: the angle between points is the right distance, and their straight-line distance mixes in the lengths.

      "kt:" u̲se< "Kit"
      (X, y) ← ᵏᵗd̲irections (7, 2, 2, 0.1)
      y
1 1 2 2
      s̲hape X
4 3
ˡd̲irections ← { (seed, k, n, cone) →
  D ← ᵐᶠs̲phereProject ʰn̲ormals (seed, k, 3)
  N ← ʰn̲ormals (seed + 1, k × n, 3)
  y ← n ʰe̲ach r̲ange k
  U ← ᵐᶠs̲phereProject ((f̲loat cone) × N) + y s̲elect D
  lengths ← 0.5 + 1.5 × ˡu̲niform (seed + 2, k × n)
  (lengths ʰs̲caleRows U, y)
}
Used in: D, D

ʰs̲caleRows : Float -> Float -> Float

function (private) · line 113

s v s̲caleRows M: each row of M scaled by the matching item of s.

ʰs̲caleRows ← { s M → M × s 'l̲eft t̲able o̲ffsets 2 s̲elect s̲hape M }

A tree

ˡt̲ree : (Int, Int) -> Int

function · line 128

t̲ree (depth, branching): the edges of a rooted tree in which every node but the leaves has branching children and every leaf is depth edges from the root: node 1 is the root and the nodes are numbered level by level, so node i's children are the nodes branching (i - 1) + 2 up to branching i + 1. The edges as Graphs takes them: two rows, from (the parent) over to (the child). Depth 4 and branching 2 give 31 nodes and 30 edges.

      "kt:" u̲se< "Kit"
      ᵏᵗt̲ree (2, 2)
1 1 2 2 3 3
2 3 4 5 6 7
ˡt̲ree ← { (depth, branching) →
  n ← ʰn̲odes (depth, branching)
  child ← 1 d̲rop r̲ange n
  parent ← 1 + (child − 2) d̲iv branching
  (2 c̲at n − 1) r̲eshape parent c̲at child
}

ʰn̲odes : (Int, Int) -> Int

function (private) · line 136

The number of nodes of the tree: 1 + b + b^2 + ... + b^depth.

ʰn̲odes ← { (depth, branching) → '+ r̲/ branching ^ (r̲ange depth + 1) − 1 }

ˡt̲reeDistances : (Int, Int) -> Float

function · line 148

t̲reeDistances (depth, branching): the distance between every two nodes of the tree along it (the number of edges on the one path between them), by Graphs' shortest paths over the tree's edges taken both ways: an n by n matrix of Floats, 0 on the diagonal, 2 depth at most (leaf to leaf through the root).

      "kt:" u̲se< "Kit"
      ᵏᵗt̲reeDistances (1, 2)
0.0 1.0 1.0
1.0 0.0 2.0
1.0 2.0 0.0
ˡt̲reeDistances ← { (depth, branching) →
  n ← ʰn̲odes (depth, branching)
  e ← ˡt̲ree (depth, branching)
  w ← n ᵍʳw̲eighted e c̲at (n − 1) r̲eshape 1
  ᵍʳs̲hortest ᵍʳu̲ndirected w
}

ˡt̲reeClasses : (Int, Int) -> Int

function · line 163

t̲reeClasses (depth, branching): the class of each node, its branch: the child of the root it descends from, 1 to branching; the root itself, which is under no branch, is given class 1 (and a classifier is excused it: it is alone). So the hierarchical set's labels, one per node.

      "kt:" u̲se< "Kit"
      ᵏᵗt̲reeClasses (2, 2)
1 1 2 1 1 2 2
ˡt̲reeClasses ← { (depth, branching) →
  n ← ʰn̲odes (depth, branching)
  parent ← f̲irst ˡt̲ree (depth, branching)
  up ← depth '{ v → '{ c → parent ʰu̲p c } e̲ach v } p̲ower r̲ange n           ⍝ each node's ancestor just under the root (the root stays)
  1 m̲ax up − 1
}

ʰu̲p : Int -> Int -> Int

function (private) · line 172

parent u̲p c: the parent of node c, unless that is the root (or c is), in which case c: one step up the tree that stops under the root.

ʰu̲p ← { parent c → c = 1 ? 1◆ p ← (c − 1) s̲elect parent◆ p = 1 ? c◆ p }

ˡt̲reeLayout : (Int, Int) -> Float

function · line 190

t̲reeLayout (depth, branching): a plain picture of the tree in the plane, one row (x, y) per node: the root at the origin, each level a circle of radius its depth, the nodes of a level spread evenly round it in their order, so that each subtree is a wedge (the two children of the root at the top and the bottom, their x cos of a quarter turn, 0 up to the machine's rounding). It is a layout, not an embedding that keeps the tree's distances: the straight-line distance between two nodes of this picture is a poor copy of their distance along the tree (two leaves of neighboring branches are near in the picture and 2 depth apart in the tree), which is what the baseline measures.

      "kt:" u̲se< "Kit"
      ᵏᵗt̲reeLayout (1, 2)
                                0.0  0.0
 0.00000000000000006123233995736766  1.0
-0.00000000000000018369701987210297 -1.0
ˡt̲reeLayout ← { (depth, branching) →
  n ← ʰn̲odes (depth, branching)
  level ← 1 s̲elect ˡt̲reeDistances (depth, branching)              ⍝ each node's distance from the root
  before ← '{ L → '+ r̲/ 0 c̲at branching ^ (r̲ange L) − 1 } e̲ach f̲loor level          ⍝ nodes on the levels above
  on ← f̲loat branching ^ f̲loor level                                 ⍝ nodes on the node's level
  pos ← (f̲loat (r̲ange n) − before) − 0.5                             ⍝ its place round the circle, from 0.5
  angle ← 6.283185307179586 × pos ÷ on
  o̲\ (2 c̲at n) r̲eshape (0.0 + level × c̲os angle) c̲at 0.0 + level × s̲in angle
}