baseline-knn.xtl

1⍝!/usr/bin/env xetal
2⍝│ Baseline, nearest neighbors: k nearest neighbors under the
3⍝│ straight-line distance on the three data sets of the kit (flat
4⍝│ clusters, directions, a tree), each split the same seeded way, the
5⍝│ held-out accuracy and the cost of each run recorded as the
6⍝│ experiment schema says; and for the tree, how badly a plain layout
7⍝│ of it keeps its distances. The row every comparison of H1 is set
8⍝│ against: no geometry chosen yet.
9"kt:" u̲se< "Kit"
10"ml:" u̲se< "Learn"
11"mf:" u̲se< "Manifold"
12"ch:" u̲se< "Check"
13
14⍝ -- the kit
15⍝│ The split, the run and its record, and the comparison of two runs
16⍝│ are machine learning and not about manifolds: X_eTaL-ML's by the
17⍝│ plan's G3, asked for as GM5 (docs/xetal-ML-asks.md), and written
18⍝│ out here, few and short, until it lands.
19
20⍝│ ᵘs̲plit (seed, n): the held-out quarter of n points, seeded: a
21⍝│ seeded order of 1 to n (the grade of seeded uniform numbers), the
22⍝│ first quarter held out and the rest trained on; (train, held), two
23⍝│ vectors of row numbers.
24ᵘs̲plit ← { (seed, n) → order ← g̲rade ᵏᵗu̲niform (seed, n)◆ h ← n d̲iv 4◆ (h d̲rop order, h t̲ake order) }
25
26⍝│ ᵘr̲un (X, y, k, seed): one run of k nearest neighbors on the set
27⍝│ (X, y) under the straight-line distance (Learn's k_nn), the
28⍝│ held-out quarter guessed from the rest; the record, as the schema
29⍝│ says: the held-out accuracy (measured), how many were held out,
30⍝│ the numbers trained (none: nearest neighbors learns no number) and
31⍝│ the operations counted (one distance per held-out point per
32⍝│ training point, each of d coordinates). Runtime is never recorded.
33ᵘr̲un ← { (X, y, k, seed) →
34  (train, held) ← ᵘs̲plit (seed, f̲irst s̲hape X)
35  guess ← (k, train s̲elect X, train s̲elect y) ᵐˡk̲nn held s̲elect X
36  right ← '+ r̲/ guess = held s̲elect y
37  accuracy ← (f̲loat right) ÷ f̲loat t̲ally held
38  accuracy c̲at (f̲loat t̲ally held) c̲at 0.0 c̲at f̲loat (t̲ally train) × (t̲ally held) × 2 s̲elect s̲hape X
39}
40
41⍝│ a ᵘc̲ompare b: two records side by side and their difference,
42⍝│ measure by measure (the second less the first), as three rows.
43ᵘc̲ompare ← { a b → (3 c̲at t̲ally a) r̲eshape (f̲loat a) c̲at (f̲loat b) c̲at (f̲loat b) − f̲loat a }
44
45⍝│ ᵘc̲hords Q: the straight-line distance between every pair of
46⍝│ rows (a general function, for X_eTaL-libraries: GL1; written out
47⍝│ here as in sphere-hello).
48ᵘc̲hords ← { Q → n ← f̲irst s̲hape Q◆ (D, _) ← (2 s̲elect s̲hape Q) '{ (d, k) → c ← k s̲elect₂ Q◆ (d + (c '− t̲able c) ^ 2.0, k + 1) } p̲ower ((n c̲at n) r̲eshape 0.0, 1)◆ s̲qrt D }
49
50⍝ -- the runs
51k ← 5
52splitSeed ← 11
53
54⍝━━ Flat clusters
55
56⍝│ Three clusters of 40 points in the plane, spread 1.6 about centers
57⍝│ a few units apart (seed 1), so the clusters touch; and four
58⍝│ clusters of 30 in five dimensions, spread 1.6 (seed 2). The record: accuracy, held out,
59⍝│ numbers trained, operations.
60(F, fy) ← ᵏᵗf̲lat (1, 3, 40, 2, 1.6)
61flat ← ᵘr̲un (F, fy, k, splitSeed)
62flat
63"flat clusters: most held-out points go to their cluster" ᶜʰt̲est 1 ᶜʰi̲s 0.8 < 1 s̲elect flat
64(F5, f5y) ← ᵏᵗf̲lat (2, 4, 30, 5, 1.6)
65flat5 ← ᵘr̲un (F5, f5y, k, splitSeed)
66flat5
67"flat clusters in five dimensions: the same" ᶜʰt̲est 1 ᶜʰi̲s 0.8 < 1 s̲elect flat5
68
69⍝━━ Directions
70
71⍝│ Four classes of 40 points that differ by direction (cone 0.6
72⍝│ about each class direction, so the classes overlap at their edges)
73⍝│ and not by length (lengths from 0.5 to 2; seed 5). The straight-line distance mixes the lengths in, which
74⍝│ carry nothing about the class: this is the row the sphere is
75⍝│ compared against (representation-explorer).
76(D, dy) ← ᵏᵗd̲irections (5, 4, 40, 0.6)
77directions ← ᵘr̲un (D, dy, k, splitSeed)
78directions
79"directions: the straight line gets most right, lengths and all" ᶜʰt̲est 1 ᶜʰi̲s 0.6 < 1 s̲elect directions
80⍝│ The same set with every point put onto the unit sphere (lengths
81⍝│ dropped), still under the straight-line distance between the unit
82⍝│ points: what the lengths were costing, measured; the comparison
83⍝│ rows are (lengths kept, lengths dropped, the difference).
84unit ← ᵘr̲un (ᵐᶠs̲phereProject D, dy, k, splitSeed)
85directions ᵘc̲ompare unit
86"directions: dropping the lengths does not lose accuracy" ᶜʰt̲est 1 ᶜʰi̲s (1 s̲elect unit) ≥ 1 s̲elect directions
87
88⍝━━ A tree
89
90⍝│ The tree of depth 4 and branching 2 (31 nodes), each node's class
91⍝│ its branch, and the plain layout of it in the plane: nearest
92⍝│ neighbors on the layout's coordinates (k = 3, the held-out quarter
93⍝│ is 7 nodes), and the distortion of the layout's straight-line
94⍝│ distances against the distances along the tree (Manifold's: the
95⍝│ mean and the worst pair), which is what the hyperbolic disk is
96⍝│ compared against (representation-explorer).
97T ← ᵏᵗt̲reeDistances (4, 2)
98L ← ᵏᵗt̲reeLayout (4, 2)
99ty ← ᵏᵗt̲reeClasses (4, 2)
100tree ← ᵘr̲un (L, ty, 3, splitSeed)
101tree
102"tree: the layout's halves give the branch away" ᶜʰt̲est 1 ᶜʰi̲s 0.8 < 1 s̲elect tree
103distortion ← T ᵐᶠd̲istortion ᵘc̲hords L
104distortion
105"tree: the layout keeps the tree's distances badly (the mean, the worst pair)" ᶜʰt̲est 1 1 ᶜʰi̲s ((1 s̲elect distortion) > 0.15) c̲at (2 s̲elect distortion) > 0.75
106⍝ -- end of the runs
107
108⍝━━ The record of every run
109⍝│ One row per run: flat (plane), flat (five dimensions), directions,
110⍝│ directions on the unit sphere, the tree's layout; the columns
111⍝│ accuracy, held out, numbers trained, operations. All measured. The
112⍝│ baseline row of the H1 table (docs/experiments.md).
1135 4 r̲eshape flat c̲at flat5 c̲at directions c̲at unit c̲at tree