Tuples
a state of several arrays as one value: packed by position, then a tuple taken apart by name

Table of Contents

An array holds items of one type in a shape. Some values are several arrays of different shapes and types that travel together: a network's weights with the running averages its optimizer keeps and a count of steps; a game's board with its score; a stone's three axes with the pointer dragging one of them. Before tuples such a state was packed into one array, and every function took it apart by position. A tuple keeps the parts side by side, each with its own type, and a pattern takes them apart by name.

A tuple and a pattern

A tuple is written with commas inside parentheses. Its type is written as the literal is, and its parts keep their own types and shapes: an Int vector, a Float and a text here.

(1 2 3, 4.5, "abc")

⍝ typed:
⍝   (1 2 3, 4.5, "abc")
(1 2 3, 4.5, abc)

A pattern, in a binding or in a lambda's parameters, gives each part a name; _ ignores a part and patterns nest. A pattern of the wrong size is a type error, found before the program runs.

(n, (x, _)) ← (3, (2.5, "unused"))
x × f̲loat n
ᵘs̲wapped ← { (a, b) → (b, a) }
ᵘs̲wapped (1, "x")

⍝ typed:
⍝   (n, (x, _)) := (3, (2.5, "unused"))
⍝   x * f_loat n
⍝   u:s_wapped := { (a, b) -> (b, a) }
⍝   u:s_wapped (1, "x")
7.5
(x, 1)

An optimizer's state, packed

Adam keeps, for every weight array, two running averages: of the gradients and of their squares. A step of Adam reads the weights, the averages and the step count, and gives all of them back, so p_ower can repeat it. Two weight arrays here (a vector and a matrix) learn to equal two targets; the gradient of the squared distance is twice the difference.

target1 ← 1.0 2.0 3.0
target2 ← 2 2 r̲eshape 0.5 -0.5 0.25 -0.25
ᵘg̲rad ← { W1 W2 → (e̲nclose 2.0 × W1 − target1) c̲at e̲nclose 2.0 × W2 − target2 }
lr ← 0.1
ᵘr̲ound ← { a → (f̲loat f̲loor 0.5 + 1000.0 × a) ÷ 1000.0 }

⍝ typed:
⍝   target1 := 1.0 2.0 3.0
⍝   target2 := 2 2 r_eshape 0.5 -0.5 0.25 -0.25
⍝   u:g_rad := { W1 W2 -> (e_nclose 2.0 * W1 - target1) c_at e_nclose 2.0 * W2 - target2 }
⍝   lr := 0.1
⍝   u:r_ound := { a -> (f_loat f_loor 0.5 + 1000.0 * a) / 1000.0 }

Without tuples the seven parts (two weight arrays, four averages and the count) are boxed into one vector, and every part is reached by its position: d_isclose 3 s_elect s is the first average because the author remembers that it is third. The count is a Float, since a vector of boxes holds one item type.

ᵘm̲oveBy ← { mv k → ((d̲isclose 1 s̲elect mv) ÷ 1.0 − 0.9 ^ k) ÷ 0.00000001 + ((d̲isclose 2 s̲elect mv) ÷ 1.0 − 0.999 ^ k) ^ 0.5 }
ᵘp̲acked ← { s →
  W1 ← d̲isclose 1 s̲elect s
  W2 ← d̲isclose 2 s̲elect s
  k ← 1.0 + d̲isclose 7 s̲elect s
  g ← W1 ᵘg̲rad W2
  G1 ← d̲isclose 1 s̲elect g
  G2 ← d̲isclose 2 s̲elect g
  M1 ← (0.9 × d̲isclose 3 s̲elect s) + 0.1 × G1
  M2 ← (0.9 × d̲isclose 4 s̲elect s) + 0.1 × G2
  V1 ← (0.999 × d̲isclose 5 s̲elect s) + 0.001 × G1 × G1
  V2 ← (0.999 × d̲isclose 6 s̲elect s) + 0.001 × G2 × G2
  N1 ← W1 − lr × ((e̲nclose M1) c̲at e̲nclose V1) ᵘm̲oveBy k
  N2 ← W2 − lr × ((e̲nclose M2) c̲at e̲nclose V2) ᵘm̲oveBy k
  (e̲nclose N1) c̲at (e̲nclose N2) c̲at (e̲nclose M1) c̲at (e̲nclose M2) c̲at (e̲nclose V1) c̲at (e̲nclose V2) c̲at e̲nclose k
}
z1 ← 0.0 0.0 0.0
z2 ← 2 2 r̲eshape 0.0
p0 ← (e̲nclose z1) c̲at (e̲nclose z2) c̲at (e̲nclose z1) c̲at (e̲nclose z2) c̲at (e̲nclose z1) c̲at (e̲nclose z2) c̲at e̲nclose 0.0
p ← 200 'ᵘp̲acked p̲ower p0
ᵘr̲ound d̲isclose 1 s̲elect p

⍝ typed:
⍝   u:m_oveBy := { mv k -> ((d_isclose 1 s_elect mv) / 1.0 - 0.9 ^ k) / 0.00000001 + ((d_isclose 2 s_elect mv) / 1.0 - 0.999 ^ k) ^ 0.5 }
⍝   u:p_acked := { s ->
⍝     W1 := d_isclose 1 s_elect s
⍝     W2 := d_isclose 2 s_elect s
⍝     k := 1.0 + d_isclose 7 s_elect s
⍝     g := W1 u:g_rad W2
⍝     G1 := d_isclose 1 s_elect g
⍝     G2 := d_isclose 2 s_elect g
⍝     M1 := (0.9 * d_isclose 3 s_elect s) + 0.1 * G1
⍝     M2 := (0.9 * d_isclose 4 s_elect s) + 0.1 * G2
⍝     V1 := (0.999 * d_isclose 5 s_elect s) + 0.001 * G1 * G1
⍝     V2 := (0.999 * d_isclose 6 s_elect s) + 0.001 * G2 * G2
⍝     N1 := W1 - lr * ((e_nclose M1) c_at e_nclose V1) u:m_oveBy k
⍝     N2 := W2 - lr * ((e_nclose M2) c_at e_nclose V2) u:m_oveBy k
⍝     (e_nclose N1) c_at (e_nclose N2) c_at (e_nclose M1) c_at (e_nclose M2) c_at (e_nclose V1) c_at (e_nclose V2) c_at e_nclose k
⍝   }
⍝   z1 := 0.0 0.0 0.0
⍝   z2 := 2 2 r_eshape 0.0
⍝   p0 := (e_nclose z1) c_at (e_nclose z2) c_at (e_nclose z1) c_at (e_nclose z2) c_at (e_nclose z1) c_at (e_nclose z2) c_at e_nclose 0.0
⍝   p := 200 'u:p_acked p_ower p0
⍝   u:r_ound d_isclose 1 s_elect p
1.0 2.0 3.0

The same state as a tuple

With a tuple the state is (W1, W2, M1, M2, V1, V2, k): the step names its parts once, in its parameter, and gives a tuple back. The count is an Int. Nothing is boxed, nothing is selected by number, and a step that gave back six parts instead of seven would not type-check.

ᵘm̲ove ← { (m, v) k → (m ÷ 1.0 − 0.9 ^ k) ÷ 0.00000001 + (v ÷ 1.0 − 0.999 ^ k) ^ 0.5 }
ᵘa̲dam ← { (W1, W2, M1, M2, V1, V2, k) →
  t ← 1 + k
  g ← W1 ᵘg̲rad W2
  (G1, G2) ← (d̲isclose 1 s̲elect g, d̲isclose 2 s̲elect g)
  m1 ← (0.9 × M1) + 0.1 × G1
  m2 ← (0.9 × M2) + 0.1 × G2
  v1 ← (0.999 × V1) + 0.001 × G1 × G1
  v2 ← (0.999 × V2) + 0.001 × G2 × G2
  (W1 − lr × (m1, v1) ᵘm̲ove f̲loat t, W2 − lr × (m2, v2) ᵘm̲ove f̲loat t, m1, m2, v1, v2, t)
}
s0 ← (z1, z2, z1, z2, z1, z2, 0)
(w1, w2, _, _, _, _, steps) ← 200 'ᵘa̲dam p̲ower s0
ᵘr̲ound w1
steps

⍝ typed:
⍝   u:m_ove := { (m, v) k -> (m / 1.0 - 0.9 ^ k) / 0.00000001 + (v / 1.0 - 0.999 ^ k) ^ 0.5 }
⍝   u:a_dam := { (W1, W2, M1, M2, V1, V2, k) ->
⍝     t := 1 + k
⍝     g := W1 u:g_rad W2
⍝     (G1, G2) := (d_isclose 1 s_elect g, d_isclose 2 s_elect g)
⍝     m1 := (0.9 * M1) + 0.1 * G1
⍝     m2 := (0.9 * M2) + 0.1 * G2
⍝     v1 := (0.999 * V1) + 0.001 * G1 * G1
⍝     v2 := (0.999 * V2) + 0.001 * G2 * G2
⍝     (W1 - lr * (m1, v1) u:m_ove f_loat t, W2 - lr * (m2, v2) u:m_ove f_loat t, m1, m2, v1, v2, t)
⍝   }
⍝   s0 := (z1, z2, z1, z2, z1, z2, 0)
⍝   (w1, w2, _, _, _, _, steps) := 200 'u:a_dam p_ower s0
⍝   u:r_ound w1
⍝   steps
1.0 2.0 3.0
200

Both reach the same weights. The tuple version is the one X_eTaL-ML's training demo uses (its ask M13): its state was 99 weights and two averages packed into one vector of 298 numbers, taken apart by t_ake and d_rop at fixed offsets.

(ᵘr̲ound w1) m̲atch ᵘr̲ound d̲isclose 1 s̲elect p
(ᵘr̲ound w2) m̲atch ᵘr̲ound d̲isclose 2 s̲elect p

⍝ typed:
⍝   (u:r_ound w1) m_atch u:r_ound d_isclose 1 s_elect p
⍝   (u:r_ound w2) m_atch u:r_ound d_isclose 2 s_elect p
1
1

The Rosetta stone's state, before and after

The Rosetta stone (demos/rosetta/Comparison.xtl) kept its whole state in one 5 x 5 Float matrix: three rows for the axes (count, steps, angle, playing), one for the pointer, one for the attract mode's clock. A cell was read by its row and column number and written through a mask, and the counts were Floats because a matrix has one element type. Before:

31 cm:s_tart 7
31.0 0.0 0.0 1.0 0.0
 7.0 0.0 0.0 1.0 0.0
 7.0 0.0 0.0 1.0 0.0
 0.0 0.0 0.0 0.0 0.0
20.0 0.0 0.0 0.0 0.0
(cm:TOP c_at cm:STEPS) cm:a_t (cm:TOP c_at 7.0) cm:c_hoose 31 cm:s_tart 7
-1.0

After, the state is (axes, pointer, clock). The axes are four vectors of three, one per field (a struct of arrays, so an axis number still selects an axis); counts and steps are Ints. A call names the axis and its argument as a pair, and a field is read by name.

"cm:" u̲se< "Comparison"
s ← 31 ᶜᵐs̲tart 7
s
ᶜᵐTOP ᶜᵐs̲teps (ᶜᵐTOP, 7) ᶜᵐc̲hoose s
ᶜᵐTOP ᶜᵐa̲ngle 0.1 ᶜᵐt̲ick (ᶜᵐTOP, 7) ᶜᵐc̲hoose s

⍝ typed:
⍝   "cm:" u_se< "Comparison"
⍝   s := 31 cm:s_tart 7
⍝   s
⍝   cm:TOP cm:s_teps (cm:TOP, 7) cm:c_hoose s
⍝   cm:TOP cm:a_ngle 0.1 cm:t_ick (cm:TOP, 7) cm:c_hoose s
((31 7 7, 0 0 0, 0.0 0.0 0.0, 1 1 1), (0, 0.0 0.0, 0.0, 0.0), (20.0, 0.0, 0, 0))
-1
-18.0

Inside the library, a function takes the state apart with a pattern and builds the new one from its parts, cm:t_ouched for one:

l:t_ouched := { state -> (axes, q, (_, _, b, i)) := state; (axes, q, (0.0, 0.0, b, i)) }

When to use a tuple

A tuple groups parts of different types or shapes that travel together and are taken apart by name. A fixed count of same-typed numbers that are added, scaled or joined like any vector stays an array: a tuple has no arithmetic (= and + do not reach into it, only m_atch compares) and no shape. A tuple can be an item of an array only in a box: m_ap of a function giving a tuple boxes each result.

'{ k → (k, f̲loat k) } m̲ap 1 2

⍝ typed:
⍝   '{ k -> (k, f_loat k) } m_ap 1 2
┌→──────────────────────┐
│ ┌────────┐ ┌────────┐ │
│ │(1, 1.0)│ │(2, 2.0)│ │
│ └∊───────┘ └∊───────┘ │
└∊──────────────────────┘

Literate documents · Live demo · Repository