programdemos/world-model/world-model.xtl

A tiny world model: a network that has watched a ball bounce in a box predicts the next frame from the last three. The world is a 2 x 2 ball in a 12 x 12 box, moving 1 or 2 cells a step on each axis and bouncing off the walls: 1,936 states in all. The network (432 pixels in, 64 ReLU units, 144 pixels out) was trained offline on four fifths of them (train/); the other fifth it never saw. Here it predicts the next frame of every state at once, as one batch, then dreams: fed its own predictions, it plays the ball forward on its own.

source · imports nn: libs/NN/src/NN.xtl; act: libs/Activations/src/Activations.xtl

The world

ᵘs̲tep : (Num a, Truthy a) => a -> a

function · line 17

One step of the states s (rows of x y vx vy): move, and bounce off a wall at 0 or 10 (the ball is 2 wide in a box of 12).

ᵘs̲tep ← { s →
  P ← 2 t̲ake o̲\ s
  V ← -2 t̲ake o̲\ s
  n ← P + V
  lo ← n < 0
  hi ← n > 10
  o̲\ (n + ((n̲eg 2 × n) × lo) + (20 − 2 × n) × hi) c̲at V × 1 − 2 × lo + hi
}
Used in: ᵘw̲indow, truth

ᵘi̲n : Num a => a -> Float

function · line 26

1.0 where a ball's corner is at the place or one before it (D is -1 or 0).

ᵘi̲n ← { D → f̲loat (D ≤ 0) × D ≥ -1 }
Used in: ᵘf̲rames

ᵘf̲rames : Int -> Float

function · line 28

The frames of the balls at xy (rows of x y), one row of 144 pixels each.

ᵘf̲rames ← { xy →
  c ← o̲ffsets 144
  (ᵘi̲n (2 s̲elect o̲\ xy) '− t̲able c d̲iv 12) × ᵘi̲n (1 s̲elect o̲\ xy) '− t̲able c m̲od 12
}
Used in: ᵘw̲indow, dream

ᵘw̲indow : Int -> (Float, Float)

function · line 34

The window of the states s: their frame and the next two, side by side (432 pixels a row), then the frame after them.

ᵘw̲indow ← { s →
  s1 ← ᵘs̲tep s
  s2 ← ᵘs̲tep s1
  s3 ← ᵘs̲tep s2
  ((ᵘf̲rames 2 t̲ake₂ s) c̲at₂ (ᵘf̲rames 2 t̲ake₂ s1) c̲at₂ ᵘf̲rames 2 t̲ake₂ s2, ᵘf̲rames 2 t̲ake₂ s3)
}
Used in: X, Xh, x1, xs

The network, read from data/ (just world-model-data)

wb1 : Float

value · line 43

Each layer's weights with its bias as one more row, as NN's d_ense takes them.

wb1 ← 433 64 r̲eshape (n̲umbers ⎕N̲GET "data/w1.txt") c̲at n̲umbers ⎕N̲GET "data/b1.txt"
Used in: ᵘp̲redict

wb2 : Float

value · line 44
wb2 ← 65 144 r̲eshape (n̲umbers ⎕N̲GET "data/w2.txt") c̲at n̲umbers ⎕N̲GET "data/b2.txt"
Used in: ᵘp̲redict

ᵘp̲redict : Float -> Float

function · line 46

Each window's next frame, a probability per pixel.

ᵘp̲redict ← { X → ᵃᶜᵗs̲igmoid (ᵃᶜᵗr̲elu X ⁿⁿd̲ense wb1) ⁿⁿd̲ense wb2 }

ᵘe̲xact : Float -> Float -> Int

function · line 48

How many rows of P are the frames T exactly: every pixel on the right side of one half.

ᵘe̲xact ← { P T → f̲loor '+ r̲/ 'm̲in r̲/₂ f̲loat (P > 0.5) = T > 0.5 }

ᵘp̲lace : Num a => a -> Int

function · line 51

The likeliest place for the ball in a predicted frame p: the 2 x 2 block with the most probability, as x y.

ᵘp̲lace ← { p →
  M ← 12 12 r̲eshape p
  S ← (11 t̲ake 11 t̲ake₂ M) + (11 t̲ake 11 t̲ake₂ 1 d̲rop M) + (11 t̲ake 11 t̲ake₂ 1 d̲rop₂ M) + 11 t̲ake 11 t̲ake₂ 1 d̲rop 1 d̲rop₂ M
  k ← (r̲avel S) i̲ndexOf 'm̲ax r̲/ r̲avel S
  1 2 r̲eshape ((k − 1) m̲od 11) c̲at (k − 1) d̲iv 11
}
Used in: dream

Every state at once

i : Int

value · line 60
i ← o̲ffsets 1936
Used in: all

all : Int

value · line 61
all ← o̲\ 4 1936 r̲eshape (i d̲iv 176) c̲at ((i d̲iv 16) m̲od 11) c̲at ((1 + (i d̲iv 4) m̲od 4) s̲elect -2 -1 1 2) c̲at (1 + i m̲od 4) s̲elect -2 -1 1 2
Used in: X

held : Int

value · line 62
held ← 387 4 r̲eshape f̲loor n̲umbers ⎕N̲GET "data/held-out.txt"
Used in: Xh, s

X : Float

value · line 63
(X, T) ← ᵘw̲indow all

T : Float

value · line 63
(X, T) ← ᵘw̲indow all

Xh : Float

value · line 64
(Xh, Th) ← ᵘw̲indow held

Th : Float

value · line 64
(Xh, Th) ← ᵘw̲indow held

x1 : Float

value · line 66
(x1, _) ← ᵘw̲indow 1 4 r̲eshape 3 7 2 -1

One it never saw

s : Int

value · line 74
s ← 1 4 r̲eshape 1 s̲elect held
Used in: xs, truth

xs : Float

value · line 75
(xs, ts) ← ᵘw̲indow s
Used in: xs, p, dream

ts : Float

value · line 75
(xs, ts) ← ᵘw̲indow s

p : Float

value · line 76
p ← ᵘp̲redict xs

Dreaming

dream : Int

value · line 84
(_, dream) ← 12 '{ (w, xys) →
  xy ← ᵘp̲lace ᵘp̲redict w
  ((144 d̲rop₂ w) c̲at₂ ᵘf̲rames xy, xys c̲at xy)
} p̲ower (xs, 0 2 r̲eshape 0)

truth : Int

value · line 88
(_, truth) ← 12 '{ (t, xys) → (ᵘs̲tep t, xys c̲at 2 t̲ake₂ t) } p̲ower (ᵘs̲tep ᵘs̲tep ᵘs̲tep s, 0 2 r̲eshape 0)