sourcedemos/world-model/world-model.xtl

1⍝!/usr/bin/env xetal 2⍝│ A tiny world model: a network that has watched a ball bounce in a box 3⍝│ predicts the next frame from the last three. The world is a 2 x 2 4⍝│ ball in a 12 x 12 box, moving 1 or 2 cells a step on each axis and 5⍝│ bouncing off the walls: 1,936 states in all. The network (432 pixels 6⍝│ in, 64 ReLU units, 144 pixels out) was trained offline on four fifths 7⍝│ of them (train/); the other fifth it never saw. Here it predicts the 8⍝│ next frame of every state at once, as one batch, then dreams: fed 9⍝│ its own predictions, it plays the ball forward on its own. 10 11"nn:" u̲se< "NN" 12"act:" u̲se< "Activations" 13 14⍝━━ The world 15⍝│ One step of the states s (rows of x y vx vy): move, and bounce off a 16⍝│ wall at 0 or 10 (the ball is 2 wide in a box of 12). 17ᵘs̲tep ← { s → 18 P ← 2 t̲ake o̲\ s 19 V ← -2 t̲ake o̲\ s 20 n ← P + V 21 lo ← n < 0 22 hi ← n > 10 23 o̲\ (n + ((n̲eg 2 × n) × lo) + (20 − 2 × n) × hi) c̲at V × 1 − 2 × lo + hi 24} 25⍝│ 1.0 where a ball's corner is at the place or one before it (D is -1 or 0). 26ᵘi̲n ← { D → f̲loat (D ≤ 0) × D ≥ -1 } 27⍝│ The frames of the balls at xy (rows of x y), one row of 144 pixels each. 28ᵘf̲rames ← { xy → 29 c ← o̲ffsets 144 30 (ᵘi̲n (2 s̲elect o̲\ xy) '− t̲able c d̲iv 12) × ᵘi̲n (1 s̲elect o̲\ xy) '− t̲able c m̲od 12 31} 32⍝│ The window of the states s: their frame and the next two, side by 33⍝│ side (432 pixels a row), then the frame after them. 34ᵘw̲indow ← { s → 35 s1 ← ᵘs̲tep s 36 s2 ← ᵘs̲tep s1 37 s3 ← ᵘs̲tep s2 38 ((ᵘf̲rames 2 t̲ake₂ s) c̲at₂ (ᵘf̲rames 2 t̲ake₂ s1) c̲at₂ ᵘf̲rames 2 t̲ake₂ s2, ᵘf̲rames 2 t̲ake₂ s3) 39} 40 41⍝━━ The network, read from data/ (just world-model-data) 42⍝│ Each layer's weights with its bias as one more row, as NN's d_ense takes them. 43wb1 ← 433 64 r̲eshape (n̲umbers ⎕N̲GET "data/w1.txt") c̲at n̲umbers ⎕N̲GET "data/b1.txt" 44wb2 ← 65 144 r̲eshape (n̲umbers ⎕N̲GET "data/w2.txt") c̲at n̲umbers ⎕N̲GET "data/b2.txt" 45⍝│ Each window's next frame, a probability per pixel. 46ᵘp̲redict ← { X → ᵃᶜᵗs̲igmoid (ᵃᶜᵗr̲elu X ⁿⁿd̲ense wb1) ⁿⁿd̲ense wb2 } 47⍝│ How many rows of P are the frames T exactly: every pixel on the right side of one half. 48ᵘe̲xact ← { P T → f̲loor '+ r̲/ 'm̲in r̲/₂ f̲loat (P > 0.5) = T > 0.5 } 49⍝│ The likeliest place for the ball in a predicted frame p: the 2 x 2 50⍝│ block with the most probability, as x y. 51ᵘp̲lace ← { p → 52 M ← 12 12 r̲eshape p 53 S ← (11 t̲ake 11 t̲ake₂ M) + (11 t̲ake 11 t̲ake₂ 1 d̲rop M) + (11 t̲ake 11 t̲ake₂ 1 d̲rop₂ M) + 11 t̲ake 11 t̲ake₂ 1 d̲rop 1 d̲rop₂ M 54 k ← (r̲avel S) i̲ndexOf 'm̲ax r̲/ r̲avel S 55 1 2 r̲eshape ((k − 1) m̲od 11) c̲at (k − 1) d̲iv 11 56} 57⍝ -- end of the core ------------------------------------------ 58 59⍝━━ Every state at once 60i ← o̲ffsets 1936 61all ← o̲\ 4 1936 r̲eshape (i d̲iv 176) c̲at ((i d̲iv 16) m̲od 11) c̲at ((1 + (i d̲iv 4) m̲od 4) s̲elect -2 -1 1 2) c̲at (1 + i m̲od 4) s̲elect -2 -1 1 2 62held ← 387 4 r̲eshape f̲loor n̲umbers ⎕N̲GET "data/held-out.txt" 63(X, T) ← ᵘw̲indow all 64(Xh, Th) ← ᵘw̲indow held 65⍝ The same pixels as the trainer's Rust model, for one window. 66(x1, _) ← ᵘw̲indow 1 4 r̲eshape 3 7 2 -1 671e-6 > 'm̲ax r̲/ r̲avel a̲bs (ᵘp̲redict x1) − 1 144 r̲eshape n̲umbers ⎕N̲GET "data/expected.txt" 68⍝ Next frames predicted exactly: of all 1,936 states, and of the 387 it never saw. 69(ᵘp̲redict X) ᵘe̲xact T 70(ᵘp̲redict Xh) ᵘe̲xact Th 71 72⍝━━ One it never saw 73⍝ Three frames, the true next one, and the network's (# above one half, + above a tenth). 74s ← 1 4 r̲eshape 1 s̲elect held 75(xs, ts) ← ᵘw̲indow s 76p ← ᵘp̲redict xs 7712 12 r̲eshape (1 + f̲loor r̲avel ts) s̲elect ".#" 7812 12 r̲eshape (1 + (p > 0.5) + p > 0.1) s̲elect ".+#" 79 80⍝━━ Dreaming 81⍝ Fed its own predictions: each step keeps only the likeliest place 82⍝ for the ball and moves the window on. 83⍝ Twelve steps on from the window xs: where the dreamed ball is, and the true one. 84(_, dream) ← 12 '{ (w, xys) → 85 xy ← ᵘp̲lace ᵘp̲redict w 86 ((144 d̲rop₂ w) c̲at₂ ᵘf̲rames xy, xys c̲at xy) 87} p̲ower (xs, 0 2 r̲eshape 0) 88(_, truth) ← 12 '{ (t, xys) → (ᵘs̲tep t, xys c̲at 2 t̲ake₂ t) } p̲ower (ᵘs̲tep ᵘs̲tep ᵘs̲tep s, 0 2 r̲eshape 0) 89o̲\ dream 90o̲\ truth 91⍝ Steps where the dream has the ball where it really is. 92'+ r̲/ 'm̲in r̲/₂ dream = truth