sourcedemos/net-macro/net-macro.xtl

1⍝!/usr/bin/env xetal 2⍝# Three networks for one task (which of three spiral arms is a point 3⍝# on?), each described once, by one line: the Net macro library turns 4⍝# a spec, sizes and activations, into the loading of the weights and 5⍝# an ordinary function of NN calls before the program is compiled. 6⍝# xetal expand shows what it wrote. 7 8ⁿⁿ⁼u̲se< "NN" 9ⁿᵉᵗ⁼u̲se< "Net" 10 11⍝## The data, read from data/ (just net-train) 12⍝# The test points, one per line: x, y and the arm (1, 2 or 3). 13pts ← n̲umbers ⎕N̲GET "data/points.txt" 14pts ← o̲\ (((t̲ally pts) d̲iv 3) c̲at 3) r̲eshape pts 15xy ← o̲\ 2 t̲ake pts 16arm ← f̲loor 3 s̲elect pts 17 18⍝## The networks: each described once 19⍝# From a spec the macro writes the loading of each dense layer's 20⍝# weights (data/a1.txt, ...: inputs + 1 by outputs, the bias last; the 21⍝# program stops, naming the layer, if a file holds the wrong count) 22⍝# and the forward function. 23"u:l_inear a" ⁿᵉᵗm̲odel< "2 3 softmax"
ⁿᵉᵗm̲odel< expands to
a1 ← { v → 9 = t̲ally v ? 3 3 r̲eshape v◆ ⎕P̲ANIC "data/a1.txt holds " c̲at (f̲ormat t̲ally v) c̲at " numbers; layer 1 of \"2 3 softmax\" (2 inputs and a bias, 3 outputs) needs 9" } n̲umbers ⎕N̲GET "data/a1.txt"
ᵘl̲inear ← { x → ⁿⁿs̲oftmax x ⁿⁿd̲ense a1 }
24"u:s_mall b" ⁿᵉᵗm̲odel< "2 4 tanh 3 softmax"
ⁿᵉᵗm̲odel< expands to
b1 ← { v → 12 = t̲ally v ? 3 4 r̲eshape v◆ ⎕P̲ANIC "data/b1.txt holds " c̲at (f̲ormat t̲ally v) c̲at " numbers; layer 1 of \"2 4 tanh 3 softmax\" (2 inputs and a bias, 4 outputs) needs 12" } n̲umbers ⎕N̲GET "data/b1.txt"
b2 ← { v → 15 = t̲ally v ? 5 3 r̲eshape v◆ ⎕P̲ANIC "data/b2.txt holds " c̲at (f̲ormat t̲ally v) c̲at " numbers; layer 2 of \"2 4 tanh 3 softmax\" (4 inputs and a bias, 3 outputs) needs 15" } n̲umbers ⎕N̲GET "data/b2.txt"
ᵘs̲mall ← { x → ⁿⁿs̲oftmax (ⁿⁿt̲anh x ⁿⁿd̲ense b1) ⁿⁿd̲ense b2 }
25"u:d_eep c" ⁿᵉᵗm̲odel< "2 16 relu 16 relu 3 softmax"
ⁿᵉᵗm̲odel< expands to
c1 ← { v → 48 = t̲ally v ? 3 16 r̲eshape v◆ ⎕P̲ANIC "data/c1.txt holds " c̲at (f̲ormat t̲ally v) c̲at " numbers; layer 1 of \"2 16 relu 16 relu 3 softmax\" (2 inputs and a bias, 16 outputs) needs 48" } n̲umbers ⎕N̲GET "data/c1.txt"
c2 ← { v → 272 = t̲ally v ? 17 16 r̲eshape v◆ ⎕P̲ANIC "data/c2.txt holds " c̲at (f̲ormat t̲ally v) c̲at " numbers; layer 2 of \"2 16 relu 16 relu 3 softmax\" (16 inputs and a bias, 16 outputs) needs 272" } n̲umbers ⎕N̲GET "data/c2.txt"
c3 ← { v → 51 = t̲ally v ? 17 3 r̲eshape v◆ ⎕P̲ANIC "data/c3.txt holds " c̲at (f̲ormat t̲ally v) c̲at " numbers; layer 3 of \"2 16 relu 16 relu 3 softmax\" (16 inputs and a bias, 3 outputs) needs 51" } n̲umbers ⎕N̲GET "data/c3.txt"
ᵘd̲eep ← { x → ⁿⁿs̲oftmax (ⁿⁿr̲elu (ⁿⁿr̲elu x ⁿⁿd̲ense c1) ⁿⁿd̲ense c2) ⁿⁿd̲ense c3 }
26⍝ -- end of the core -------------------------------------- 27 28⍝ How many numbers each has to learn, counted by the macro when the 29⍝ program is compiled. 30("2 3 softmax" ⁿᵉᵗp̲arams< @) c̲at ("2 4 tanh 3 softmax" ⁿᵉᵗp̲arams< @) c̲at "2 16 relu 16 relu 3 softmax" ⁿᵉᵗp̲arams< @
ⁿᵉᵗp̲arams< expands to
(9)
ⁿᵉᵗp̲arams< expands to
(27)
ⁿᵉᵗp̲arams< expands to
(371)
31⍝ The share of the 120 test points each reads right: a line cannot 32⍝ follow a spiral, a small network nearly can, the deep one does. 33(arm ⁿⁿa̲ccuracy ᵘl̲inear xy) c̲at (arm ⁿⁿa̲ccuracy ᵘs̲mall xy) c̲at arm ⁿⁿa̲ccuracy ᵘd̲eep xy 34⍝# What each decides over the square -1.1 .. 1.1 (a, b, c: the arms). 35g ← 20 36c ← -1.1 + 2.2 × (0.5 + f̲loat o̲ffsets g) ÷ f̲loat g 37grid ← o̲\ (2 c̲at g × g) r̲eshape (r̲avel (o̲ffsets g) 'r̲ight t̲able c) c̲at r̲avel (r̲ev c) 'l̲eft t̲able o̲ffsets g 38ᵘa̲rms ← { y → (g c̲at g) r̲eshape (ⁿⁿa̲rgmax y) s̲elect "abc" } 39ᵘa̲rms ᵘl̲inear grid 40ᵘa̲rms ᵘs̲mall grid 41ᵘa̲rms ᵘd̲eep grid