programdemos/cnn-digits/cnn-digits.xtl

A tiny convolutional network reads a handwritten digit: 28 x 28 -> 8 filters of 3 x 3 -> ReLU -> 2 x 2 max-pooling -> dense -> softmax. The convolution is the picture's nine shifted copies times the filters: one matrix product. Trained offline on MNIST (97.82% of the 10,000 test digits right); just cnn-train writes data/.

source · imports nn: libs/NN/src/NN.xtl

The network, read from data/

fb : Float

value · line 11

fb ← 8 10 r̲eshape n̲umbers ⎕N̲GET "data/filters.txt"

Used in: k9, bc

k9 : Float

value · line 12

k9 ← 9 t̲ake₂ fb                         ⍝ 8 filters, 3 x 3 each as 9

Used in: ᵘc̲onv, demos/cnn-digits/cnn-digits.xtl:47

bc : Float

value · line 13

bc ← r̲avel -1 t̲ake₂ fb                  ⍝ their biases

Used in: ᵘc̲onv

wb : Float

value · line 14

wb ← 1353 10 r̲eshape n̲umbers ⎕N̲GET "data/dense.txt"

Used in: ᵘc̲lassify

samples : Float

value · line 15

samples ← 10 28 28 r̲eshape n̲umbers ⎕N̲GET "data/samples.txt"

Used in: x, ᵘr̲ead, ᵘs̲ure, ᵘg̲ap

The core: every stage one array expression

ᵘc̲onv : Float -> Float

function · line 18

ᵘc̲onv ← { x →
  v ← 1 d̲rop₄ -1 d̲rop₄ 1 d̲rop₃ -1 d̲rop₃ -1 0 1 o̲-₂ -1 0 1 o̲-₂ x
  8 26 26 r̲eshape (k9 '+ '× i̲nner 9 676 r̲eshape v) + bc 'l̲eft t̲able o̲ffsets 676
}

Used in: ᵘc̲lassify, c

ᵘp̲ool : Num a => a -> a

function · line 22

ᵘp̲ool ← { x → 'm̲ax r̲/₃₅ 8 13 2 13 2 r̲eshape x }

Used in: ᵘc̲lassify, m

ᵘc̲lassify : Float -> Float

function · line 23

ᵘc̲lassify ← { x → f̲irst ⁿⁿs̲oftmax (1 1352 r̲eshape r̲avel ᵘp̲ool ⁿⁿr̲elu ᵘc̲onv x) ⁿⁿd̲ense wb }

Used in: p, ᵘr̲ead, ᵘs̲ure, ᵘg̲ap

Drawing: values as shades, maps side by side

ᵘs̲hade : Float -> Char

function · line 27

ᵘs̲hade ← { m → (1 + f̲loor 0 m̲ax 7 m̲in 8.0 × m) s̲elect " .:-=+*#" }

Used in: ᵘm̲aps, demos/cnn-digits/cnn-digits.xtl:45

ᵘm̲aps : Float -> Char

function · line 29

Each of n maps scaled from its own smallest to largest value, then shaded.

ᵘm̲aps ← { m →
  n ← 1 t̲ake s̲hape m
  e ← '× r̲/ 1 d̲rop s̲hape m
  f ← (n c̲at e) r̲eshape m
  lo ← (n c̲at e) r̲eshape (n r̲eshape e) r̲eplicate 'm̲in r̲/₂ f
  hi ← (n c̲at e) r̲eshape (n r̲eshape e) r̲eplicate 'm̲ax r̲/₂ f
  ᵘs̲hade (s̲hape m) r̲eshape (f − lo) ÷ 0.000001 m̲ax hi − lo
}

Used in: demos/cnn-digits/cnn-digits.xtl:50, demos/cnn-digits/cnn-digits.xtl:51, demos/cnn-digits/cnn-digits.xtl:54, demos/cnn-digits/cnn-digits.xtl:55

ᵘw̲ide : a -> a

function · line 37

ᵘw̲ide ← { c → ((-1 t̲ake s̲hape c) r̲eshape 2) r̲eplicate₂ c }

Used in: demos/cnn-digits/cnn-digits.xtl:45, demos/cnn-digits/cnn-digits.xtl:54, demos/cnn-digits/cnn-digits.xtl:55

ᵘs̲ide : Char -> Char

function · line 38

ᵘs̲ide ← { c →
  g ← c c̲at₃ ((2 t̲ake s̲hape c) c̲at 2) r̲eshape " "
  ((1 s̲elect 1 d̲rop s̲hape g) c̲at '× r̲/ 1 0 1 r̲eplicate s̲hape g) r̲eshape 2 1 3 t̲ranspose g
}

Used in: demos/cnn-digits/cnn-digits.xtl:47, demos/cnn-digits/cnn-digits.xtl:50, demos/cnn-digits/cnn-digits.xtl:51, demos/cnn-digits/cnn-digits.xtl:54, demos/cnn-digits/cnn-digits.xtl:55

x : Float

value · line 44

A test digit: a 7.

x ← 8 s̲elect samples

Used in: demos/cnn-digits/cnn-digits.xtl:45, c, p

c : Float

value · line 49

Convolution and ReLU: 8 maps of 26 x 26, what each filter finds.

c ← ⁿⁿr̲elu ᵘc̲onv x

Used in: demos/cnn-digits/cnn-digits.xtl:50, demos/cnn-digits/cnn-digits.xtl:51, m

m : Float

value · line 53

Max-pooling: 8 maps of 13 x 13 (drawn twice as wide).

m ← ᵘp̲ool c

Used in: demos/cnn-digits/cnn-digits.xtl:54, demos/cnn-digits/cnn-digits.xtl:55

p : Float

value · line 57

The ten probabilities, as bars: the network reads a 7.

p ← ᵘc̲lassify x

Used in: demos/cnn-digits/cnn-digits.xtl:59, demos/cnn-digits/cnn-digits.xtl:60

ᵘb̲ars : Float -> Char

function · line 58

ᵘb̲ars ← { p → (1 + (f̲loor 0.5 + 50.0 × p) '> t̲able o̲ffsets 50) s̲elect " #" }

Used in: demos/cnn-digits/cnn-digits.xtl:59

ᵘr̲ead : Int -> Int

function · line 62

The ten test digits (0 to 9): what the network reads, how sure (%).

ᵘr̲ead ← { i → (ⁿⁿa̲rgmax ᵘc̲lassify i s̲elect samples) − 1 }

Used in: demos/cnn-digits/cnn-digits.xtl:64

ᵘs̲ure : Int -> Int

function · line 63

ᵘs̲ure ← { i → f̲loor 0.5 + 100.0 × 'm̲ax r̲/ ᵘc̲lassify i s̲elect samples }

Used in: demos/cnn-digits/cnn-digits.xtl:65

expected : Float

value · line 67

The same probabilities as the trainer's Rust forward pass, for all ten?

expected ← 10 10 r̲eshape n̲umbers ⎕N̲GET "data/expected.txt"

Used in: ᵘg̲ap

ᵘg̲ap : Int -> Float

function · line 68

ᵘg̲ap ← { i → 'm̲ax r̲/ a̲bs (ᵘc̲lassify i s̲elect samples) − i s̲elect expected }

Used in: demos/cnn-digits/cnn-digits.xtl:69