programlibs/Quant/demos/formats.xtl

One weight matrix stored five ways, and how far each moves a layer's output. Run with "just demo-lib Quant formats".

source · imports qz: libs/Quant/src/Quant.xtl

W : Float

value · line 8

A 16 x 16 weight matrix and an input, by a fixed ripple (no chance).

W ← 16 16 r̲eshape 0.4 × s̲in 2.3 × f̲loat r̲ange 256

x : Float

value · line 9
x ← c̲os 0.9 × f̲loat r̲ange 16

y : Float

value · line 10
y ← x '+ '× i̲nner W
Used in: ᵘe̲rr

ᵘe̲rr : Float -> Float

function · line 12

Each format's weights, and the largest change in the layer's output.

ᵘe̲rr ← { V → 'm̲ax r̲/ a̲bs y − x '+ '× i̲nner V }

q : Float

value · line 13
q ← 0.7 qz:t̲ernary W

ᵘr̲ound : Float -> Float

function · line 14
ᵘr̲ound ← { a → (f̲loat f̲loor 0.5 + 1000000.0 × a) ÷ 1000000.0 }