programdemos/diffusion/diffusion.xtl

A learned denoiser: pictures from noise, step by step. The pictures are 10 x 10, one of three shapes (a disk, a hollow square, a cross) at some place, a pixel -1 off or 1 on. A small convolutional network (train/) was trained to see the clean picture in a noisy one at ten levels of noise. Sampling runs it backward: from a picture of pure noise, each step predicts the clean picture, infers the noise that must have been added, and steps down one level (DDIM, no new noise), so the picture that comes out depends only on the noise that went in.

source · imports nn: libs/NN/src/NN.xtl; act: libs/Activations/src/Activations.xtl; cv: libs/Conv/src/Conv.xtl

The schedule

ab : Float

value · line 18

The kept share of the picture at each level 0 to 10 (1.0 clean, 0.001 all noise); the noisy picture is sqrt ab x + sqrt (1 - ab) e.

ab ← 0.001 m̲ax (c̲os ((p̲i @) ÷ 2.0) × (f̲loat o̲ffsets 11) ÷ 10.0) ^ 2.0
Used in: ra, rb

ra : Float

value · line 20

The roots the steps mix with, taken once: of the kept share and of the noise's.

ra ← ab ^ 0.5
Used in: ᵘs̲tep

rb : Float

value · line 21
rb ← (1.0 − ab) ^ 0.5
Used in: ᵘs̲tep

The denoiser, read from data/ (just diffusion-data)

kb : Float

value · line 25

The 8 filters, 3 x 3 each as a row of 9 with its bias last (as Conv takes them).

kb ← 8 10 r̲eshape n̲umbers ⎕N̲GET "data/filters.txt"
Used in: ᵘn̲et

wb1 : Float

value · line 29

The dense layers, each with its bias as one more row, as NN's d_ense takes them: the first over the 512 features, and the level's own weights wh (a row per level) added to it.

wb1 ← 513 256 r̲eshape (n̲umbers ⎕N̲GET "data/w1.txt") c̲at n̲umbers ⎕N̲GET "data/b1.txt"
Used in: ᵘn̲et

wh : Float

value · line 30
wh ← 10 256 r̲eshape n̲umbers ⎕N̲GET "data/wh.txt"
Used in: ᵘn̲et

wb2 : Float

value · line 31
wb2 ← 257 100 r̲eshape (n̲umbers ⎕N̲GET "data/w2.txt") c̲at n̲umbers ⎕N̲GET "data/b2.txt"
Used in: ᵘn̲et

ᵘn̲et : Float -> Float

function · line 38

The network over the joined input J (a row per picture: its 100 pixels, then the level as a one-hot): the picture's 3 x 3 windows through the 8 filters (valid, so 8 maps of 8 x 8) with ReLU, the 512 features through a dense layer with the level's weights added and ReLU, and the output layer; the clean pictures it sees in -1 .. 1. One function of arrays, the part a GPU takes (docs/gpu-asks.md).

ᵘn̲et ← { J →
  X ← ((t̲ally J) c̲at 10 10) r̲eshape 100 t̲ake₂ J
  V ← ((t̲ally J) c̲at 512) r̲eshape ᵃᶜᵗr̲elu (9 t̲ake₂ kb, r̲avel -1 t̲ake₂ kb) ᶜᵛc̲onv X
  A ← ᵃᶜᵗr̲elu (V ⁿⁿd̲ense wb1) + (-10 t̲ake₂ J) '+ '× i̲nner wh
  (2.0 × ᵃᶜᵗs̲igmoid A ⁿⁿd̲ense wb2) − 1.0
}
Used in: ᵘc̲lean

ᵘc̲lean : Int -> Float -> Float

function · line 46

The clean pictures the denoiser sees in the noisy pictures X (a row each) at level t: the level joins the pixels as a one-hot.

ᵘc̲lean ← { t X → ᵘn̲et X c̲at₂ 10 ⁿⁿo̲neHot (t̲ally X) r̲eshape t }
Used in: ᵘs̲tep

ᵘs̲tep : (Float, Int) -> (Float, Int)

function · line 49

One step down from level t: the clean pictures C it sees, the noise that would make X of them, and the pictures one level cleaner.

ᵘs̲tep ← { (X, t) →
  C ← t ᵘc̲lean X
  (((t s̲elect ra) × C) + (t s̲elect rb) × (X − ((1 + t) s̲elect ra) × C) ÷ (1 + t) s̲elect rb, t − 1)
}
Used in: ᵘs̲ample, L

ᵘs̲ample : Float -> Float

function · line 55

Ten steps from the pictures of noise E: at level 0 what is left is the last clean picture seen.

ᵘs̲ample ← { E → (X, _) ← 10 'ᵘs̲tep p̲ower (E, 10)◆ X }

Pictures from noise

ᵘg̲auss : Int -> Float

function · line 60

n Gaussian numbers by Box-Muller from r_oll!

ᵘg̲auss ← { n →
  u1 ← (1.0 + f̲loat r̲oll! n r̲eshape 999999) ÷ 1000000.0
  u2 ← (f̲loat r̲oll! n r̲eshape 1000000) ÷ 1000000.0
  ((n̲eg 2.0 × l̲og u1) ^ 0.5) × c̲os (2.0 × p̲i @) × u2
}
Used in: F

e : Float

value · line 66
e ← 1 100 r̲eshape n̲umbers ⎕N̲GET "data/noise.txt"

F : Float

value · line 69
F ← ᵘs̲ample 128 100 r̲eshape ᵘg̲auss 12800

on : Float

value · line 73
on ← '+ r̲/₂ f̲loat F > 0.0

One picture, level by level

L : Float

value · line 78

The noisy picture at each level from 10 down to 0, # above zero.

(_, _, L) ← 10 '{ (X, t, L) → (X1, t1) ← ᵘs̲tep (X, t)◆ (X1, t1, L c̲at X1) } p̲ower (e, 10, e)