sourcedemos/diffusion/diffusion.xtl

1⍝!/usr/bin/env xetal 2⍝│ A learned denoiser: pictures from noise, step by step. The pictures 3⍝│ are 10 x 10, one of three shapes (a disk, a hollow square, a cross) 4⍝│ at some place, a pixel -1 off or 1 on. A small convolutional network 5⍝│ (train/) was trained to see the clean picture in a noisy one at ten 6⍝│ levels of noise. Sampling runs it backward: from a picture of pure 7⍝│ noise, each step predicts the clean picture, infers the noise that 8⍝│ must have been added, and steps down one level (DDIM, no new noise), 9⍝│ so the picture that comes out depends only on the noise that went in. 10 11"nn:" u̲se< "NN" 12"act:" u̲se< "Activations" 13"cv:" u̲se< "Conv" 14 15⍝━━ The schedule 16⍝│ The kept share of the picture at each level 0 to 10 (1.0 clean, 17⍝│ 0.001 all noise); the noisy picture is sqrt ab x + sqrt (1 - ab) e. 18ab ← 0.001 m̲ax (c̲os ((p̲i @) ÷ 2.0) × (f̲loat o̲ffsets 11) ÷ 10.0) ^ 2.0 19⍝│ The roots the steps mix with, taken once: of the kept share and of the noise's. 20ra ← ab ^ 0.5 21rb ← (1.0 − ab) ^ 0.5 22 23⍝━━ The denoiser, read from data/ (just diffusion-data) 24⍝│ The 8 filters, 3 x 3 each as a row of 9 with its bias last (as Conv takes them). 25kb ← 8 10 r̲eshape n̲umbers ⎕N̲GET "data/filters.txt" 26⍝│ The dense layers, each with its bias as one more row, as NN's d_ense 27⍝│ takes them: the first over the 512 features, and the level's own 28⍝│ weights wh (a row per level) added to it. 29wb1 ← 513 256 r̲eshape (n̲umbers ⎕N̲GET "data/w1.txt") c̲at n̲umbers ⎕N̲GET "data/b1.txt" 30wh ← 10 256 r̲eshape n̲umbers ⎕N̲GET "data/wh.txt" 31wb2 ← 257 100 r̲eshape (n̲umbers ⎕N̲GET "data/w2.txt") c̲at n̲umbers ⎕N̲GET "data/b2.txt" 32⍝│ The network over the joined input J (a row per picture: its 100 33⍝│ pixels, then the level as a one-hot): the picture's 3 x 3 windows 34⍝│ through the 8 filters (valid, so 8 maps of 8 x 8) with ReLU, the 512 35⍝│ features through a dense layer with the level's weights added and 36⍝│ ReLU, and the output layer; the clean pictures it sees in -1 .. 1. 37⍝│ One function of arrays, the part a GPU takes (docs/gpu-asks.md). 38ᵘn̲et ← { J → 39 X ← ((t̲ally J) c̲at 10 10) r̲eshape 100 t̲ake₂ J 40 V ← ((t̲ally J) c̲at 512) r̲eshape ᵃᶜᵗr̲elu (9 t̲ake₂ kb, r̲avel -1 t̲ake₂ kb) ᶜᵛc̲onv X 41 A ← ᵃᶜᵗr̲elu (V ⁿⁿd̲ense wb1) + (-10 t̲ake₂ J) '+ '× i̲nner wh 42 (2.0 × ᵃᶜᵗs̲igmoid A ⁿⁿd̲ense wb2) − 1.0 43} 44⍝│ The clean pictures the denoiser sees in the noisy pictures X (a row 45⍝│ each) at level t: the level joins the pixels as a one-hot. 46ᵘc̲lean ← { t X → ᵘn̲et X c̲at₂ 10 ⁿⁿo̲neHot (t̲ally X) r̲eshape t } 47⍝│ One step down from level t: the clean pictures C it sees, the noise 48⍝│ that would make X of them, and the pictures one level cleaner. 49ᵘs̲tep ← { (X, t) → 50 C ← t ᵘc̲lean X 51 (((t s̲elect ra) × C) + (t s̲elect rb) × (X − ((1 + t) s̲elect ra) × C) ÷ (1 + t) s̲elect rb, t − 1) 52} 53⍝│ Ten steps from the pictures of noise E: at level 0 what is left is 54⍝│ the last clean picture seen. 55ᵘs̲ample ← { E → (X, _) ← 10 'ᵘs̲tep p̲ower (E, 10)◆ X } 56⍝ -- end of the core ------------------------------------------ 57 58⍝━━ Pictures from noise 59⍝│ n Gaussian numbers by Box-Muller from r_oll! 60ᵘg̲auss ← { n → 61 u1 ← (1.0 + f̲loat r̲oll! n r̲eshape 999999) ÷ 1000000.0 62 u2 ← (f̲loat r̲oll! n r̲eshape 1000000) ÷ 1000000.0 63 ((n̲eg 2.0 × l̲og u1) ^ 0.5) × c̲os (2.0 × p̲i @) × u2 64} 65⍝ The same picture as the trainer's Rust sampler, from its starting noise. 66e ← 1 100 r̲eshape n̲umbers ⎕N̲GET "data/noise.txt" 671e-6 > 'm̲ax r̲/ r̲avel a̲bs (ᵘs̲ample e) − 1 100 r̲eshape n̲umbers ⎕N̲GET "data/expected.txt" 68⍝ 128 pictures at once (the slow part: ten steps of two dense layers over all of them). 69F ← ᵘs̲ample 128 100 r̲eshape ᵘg̲auss 12800 70⍝ Four of them, # on. 7140 10 r̲eshape (1 + (4 t̲ake F) > 0.0) s̲elect ".#" 72⍝ How many of the 128 have between 8 and 40 pixels on, as every shape does. 73on ← '+ r̲/₂ f̲loat F > 0.0 74'+ r̲/ f̲loat (on ≥ 8.0) × on ≤ 40.0 75 76⍝━━ One picture, level by level 77⍝│ The noisy picture at each level from 10 down to 0, # above zero. 78(_, _, L) ← 10 '{ (X, t, L) → (X1, t1) ← ᵘs̲tep (X, t)◆ (X1, t1, L c̲at X1) } p̲ower (e, 10, e) 79110 10 r̲eshape (1 + L > 0.0) s̲elect ".#"