sourcedemos/image-pipeline/image-pipeline.xtl

1⍝!/usr/bin/env xetal 2⍝# An image pipeline as array programs: a picture built from masks, 3⍝# blurred, its edges found with the Sobel kernels, thresholded, and 4⍝# shrunk by max-pooling. Every 3 x 3 filter is the same two steps: 5⍝# the stack of the nine shifted copies of the image (its windows), 6⍝# then the kernel times the stack, summed. Run it with "xetal run 7⍝# demos/image-pipeline/image-pipeline.xtl"; it prints the windows' 8⍝# and the pooled map's shapes, then the picture, the edge strength, 9⍝# the edges and the pooled edges, each drawn with characters. The 10⍝# live page lets you edit the kernels and pick a picture, and click 11⍝# a pixel for its window times each kernel: 12⍝# https://softwarewrighter.github.io/X_eTaL-demos/image-pipeline/ 13 14rows ← 20 15cols ← 40 16 17⍝# -- coordinates: each pixel's row and column. row, col: by 18⍝# broadcasting (t_able). 19row ← (o̲ffsets rows) 'l̲eft t̲able o̲ffsets cols 20col ← (o̲ffsets rows) 'r̲ight t̲able o̲ffsets cols 21⍝ -- the picture: masks of coordinates, and noise (plain #: the page's 22⍝ web app finds this exact text to cut the program it runs, right up 23⍝ to it, so a ## here would leave a stray # in what it cuts). Masks 24⍝ are conditions (true inside), made Floats where used. 25disk ← (((row − 9) × row − 9) + ((col − 10) × col − 10) d̲iv 2) < 36 26box ← ((row ≥ 5) ∧ row < 15) ∧ (col ≥ 22) ∧ col < 33 27img ← 0.15 + (0.7 × f̲loat disk) + (0.45 × f̲loat box) + 0.001 × f̲loat r̲oll! (s̲hape row) r̲eshape 60 28 29⍝# -- the kernels. blur: a Gaussian blur. kx, ky: Sobel edge kernels 30⍝# (ky is kx turned a quarter, o_\, the transpose). thresh: the edge 31⍝# strength that counts as an edge. 32blur ← (3 c̲at 3) r̲eshape 0.0625 0.125 0.0625 0.125 0.25 0.125 0.0625 0.125 0.0625 33kx ← (3 c̲at 3) r̲eshape -1.0 0.0 1.0 -2.0 0.0 2.0 -1.0 0.0 1.0 34ky ← o̲\ kx 35thresh ← 0.8 36 37⍝# -- the windows: nine shifted copies. u:w_indows x: rotating by a 38⍝# list of amounts gives one copy per amount, on a new first axis. 39⍝# Columns are shifted first; then the rows, which the new axis moved 40⍝# to axis 2. Item [a;b;r;c] is the pixel at row r + a, column c + b, 41⍝# for a and b in -1 0 1. 42ᵘw̲indows ← { x → -1 0 1 o̲-₂ -1 0 1 o̲-₂ x } 43 44⍝## A filter: kernel times windows, summed 45 46⍝# u:f_ilter k x: the kernel spread over the picture (3 x 3 x rows x 47⍝# cols) times the windows, summed over its two axes. 48ᵘf̲ilter ← { k x → '+ r̲/₁₂ (ᵘw̲indows x) × k 'l̲eft t̲able x } 49 50⍝## The pipeline 51 52⍝# smooth, gx, gy, mag, edges: the blur, the two Sobel gradients, the 53⍝# edge strength (their length), and where it passes the threshold. 54smooth ← blur ᵘf̲ilter img 55gx ← kx ᵘf̲ilter smooth 56gy ← ky ᵘf̲ilter smooth 57mag ← ((gx × gx) + gy × gy) ^ 0.5 58edges ← f̲loat mag > thresh 59 60⍝# -- max-pooling: 2 x 2 blocks by reshape, reduce. pool: reshaped so 61⍝# each 2 x 2 block has two axes of its own, then the largest over them. 62pool ← 'm̲ax r̲/₂ 'm̲ax r̲/₄ ((rows d̲iv 2) c̲at 2 c̲at (cols d̲iv 2) c̲at 2) r̲eshape mag 63⍝ -- end of the core -------------------------------- 64 65⍝ The shapes: the windows, each stage, the pooled map. 66s̲hape ᵘw̲indows img 67s̲hape pool 68⍝# u:d_raw a: the picture, the smoothed edge strength, the edges and 69⍝# the pooled edges, drawn with characters (darkest first). 70ᵘd̲raw ← { a → (1 + f̲loor 0 m̲ax 7 m̲in 8.0 × a) s̲elect " .:-=+*#" } 71ᵘd̲raw img 72ᵘd̲raw mag ÷ 3.0 73(1 + f̲loor edges) s̲elect ".#" 74(1 + 1 × pool > thresh) s̲elect ".#"