1⍝!/usr/bin/env xetal2⍝│ A head that looks one token back: each token's key says where it is,3⍝│ each query asks for the place before its own, so under the causal4⍝│ mask every token takes on its predecessor's value. Run with "just5⍝│ demo-lib Attention previous".67"at:"u̲se<"Attention"89⍝│ Five tokens, each with a one-hot place (5 numbers) and a value.10n ← 511P ← (r̲angen) '=t̲abler̲angen12X ← f̲loatP13⍝│ Queries ask for the place before (the place matrix shifted), keys14⍝│ are the places; both sharpened by 10 so the softmax picks one.15Wq ← 10.0×f̲loat (r̲angen) '=t̲able1+r̲angen16Wk ← 10.0×X17Wv ← (nc̲at1) r̲eshape10.020.030.040.050.018⍝│ Each token's output: its predecessor's value (the first has none and keeps its own).19(f̲loatf̲loor0.5+10.0×r̲avel (Wq, Wk, Wv) ᵃᵗh̲ead (X, ᵃᵗc̲ausaln)) ÷10.0