sourcelibs/Sample/src/Sample.xtl

1⍝│ Sample: choosing the next token from scores: greedy, temperature, top-k, drawing by given uniform numbers 2⍝│ 3⍝│ Import it with an alias of your choice: "sm:" u_se< "Sample". 4⍝│ Put libs/Sample/src on XETAL_PATH ("just path"); the reference is libs/Sample/docs. 5⍝│ Names with l: are exported; h: names are private to this file. 6⍝│ 7⍝│ Scores (logits) are a row per position, a column per token; tokens 8⍝│ count from 1. The library draws no chance itself: a draw takes 9⍝│ uniform numbers in 0 .. 1 from the caller (r_oll!, seeded, or any 10⍝│ other source), so the same numbers give the same tokens. 11 12"nn:" u̲se< "NN" 13 14⍝│ A value per row spread across that row's columns (of an n x m matrix). 15ʰs̲pread ← { v m → v 'l̲eft t̲able o̲ffsets 2 s̲elect s̲hape m } 16 17⍝━━ Choosing 18 19⍝│ g̲reedy s: the most likely token of each row. 20⍝│ >> "sm:" u_se< "Sample" 21⍝│ >> sm:g_reedy 2 3 r_eshape 0.1 2.0 0.5 3.0 0.0 1.0 22⍝│ 2 1 23ˡg̲reedy ← { s → ⁿⁿa̲rgmax s } 24 25⍝│ t p̲robs s: the probabilities at temperature t: softmax of the 26⍝│ scores over t (below 1 sharper, above 1 flatter). 27⍝│ >> "sm:" u_se< "Sample" 28⍝│ >> 1.0 sm:p_robs 0.0 0.6931471805599453 29⍝│ 0.3333333333333333 0.6666666666666666 30ˡp̲robs ← { t s → ⁿⁿs̲oftmax s ÷ t } 31 32⍝│ k t̲op p: only each row's k largest probabilities kept (the rest 33⍝│ 0), then scaled to sum to 1 again. On a tie at the k-th, all tied 34⍝│ are kept. 35⍝│ >> "sm:" u_se< "Sample" 36⍝│ >> 2 sm:t_op 0.1 0.5 0.15 0.25 37⍝│ 0.0 0.6666666666666666 0.0 0.3333333333333333 38ˡt̲op ← { k p → 39 m ← ((t̲ally r̲avel p) d̲iv -1 t̲ake s̲hape p) c̲at -1 t̲ake s̲hape p 40 rows ← m r̲eshape p 41 ⍝│ Each row's k-th largest: its largest taken and shifted below the 42 ⍝│ rest, k times (X_eTaL has no grade along a row yet). The shift is 43 ⍝│ finite, 1 + the range of the values, so it runs in 32-bit floats 44 ⍝│ too (a huge constant is infinity there, and 0 times it NaN). 45 (_, kth) ← k '{ (q, t) → 46 top ← 'm̲ax r̲/₂ q 47 (q − (1.0 + ('m̲ax r̲/ r̲avel q) − 'm̲in r̲/ r̲avel q) × f̲loat q ≥ top ʰs̲pread q, top) 48 } p̲ower (rows, (1 t̲ake m) r̲eshape 0.0) 49 kept ← rows × f̲loat rows ≥ kth ʰs̲pread rows 50 (s̲hape p) r̲eshape kept ÷ ('+ r̲/₂ kept) ʰs̲pread kept 51} 52 53⍝│ u d̲raw p: a token for each row of probabilities p, drawn by the 54⍝│ uniform number u (one per row): the first token where the running 55⍝│ sum of the row passes u. 56⍝│ >> "sm:" u_se< "Sample" 57⍝│ >> 0.1 0.7 sm:d_raw 2 3 r_eshape 0.2 0.3 0.5 0.2 0.3 0.5 58⍝│ 1 3 59ˡd̲raw ← { u p → 60 m ← ((t̲ally r̲avel p) d̲iv -1 t̲ake s̲hape p) c̲at -1 t̲ake s̲hape p 61 c ← '+ s̲\₂ m r̲eshape p 62 1 + '+ r̲/₂ c < u 'l̲eft t̲able o̲ffsets 2 s̲elect m 63}