How a trained model is adapted, in X_eTaL, a typed array language: fine-tuning, low-rank adapters and direct edits of a model's weights, as lessons you can step through.
xetal doc):
every program and library here, its documentation and sections, its source drawn as X_eTaL
draws it, every name linked to where it is defined and used.New here? Five minutes on how these lessons work, with nothing to know beforehand: words that open a plain explanation, a question to answer first, a calculation you can step through and watch, pictures of the numbers, a sum written out, a change put into words, and how much to show. All on one small calculation, the same one the first lesson explains.
Live
LoRA · lesson 1
How do you teach a model something new without it forgetting what it knows, and without retraining all of it? The idea, LoRA, on one layer small enough to read: a frozen 4 x 4 table of weights and a small correction beside it. Change the input or its strength and see what the adapter adds while the weights never move; what it would cost on a real layer; and what that does and does not settle.
y ← (W, A, B, s) ˡᵒf̲orward x
An adapter is one tuple, (W, A, B, s): the frozen matrix, the two small trained ones and the scale, taken apart by the library.
How is an adapter's correction worked out, number by number, and what does its rank change? One layer of 64 frozen numbers and an adapter beside it: take the calculation a stage at a time and watch each stage's numbers appear, click any number to see the sum behind it, pick a letter of the formulas to see what it stands for, and move the rank to see what changes, what it costs, and what does not move at all.
"h[k] | r" ᵉe̲q< "sum j:=0..7 | A[k,j] * x[j]"
A stage of the layer is its equation, written as text: the same text is run to give the numbers and drawn to give the formula.
Live
LoRA · lesson 3
Where do the numbers of an adapter come from? One step of training told stage by stage on an 8 x 8 layer, every number quoted from the run: the prediction, the error, the gradients, the update, and the frozen table untouched. Then training by Adam with a record you can step back through, the loss on the examples trained on beside the loss on examples held out, and three experiments set up as data: a first run, too large a learning rate, too few examples.
▷ "dB[i,k] | 8 r" ᵉe̲q< "sum m:=0..n-1 | s * g[i,m] * h[k,m]" ▹dB
A gradient is one more stage of the pipeline, an equation like the forward ones: the error times what the stage read, summed over the examples.
Live
LoRA · lesson 4
What does the rank buy, and what does it cost? The adapter's rank swept from 1 to 8 on the learning task of lesson 3, under LoRA's strength alpha / r and rsLoRA's alpha / sqrt r: for each, the numbers trained (analytical), the loss on the examples trained on and on the held-out ones after the same steps (measured), and the size of the adapter's table beside the hidden shift's. Move alpha and the steps; the page reruns the sweep and says what moved, never that a rank is better.
a2 ← (W, 1 ˡᵒs̲eeded k c̲at 8, (8 c̲at k) r̲eshape 0.0, alpha ˡᵒs̲calingRs k)
The two rules differ in one number of the adapter's bundle, the strength; everything else, the start, the steps and the examples, is the same.