programextensions/gpu/demos/accelerate.xtl
A layer: a dense product with its bias, then ReLU
ᵘl̲ayer : (Num a, Num b) => a -> b -> Float
ᵘl̲ayer ← "float float" ᵍᵖa̲ccel< "{ x wb -> ac:r_elu x ac:d_ense wb }"
ᵍᵖa̲ccel< expands to
({ x wb → ᵍᵖo̲utput 1 ᵍᵖa̲fter ᵍᵖr̲un 0 ᵍᵖa̲fter 2 ᵍᵖb̲ind wb ᵍᵖa̲fter 1 ᵍᵖb̲ind x ᵍᵖa̲fter "{ x wb -> ac:r_elu x ac:d_ense wb }" ᵍᵖl̲ower "extensions/gpu/demos/accelerate.xtl|float float" })Used in: h
x : Float
x ← 4 3 r̲eshape 0.5 -1.0 0.25 1.0 0.0 -0.5 -0.75 0.5 1.0 0.2 0.4 -0.6
Used in: h, extensions/gpu/demos/accelerate.xtl:19
wb : Float
wb ← 4 5 r̲eshape 0.5 -0.25 0.75 0.1 -0.6 -0.3 0.8 0.2 -0.5 0.4 0.6 0.1 -0.7 0.3 0.2 0.05 -0.1 0.2 0.0 0.1
Used in: h, extensions/gpu/demos/accelerate.xtl:19
Attention scores of a batch of queries against the keys, as probabilities
ᵘa̲ttend : (Num a, Num b) => a -> b -> Float
ᵘa̲ttend ← "float float" ᵍᵖa̲ccel< "{ q k -> ac:s_oftmax q ac:s_cores k }"
ᵍᵖa̲ccel< expands to
({ q k → ᵍᵖo̲utput 1 ᵍᵖa̲fter ᵍᵖr̲un 0 ᵍᵖa̲fter 2 ᵍᵖb̲ind k ᵍᵖa̲fter 1 ᵍᵖb̲ind q ᵍᵖa̲fter "{ q k -> ac:s_oftmax q ac:s_cores k }" ᵍᵖl̲ower "extensions/gpu/demos/accelerate.xtl|float float" })Used in: p
q : Float
q ← 2 3 r̲eshape 1.0 0.5 -1.0 0.0 2.0 0.25
Used in: p, extensions/gpu/demos/accelerate.xtl:27
k : Float
k ← 4 3 r̲eshape 0.5 1.0 0.0 -1.0 0.5 2.0 1.5 -0.5 1.0 0.0 0.0 1.0
Used in: p, extensions/gpu/demos/accelerate.xtl:27
An embedding lookup: the table resident, the ids the only upload
ᵘe̲mbed : (Num a, Num b) => a -> b -> Float
ᵘe̲mbed ← "float int" ᵍᵖa̲ccel< "{ t ids -> t ac:e_mbed ids }"
ᵍᵖa̲ccel< expands to
({ t ids → ᵍᵖo̲utput 1 ᵍᵖa̲fter ᵍᵖr̲un 0 ᵍᵖa̲fter 2 ᵍᵖb̲ind ids ᵍᵖa̲fter 1 ᵍᵖb̲ind t ᵍᵖa̲fter "{ t ids -> t ac:e_mbed ids }" ᵍᵖl̲ower "extensions/gpu/demos/accelerate.xtl|float int" })Used in: e
t : Float
t ← 5 4 r̲eshape 0.0 0.1 0.2 0.3 1.0 1.1 1.2 1.3 2.0 2.1 2.2 2.3 3.0 3.1 3.2 3.3 4.0 4.1 4.2 4.3
Used in: e, extensions/gpu/demos/accelerate.xtl:35