programdemos/embedding-explorer/embedding-explorer.xtl
Word embeddings from counts, and a 3-D look at them: 4,000 short sentences over 70 words, each word's embedding a row of 64 numbers made from which words occur near it (the Embed library: counts, PPMI, a fixed projection). Words used alike point alike: the animals together, the foods, the colors, the numbers, the places. Learn's PCA turns the 64 dimensions into the 3 that spread the words most, for a cloud to look at.
The corpus, read from data/ (just embedding-data)
words : Box Char
The words, one a line, and the sentences as word numbers, each ended by 0.
words ← ᵗl̲ines ⎕N̲GET "data/words.txt"
The embeddings
C : Int
How often each word occurs within 2 words of each other, then PPMI.
C ← (v, 2) ᵉᵐc̲ooccur ids
Used in: P
What it shows
ᵘn̲umber : Char -> Int
A word's number, and its 5 nearest words by cosine in 64 dimensions.
ᵘn̲umber ← { w → f̲irst w̲here '{ x → w m̲atch d̲isclose x } e̲ach words }
Used in: ᵘn̲ear