Solutions engineering onboarding
Build a language model.
Watch every step happen.
Nine lessons, each broken into steps you can scrub through. Every animation is driven by real numbers, and the model behind the later lessons is one we trained ourselves, on a laptop, from nothing.
Start with lesson oneAbout 20 minutes per lesson
ours
Lab tiny
1.35M
1 dot = 100 params
The path
In the order the data flows: text in, tokens, vectors, attention, a prediction, then how it was trained and what it costs to run.
runs in your browser now screens built, waiting on the model planned
- 01How big is a model?From one learned number to a trillion of them.
- 02Text becomes tokensThe model never sees letters. It sees integers.
- 03Tokens become vectorsAn integer is meaningless. A point in space has neighbours.
- 04AttentionEvery token asks every earlier token: are you relevant to me?
- 05The transformer blockAttention reads, the MLP thinks, the residual stream remembers.
- 06Predicting the next tokenThe output is not a word. It is a probability for every word.
- 07TrainingGuess, measure the miss, nudge every parameter, repeat a million times.
- 08From autocomplete to assistantA base model finishes your sentence. A chat model answers you.
- 09Inference in productionWhy the first word is slow and the rest are fast.