eqho
/
LLM Lab
model
lab-tiny
the run
02 / 09
Lesson 02
Text becomes tokens
The model never sees letters. It sees integers.
Next: Tokens become vectors
01
How big is a model?
02
Text becomes tokens
Start with characters
Merge the most common pair
A trained vocabulary
Tokens are the unit of everything
03
Tokens become vectors
04
Attention
05
The transformer block
06
Predicting the next token
07
Training
08
From autocomplete to assistant
09
Inference in production
01 / 01
Start with characters
The simplest vocabulary is every distinct character. It works, but sequences get long.
←
→
space