eqho
/
LLM Lab
model
lab-tiny
the run
03 / 09
Lesson 03
Tokens become vectors
An integer is meaningless. A point in space has neighbours.
Next: Attention
01
How big is a model?
02
Text becomes tokens
03
Tokens become vectors
The lookup table
Nearby means similar
Add the position
04
Attention
05
The transformer block
06
Predicting the next token
07
Training
08
From autocomplete to assistant
09
Inference in production
01 / 01
The lookup table
Token id in, a row of d_model numbers out. That row is learned.
←
→
space