eqho
/
LLM Lab
model
lab-tiny
the run
05 / 09
Lesson 05
The transformer block
Attention reads, the MLP thinks, the residual stream remembers.
Next: Predicting the next token
01
How big is a model?
02
Text becomes tokens
03
Tokens become vectors
04
Attention
05
The transformer block
The residual highway
The MLP
Normalise, repeat
06
Predicting the next token
07
Training
08
From autocomplete to assistant
09
Inference in production
01 / 01
The residual highway
Each block adds to a running vector rather than replacing it. Information is never thrown away.
←
→
space