eqho
/
LLM Lab
model
lab-tiny
the run
07 / 09
Lesson 07
Training
Guess, measure the miss, nudge every parameter, repeat a million times.
Next: From autocomplete to assistant
01
How big is a model?
02
Text becomes tokens
03
Tokens become vectors
04
Attention
05
The transformer block
06
Predicting the next token
07
Training
Cross-entropy loss
The gradient step
Watch it learn
08
From autocomplete to assistant
09
Inference in production
01 / 01
Cross-entropy loss
How surprised was the model by the real next token? Average that over a batch.
←
→
space