Learn by making the model choose
How an LLM gets from context to a token.
Trace prediction, probability, sampling, and training updates in one short experiment.
Prediction bench
Context: The model reads
Which token should a deterministic decoder choose?
likely
quiet
blue
Make a prediction before reading the explanation.
Training loop
Backpropagation sends an error signal backward; optimization adjusts parameters. Set a learning rate and predict what happens.
No update run yet.
Assessment
Why can a sampled token differ from the highest bar?
Mastery is waiting for your answer.