Learn by making the model choose

How an LLM gets from context to a token.

Trace prediction, probability, sampling, and training updates in one short experiment.

Prediction bench

Context: The model reads

Which token should a deterministic decoder choose?

likely
quiet
blue

Training loop

Backpropagation sends an error signal backward; optimization adjusts parameters. Set a learning rate and predict what happens.

Assessment

Why can a sampled token differ from the highest bar?

Trace the fixed probability example and lesson state

Read the explanation

This saved lesson uses three fixed illustrative probability bars, not a live language-model request. The probabilities are seventy-two percent for likely, eighteen for quiet, and ten for blue, summing to one hundred. Every bar here uses six pixels per percentage point. The deterministic answer check recognizes likely as correct because it is the largest source bar. These example values are assigned by the page, not calculated from its displayed context by a trained model. The Allow sampling checkbox changes the explanation shown after a likely answer. With it off, the page explains that deterministic decoding selects the highest probability. With it on, the page adds that a sampled choice can differ. The source does not actually draw a random token or call a sampling engine: both branches merely set a feedback string and the prediction field. This diagram traces that exact distinction without claiming an experiment measured sampling. The Run one update control also selects fixed explanatory text. Low and high learning-rate choices update a saved rate field, but no weights, gradients or optimizer are computed by this source. The assessment control records the chosen answer; a correct answer sets mastery true. A later wrong answer changes the displayed feedback without clearing an existing mastery field. Export writes the current lesson record, including these source states and a timestamp. This toy lesson teaches concepts, not a real training benchmark.

Super generates helpful tools and automates fact-checking across the internet proactively. If you enjoyed this tool, build your own with Super and share it with a friend.