Next-token distribution
Why do neural networks hallucinate?
A neural network does not "know" facts the way a database does. It learns a probability distribution over the next token given the tokens before it. When the evidence in the prompt is strong, that distribution concentrates on the correct continuation. When the evidence is thin, contradictory, or absent, the distribution spreads out â and the model still has to emit some token. That token is often fluent, confident, and wrong. That is a hallucination.
What this lab shows
Each bar is a candidate next token. The height is the model's probability for that token. The green bar is the grounded answer; the red bar is a plausible-sounding confabulation. Move the sliders and watch how the distribution reshapes:
- Evidence strength â how much the prompt actually supports the correct answer. High evidence concentrates probability on the grounded token.
- Confident-prompt pressure â the effect of a prompt that demands an answer ("You are certain. Answer now."). It pushes probability toward the most fluent candidate, whether or not it is correct.
- Candidate diversity â how many plausible alternatives exist. More alternatives spread probability and make confident wrong answers more likely.
- Temperature â sampling sharpness. Low temperature picks the top token almost deterministically; high temperature samples more broadly.
Reading the verdict
The verdict compares the sampled token to the grounded answer. A grounded verdict means the model landed on the supported token. A confabulated verdict means the model produced a fluent but unsupported token â the classic hallucination. The confidence number is the probability the model assigned to whatever it emitted, which is why a hallucination can still look confident.
What this model is and is not
This is a small, transparent softmax model over a fixed candidate set. It is a teaching instrument, not a real language model. The probabilities are computed from the parameters you set, not from a trained network. The point is the shape of the distribution and the relationship between evidence, pressure, and confidence â not any specific number.
How is the probability computed?
Each candidate gets a score: score = evidence à grounded + pressure à fluency â diversity à rank, then a softmax with temperature is applied. The grounded candidate receives the evidence bonus; the confabulation receives the pressure bonus. This is a simplified illustration of the tension between retrieval-grounded evidence and next-token fluency.
Why does a confident model still hallucinate?
Because confidence is a property of the distribution, not of the truth. A model can assign 0.9 probability to a wrong token if the training signal, prompt framing, or sampling path favors fluency over factuality. Hallucination is a calibration failure, not a knowledge failure.
What reduces hallucination in real systems?
Retrieval-augmented generation (grounding the prompt in retrieved evidence), lower temperature for factual tasks, explicit uncertainty instructions, and post-hoc verification against sources. None of these eliminate hallucination; they shift the distribution toward grounded tokens.