Neural Network Hallucination Playground

Why do neural networks hallucinate?

A neural network does not "know" facts. It learns a probability distribution over what token comes next, given the tokens it has seen. At inference time it samples from that distribution. When the prompt is close to the training distribution, the distribution is sharp and the sampled token is usually right. When the prompt is far from anything the model saw, the distribution is still sharp — the model has no mechanism to say "I don't know" — so it commits confidently to a token that has no grounding. That confident, fluent, wrong output is a hallucination.

This playground makes the mechanism visible. It uses a small, transparent, hand-authored n-gram model so you can see exactly what the network is doing. The distribution you see is the model's actual next-token probability. The bars show how much probability mass is concentrated on the top candidate; the "off-distribution" flag fires when the prompt contains tokens the model has never seen.

What you are looking at

• Next-token distribution — the model's probability over candidate continuations. Sharp bars mean high confidence.
• Temperature — divides the logits before softmax. Higher temperature flattens the distribution, so the model samples more diverse (and more often wrong) tokens.
• Top-k — restricts sampling to the k most likely tokens. Top-k = 1 is greedy decoding: always pick the single most likely token.
• Mean confidence / perplexity — how sure the model was across the sampled tokens. Low confidence and high perplexity are the signature of a hallucination-prone region.

How to reproduce a hallucination

1. Start with an in-distribution prompt like "The capital of France is". The distribution is sharp and the continuation is correct.
2. Change the prompt to something the model has never seen, e.g. "The capital of Zorblax is". The model still produces a confident token — but it is invented.
3. Raise the temperature to 2.0 and sample again. The distribution flattens and the model starts producing plausible-looking but wrong tokens even on in-distribution prompts.
4. Export the JSON evidence to keep the exact distribution, samples, and confidence metrics.

Limitations

This is a pedagogical model, not a large language model. It is a small n-gram table with hand-authored probabilities, chosen so the mechanism is legible. Real LLMs have billions of parameters and their distributions are far more complex, but the core mechanism — sampling from a learned distribution with no calibrated "unknown" signal — is the same. The "off-distribution" flag is a heuristic based on whether the prompt contains tokens absent from the model's vocabulary; it is not a rigorous OOD detector.