SuperSuper

Neural Network Hallucination Lab

Tiny next-token model

Next-token probability distribution

Sampled token Other candidates

Pick a prompt and press “Sample a token”.

Readout

Top guess
—
Confidence
—
Entropy
—
Awaiting a sample.

Why do neural networks hallucinate?

A neural network does not look up facts. It predicts the next token from patterns in its training data. When the pattern is thin, the model still produces a distribution — it just spreads probability across many candidates. Sampling from that distribution produces fluent, confident-sounding text that has no grounding. That is a hallucination.

What this lab shows

How to read the chart

Each bar is a candidate next token. Bar height is the probability the model assigns after temperature scaling. The teal bar is the token actually sampled. When the teal bar is short, the model is inventing.

Practical takeaways

Is this a real neural network?

No. It is a hand-authored toy distribution that mirrors the shape of a real model's softmax output. The math (softmax with temperature, entropy) is real; the probabilities are illustrative.

Why does “no evidence” still produce an answer?

Because the model always produces a distribution. There is no “I don't know” token unless training taught one. Sampling from a flat distribution yields a random, confident-looking token.

What is entropy measuring here?

Entropy is the average surprise of a sample. Low entropy means the model is concentrated; high entropy means it is spread out and uncertain.