Super Super

Neural Network Hallucination Lab

tiny net · live probe

Decision surface

predicts A predicts B training point your probe
0.50
0.50
Prediction at probe
—
confidence —
Ready. Press Train to fit the tiny network.

Network & training

8
600
0.20
0%
A tiny 2→hidden→2 network. It has no concept of "unknown" — every input gets a confident softmax answer.
Why hallucinate? The network only ever sees the training points. Everywhere else it interpolates — smoothly, confidently, and often wrongly. That confident wrong answer is the same mechanism behind a language model inventing a citation.

Why do neural networks hallucinate?

A neural network is a function approximator. It learns a mapping from inputs to outputs by adjusting weights to fit the examples it was shown. It does not learn a rule that says "I have never seen this before." It learns a smooth surface that passes near the training data, and then it extrapolates that surface into every corner of the input space — including corners where the training data is empty.

When the surface is confident in a region with no evidence, the output is a hallucination. The model is not lying and it is not malfunctioning. It is doing exactly what it was trained to do: produce the most likely output given its weights. The problem is that "most likely given its weights" is not the same as "true."

Four mechanisms you can see in this lab

How this maps to language models

A large language model is a much bigger function approximator trained on text. When it produces a citation, a date, or a name that does not exist, it is doing the same thing this tiny network does in the empty center of the ring: filling a region of input space with a confident, smooth, plausible-looking output that was never grounded in evidence.

The fix is not to make the network less confident in general — that would make it useless. The fix is to ground the output in retrieved evidence (retrieval-augmented generation), to check claims against sources, and to treat confident output in low-evidence regions as a hypothesis rather than a fact.

Try this

  1. Train on Sparse. Drag the probe into the gap between the two clusters. Note the confidence.
  2. Switch to Ring and train. Probe the center. The network is confident; the data is silent.
  3. Set hidden units to 2 and train. The surface becomes almost linear — a different kind of wrong.
  4. Set label noise to 30% and train. Watch the boundary warp to accommodate bad labels.
What is actually being computed?

A two-layer network: input (x, y) → hidden layer with tanh activation → output layer with softmax. Weights are trained with full-batch gradient descent on cross-entropy loss. The background color is the network's predicted class at each pixel; the bar chart shows the softmax probabilities at the probe.

Why does the surface look so smooth?

Because it is. A small network with tanh activations produces a smooth, continuous decision surface. Smoothness is a prior, not evidence. It is what lets the network generalize — and it is also what lets it hallucinate.

Does this prove anything about large models?

No. It illustrates a mechanism. A tiny 2D classifier and a multi-billion-parameter language model are not the same thing, and this lab does not measure hallucination rates in any real model. What it shows is the structural reason confident wrong answers are possible: the model has no built-in representation of "I don't know."