The Anatomy of Neural Confabulation

Neural networks do not "lie" out of intent. Hallucinations are the mathematical consequence of sampling tokens in low training density regions, softmax temperature dispersion, and error compounding in autoregressive decoding. Drag the query anchor in the latent space or adjust decoding parameters below to witness confabulation emerge in real time.

PRESET SCENARIO

2D Latent Representation Space ($\mathbb{R}^d \to \mathbb{R}^2$)

[x: 0.42, y: 0.65]

✦ Click & drag the Query Anchor (blue crosshair) toward low-density space to trigger OOD hallucinations

Ground Truth Basin (Dense Training Data)

Hallucination Basin (Spurious Correlation)

Contextual Prompt Prior

Active Generation Query $h_t$

Autoregressive Token Stream Step: 0 / 12

"The astronauts who landed during Apollo 18 were "

Next-Token Candidate Probabilities $P(w_{t} \mid w_{<t})$ Entropy: 1.28 nats

Softmax Temperature ($T$) 0.70

Context Attention Weight ($\alpha$) 0.50

Factuality Penalty ($\lambda_{\text{fact}}$) 0.20

Nucleus Filtering (Top-$p$) 0.90

Why Do Hallucinations Inevitably Occur?

1. Manifold Boundary Extrapolation

Language models learn a low-dimensional manifold embedded in high-dimensional activation space. Outside dense training clusters, the model does not output a null or "I don't know"; the continuous geometry forces it to project onto the nearest high-probability linguistic trajectory.

$P(w) = \frac{\exp(z_w / T)}{\sum_k \exp(z_k / T)} > 0, \quad \forall w \in \mathcal{V}$

2. Compounding Autoregressive Error

Generation is sequential: $P(y_1, \dots, y_N) = \prod_{t=1}^N P(y_t \mid y_{<t})$. A single hallucinated token $y_k$ becomes unconditional ground truth conditioning for all future steps $y_{k+1}, y_{k+2}$, causing catastrophic drift from reality into internally consistent fiction.

\mathcal{D}_{\text{drift}}(t) \propto \sum_{\tau=1}^t \mathrm{KL}(P_{\text{truth}} \parallel P_{\text{sample}}^{(\tau)})

3. Parametric vs Contextual Conflict

Models store facts implicitly in feed-forward weight matrices ($W_{\text{FFN}}$) while attending over current context tokens. When context is ambiguous, long, or conflicting, attention weights disperse, and the frozen pre-training prior overrides truth.

\mathbf{h}_{t} = (1 - \alpha)\mathbf{h}_{\text{weights}} + \alpha \sum_{j} A_{tj}\mathbf{x}_j