Mechanistic Interpretability Empirical Manifold Lab Autoregressive Rollout Engine

Why Neural Networks Hallucinate

Neural networks do not "lie"—they perform maximum-likelihood token continuation across an incomplete, high-dimensional probability manifold. Drag the probe through semantic space to observe how geometric extrapolation voids, calibrated entropy loss, and autoregressive drift force fluent factual confabulations.

Scenarios:

2D Manifold Projection & Sparsity Voids

Click & drag the probe ● out of training distribution clusters into sparse extrapolation basins.

Drag Probe to Traverse Latent Voids

Ground Truth Training Data

Semi-Sparse Transition Zone

Extrapolation Void (Hallucination Zone)

Current Active Query Vector

Generation & Decoding Hyperparameters

Control temperature, sampling strategy, and attention decay

Sampling Temperature ($T$) 0.70

Training Manifold Density Gap 0.65

Sampling Strategy

Prompt: "In which year was the Treaty of Veritas signed by Emperor Lucian?"

Autoregressive Generated Tokens:

Manifold Distance ($d_M$)

0.74 σ

Extrapolating in void

Softmax Confidence

94.8%

Overconfident output

Factual Alignment

18.2%

Hallucinated premise

Compounded Drift Risk

0.88

Cascading divergence

Active Step Logit Probabilities $P(w_{t+1} \mid w_{1:t})$

Observe how high fluency and smooth grammar masquerade as ground truth even when epistemic certainty is zero.

Step 4 / 8

Cause 01: Geometry

High-Dimensional Manifold Sparsity

Natural text data lies on a thin, curved manifold within millions of latent dimensions. Between training clusters exist vast empty spaces. When a query lands in a sparsity void, the continuous neural function interpolates smoothly, producing grammatically immaculate yet factually baseless tokens.

f_\theta(x) = \text{softmax}(W \cdot \text{gelu}(Vx + b))

Cause 02: Loss Function

Cross-Entropy Calibration Blindspot

Language models are trained on maximum-likelihood next-token prediction ($\min -\sum \log P(x_t)$), not truth verification. The network is rewarded equally for asserting a truth or asserting a plausible falsehood with high certainty, resulting in extreme overconfidence in uncalibrated regions.

\mathcal{L}_{CE} = -\sum_{k} y_k \log \frac{e^{z_k / T}}{\sum_j e^{z_j / T}}

Cause 03: Dynamics

Autoregressive Error Cascading

Because generation is causal ($P(w_{t+1} \mid w_{1:t})$), once a hallucinated token is generated, it becomes an immutable ground-truth premise in the attention key-value cache. Subsequent self-attention heads condition strictly on the fictional token, making recovery mathematically impossible without backtracking.

\text{Attn}(Q, K, V) = \text{softmax}\left(\frac{Q K^T}{\sqrt{d_k}}\right) V