Why Do Neural Networks Hallucinate? — Latent Manifold & Exposure Drift Laboratory

Mechanistic Controls

Diagnostic Presets

Sampling Temperature (τ) 0.70

Latent Activation Steering (α) 0.00

Context Noise / Ambiguity (σ) 0.12

The Mathematical Root Cause

Training Data Manifold: Real facts cluster on an intricate low-dimensional submanifold embedded in high-dimensional space (e.g. 4096D). Outside this ribbon, the model receives zero negative training examples.

When softmax temperature or exposure drift pushes activations off-manifold, decision boundaries stay sharp: the network outputs 99% confidence for completely fabricated statements.

Latent Space Manifold & Vector Probe Drag probe dot anywhere on the canvas

Manifold Distance (d): 0.08

Off-Manifold Curvature: Low (Stable)

Entropy H(p): 0.42 nats

Ground-Truth Data Manifold

Hallucination Basin (Density Void)

Current Token Vector Probe

Autoregressive Token Generation Sequence Factual State

Model Probabilities & Risk

Factual Confidence

94.2%

Ground-Truth

Hallucination Risk

5.8%

Well-Bounded

Next-Token Distribution P(x | h) τ = 0.70

Attention Head Query-Key Alignment Entropy: Sharp

Why does this happen?

Neural networks are interpolators, not factual truth-verifiers. Inside the training ribbon, high density guarantees coherent grounded associations. As soon as latent state leaves the manifold, the dot-product attention maps to spurious memorized tokens with high mathematical certainty.