Why Do Neural Networks Hallucinate?
LLMs predict plausible next tokens via statistical associations, not verified truth. When prompts cross sparse training regions or high sampling temperatures flatten probability distributions, autoregressive drift compounds errors into confident confabulations.
Inference Parameters
Lower values simulate out-of-distribution prompts with sparse training support.
Higher softmax temperature flattens logits, boosting improbable, non-factual tokens.
Each hallucinated token becomes context for the next, exponentially increasing drift.
Model remains within plausibility bounds but minor factual deviations may emerge.
1. Statistical Plausibility ≠ Truth
Neural networks optimize for maximum likelihood estimation over token sequences, learning syntax and stylistic cadence rather than formal causal proofs or truth invariants.
2. OOD Manifold Gaps
When prompts land in unrepresented feature space, attention heads interpolate smoothly across nearest neighbors, synthesizing confident fiction from disjoint memory vectors.
3. Compounding Autoregression
Generation is autoregressive: output tokens are fed back as immutable ground truth. A single early deviation skews all subsequent attention weights down an invalid path.