1. Statistical Plausibility ≠ Truth
Neural models are trained on next-token conditional log-likelihood $\max \sum \log P(w_t | w_{<t})$. They optimize for linguistic coherence and statistical mimicry, not verifiable factual state verification.
0.70
Controls entropy. Higher $T$ flattens token logits, boosting improbable distractors.
35%
Distance from training manifold. Sparse data forces smooth probabilistic guessing.
4 tokens
Sequence horizon. Early ungrounded tokens compound error cascades exponentially.
Token Probability Distribution vs. Grounded Truth Manifold
Grounded Fact
Plausible Fiction
High Hallucination
Hallucination Risk
38%
Moderate
Dominant Failure Vector
Manifold Extrapolation
When queries target sparse training clusters, objective likelihood mimics facts by interpolating smooth syntax over factual voids.
Autoregressive Generation Stream Simulation
Prompt: "The primary catalyst for the 1683 Ottoman treaty was..."
2. Manifold Smoothing Gaps
High-dimensional latent embeddings interpolate continuously across sparse training spaces. When data is absent, the model samples fluent intermediate vectors rather than returning a null representation.
3. Compounding Drift
Autoregressive generation feeds each predicted token back into its context window. A single slightly hallucinated token reshapes future attention scores, rapidly steering the generation path away from ground truth.