Click & drag anywhere on the manifold to perturb the current latent hidden state.
The Three Fundamental Mechanics of Neural Hallucination
1. Next-Token Autoregression Cascade
LLMs generate sequences token-by-token using \(P(w_{t} \mid w_{1}, \dots, w_{t-1})\). When a single ungrounded token is selected (even with low probability), it becomes an immutable fact in the prefix. The model then conditions all subsequent probabilities on its own error.
2. Low-Dimensional Manifold Gaps
Real-world factual knowledge resides on thin manifolds in high-dimensional vector space. Between factual clusters lie "off-manifold voids". When sampled with temperature or misleading prompts, representations slip into these voids where interpolation creates plausible-sounding nonsense.
3. Plausibility vs. Truth Objective
The training objective is Maximum Likelihood Estimation (MLE) of human text corpora, which maximizes stylistic and syntactic fluency. The network has no internal epistemological verification engine; it models statistical co-occurrence, not verified reality.