1. The High-Dimensional Manifold Void
In thousands of dimensions, real training data occupies a thin, folded sub-manifold. Vast expanses of embedding space are completely empty ("the curse of dimensionality"). When a user asks an unusual question, the input embedding lands in this void where no truth anchors exist. Because neural architectures compute continuous affine transformations, they cannot represent an explicit knowledge boundary; they forcefully project the void query onto the closest parametric surface.
2. The Softmax Normalization Fallacy
The final classification layer uses Softmax to produce probabilities. By definition, Softmax is insensitive to the absolute magnitude of all logits—it only measures relative differences. Even if every concept in the vocabulary has near-zero empirical evidence, any tiny relative logit variation is amplified into an 85%–99% confidence score. The model appears wildly confident precisely because the architecture lacks an intrinsic uncertainty channel.
3. Cascading Exposure Bias
During training, teacher forcing supplies ground-truth prior tokens at every single step. In inference, however, the model is fed its own outputs autoregressively ($y_t \sim P(\cdot | \hat{y}_1, \dots, \hat{y}_{t-1})$). If token $t=3$ deviates slightly from reality, it becomes permanent input conditioning for $t=4$. The model rationalizes its own prior error to maintain linguistic coherence, spiraling into fluent, untethered fabrications.