Autoregressive Token Context & Hallucination Tracker
Grounded Truth (Purity 98%)
Current Token Position:
Step 3 / 8
|
Click any token above to inspect its attention distribution & manifold state.
Latent Manifold: Grounded Data vs Ghost Attractors
2D t-SNE / PCA Projection
Attention Allocation & Next-Token Probabilities
Softmax($QK^T / \sqrt{d}$)
Next-Token Softmax Logit Competition
The 3 Pillars of Neural Hallucination
1. Maximum Likelihood Bias
Neural LLMs are trained to maximize $P(w_t \mid w_{
2. Autoregressive Exposure Trap
During generation, every sampled token is appended to the prompt as ground truth without back-tracking. If sampling noise ($T>0$) picks a distractor token $w_k$, the model conditions all future tokens on that fiction, creating an inescapable confabulation chain.
3. Uncalibrated Manifold Voids
In high-dimensional spaces ($d \approx 4096$), true training samples occupy a thin curved submanifold. In between factual clusters lies immense empty space where softmax extrapolates arbitrary smooth probability gradients, rendering the model blindly overconfident.