2. Self-Attention Heatmap $\mathbf{A} = \text{softmax}(\frac{\mathbf{Q}\mathbf{K}^T}{\sqrt{d_k}})$
Click any matrix cell to trace its exact scaled dot-product and softmax resolution.
3. Matrix Arithmetic Inspector
Cell [0,0]
Query · Key Vectors ($d_k=4$)
Q [token] • K [token]
q · k = 0.00
Scaling & Temperature Factor
Divide by $\sqrt{d_k} = \sqrt{4} = 2.0$ with $\tau$ scaling:
score = 0.00
Row Softmax Exponentiation
$\alpha_{ij} = \exp(z_{ij}/\tau) / \sum_m \exp(z_{im}/\tau)$
Weight = 0.000
Projected Next-Token Distribution