Live Token Order (Drag to Reorder)
4 Tokens
Self-Attention Matrix ($A = \text{softmax}(QK^T / \sqrt{d})$)
NO POS
Pairwise token attention weights. Observe how symmetry or distance decay responds to order.
Positional Embeddings Grid ($P \in \mathbb{R}^{N \times d}$)
SINE / COS
Wavelength harmonics across dimensions $2i$ and $2i+1$ ($\lambda = 10000^{2i/d}$).
INSPECTED REPRESENTATION METRICS
Active Sequence:
"Dog", "bites", "man"
Attention Frobenius Norm:
1.732
Relative Dot Product $\Delta(q_0, k_i)$:
[0.84, 0.42, 0.12]
RoPE Angle Shift $\theta_0$:
0.000 rad (Pos 0) → 1.000 rad (Pos 1)
Ready: Sequence state computed locally via exact Transformer linear projections.