Inside the Transformer Attention Lab

1. Token Sequence & Attention Arcs Focus: it [Token #7]
Attention(Q, K, V) = Softmax( (Q · Kᵀ) / √dₖ ) · V
2. Attention Weight Matrix (N × N Heatmap) Softmax Probabilities
3. Projected Vectors for Focused Token dₖ = 4
Query (Q) W_q·x
[+0.42, -0.81, +0.15, +0.94]
Key (K) W_k·x
[+0.38, -0.79, +0.19, +0.88]
Value (V) W_v·x
[+0.12, +0.65, -0.44, +0.31]
Cosine Positional Waves merged with token embedding: PE(pos, 2i) = sin(pos / 10000^(2i/d))
4. Feed-Forward & Next Token Prediction LayerNorm + MLP