Token Ribbon & Positional Encoding
0 tokens
Multi-Head Attention Arcs
Attention Weight Matrix: Softmax(Q · Kᵀ / √d_k)
Query (q)
Key (k)
Value (v)
Attention Score (α)
Step-by-Step Arithmetic
Inspect dot-product & softmax for token interaction:
Select a cell in the matrix
1. Dot Product: q_i · k_j = 0.00
2. Scaled (/ √d_k): z_ij = 0.00
3. Softmax: α_ij = exp(z_ij)/Σ = 0.00
4. Value contribution: α_ij * v_j
Next Token Prediction Logits