1. Token Sequence
9 tokens
Click a query token to inspect its attention distribution across the sequence:
Active Token Vectors
Query (Q) [d_k = 4]
[0.00, 0.00, 0.00, 0.00]
What this token is looking for.
Key (K) [d_k = 4]
[0.00, 0.00, 0.00, 0.00]
What information this token advertises.
Value (V) [d_v = 4]
[0.00, 0.00, 0.00, 0.00]
The content payload transferred if attended.
2. Attention Weight Matrix
Scaling: 1/√4 = 0.5
0.00 (No Attention)
0.50 (Moderate)
1.00 (Maximum Attention)
Click any cell to inspect dot product math
3. Scaled Dot-Product Math Breakdown
Cell: "it" → "animal"
1. Raw Dot Product (q · k)
2.42
Similarity between Query and Key vectors.
2. Scaled Score (q · k / √d_k)
1.21
Prevents vanishing gradients for large d_k.
3. Softmax Weight (α)
0.68 (68%)
Normalized probability over all keys.
Contextual Representation: Attention Output zi = ∑ αij vj
it
Token "it" places its strongest attention weight (68%) onto "animal", resolving the pronoun ambiguity using syntactic and semantic alignment in Head 1.