1. Query-Key Dot Products (Raw Compatibility)
Tokens are projected into Queries (Q) and Keys (K). The matrix product $Q \cdot K^T$ calculates similarity scores between each token pair before normalization.
Query (Q)
Key (K)
Value (V)
Interactive Tokens (Click to inspect focus vector)
Focus Token: "it" (Index 7)
Mathematical Operation
Attention(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V
Computes pairwise compatibility scaled by $1/\sqrt{d_k}$ to prevent gradient vanishing in large dimensions.
Sequence Length
11 Tokens
Selected Query
"it"
Max Attention Partner
"animal" (0.58)
Matrix Sparsity
2.14 nats