Phase 3: Scaled Dot-Product Self-Attention
Observing how token vectors attend to all other token positions simultaneously via Q · Kᵀ / √d_k.
Attention(Q, K, V) = softmax(Q·Kᵀ / √d_k) · V
Query Vector (Q)
"What am I looking for?"
Key Vector (K)
"What features do I offer?"
Value Vector (V)
"What information do I pass along?"
Attention Weight:
0.842
Key Mechanism: When token
"it" computes its attention query, its highest dot-product match is with "animal" rather than "street", enabling the transformer to accurately resolve pronoun coreference.