Formula:
Attention(Q,K,V) = softmax(Q·Kᵀ / √dₖ)·V
Attention Distribution Heatmap
Click any token badge below to inspect how it queries the rest of the sentence.
Unlike recurrent networks that read sequentially, Transformers process all tokens simultaneously. Each token generates Query (Q), Key (K), and Value (V) vectors to calculate dynamic attention weights across the entire sequence.
Attention(Q,K,V) = softmax(Q·Kᵀ / √dₖ)·V
Click any token badge below to inspect how it queries the rest of the sentence.