Formula:
Attention(Q,K,V) = softmax(Q·Kᵀ / √dₖ)·V
Attention Distribution Heatmap
Click any token badge below to inspect how it queries the rest of the sentence.
Unlike recurrent networks that read sequentially, Transformers process all tokens simultaneously. Each token generates Query (Q), Key (K), and Value (V) vectors to calculate dynamic attention weights across the entire sequence.
Attention(Q,K,V) = softmax(Q·Kᵀ / √dₖ)·V
Click any token badge below to inspect how it queries the rest of the sentence.
Where no authored weight row overrides it, this source positional fallback gives a non-self key score one divided by distance plus one. At distances one and two the scores are one half and one third. Bars share six hundred pixels per raw score. This educational source mixes authored rows and heuristic normalized scores; it does not calculate learned query-key dot products or run a trained transformer. Before the source rounds normalized fallback entries to three decimals, keys with raw scores one half and one third retain weight ratio one point five. Bars compare ratio one point five against equal-weight ratio one using two hundred pixels per ratio unit. Dividing both scores by their shared sum preserves relative preference. The source performs direct normalization for this fallback rather than exponential softmax. A hypothetical dense attention matrix for five and ten tokens has twenty-five and one hundred query-key pairs. Bars share four pixels per pair. This source teaching interface shows the current query distribution and contextual explanation with authored and fallback weights. All sentences, heads, focus dropdown options, clickable tokens and actual reset must remain preserved through native resize, original first-footer placement and current public playback checks; independent video-scene approval remains separate.