1. Interactive Attention Connections
Click any token to inspect its Query (Q) attending to all Keys (K)
Attention Weight Distribution ($\mathrm{Softmax}(\frac{Q \cdot K^T}{\sqrt{d_k}})$)
Normalized [0.00 - 1.00]
2. Attention Weight Matrix
Full sequence pairwise correlation heatmap
0.0 (Low)
1.0 (High)
1
Query, Key & Value Vectors
Each word embedding transforms into 3 vectors: Query (what it seeks), Key (what it contains), and Value (its substantive features).
2
Scaled Dot-Product Softmax
Dot-products $Q \cdot K^T$ scale by $\sqrt{d_k}$ to prevent gradient saturation. A softmax function converts raw affinity scores into probabilities summing to 1.0.
3
Weighted Value Aggregation
The output vector is the linear blend of all Value ($V$) vectors weighted by attention scores, letting words contextualize simultaneously across entire sequences.