Transformer Architecture Deconstructed
Interactive Multi-Head Self-Attention Sandbox
Scaled Dot-Product: Softmax(QKᵀ / √dₖ)V
Preset Sentences
Pronoun Resolution (it -> animal)
Syntactic Context (bank -> river)
Self-Attention & Positional
Custom Tokens (Max 10)
Attention Head Archetype
H1: Coref
H2: Syntax
H3: Local
H4: Global
Interactive Attention Flow (Click or hover query token)
1. Token + Pos. Embedding
2. Q, K, V Projections
3. QKᵀ / √dₖ Attention
4. Weighted Sum (V)
5. FFN & Output Logits
Attention Matrix (Softmax)
Q, K, V Vector Inspector (dₖ = 4)
Token: "it"
Query (Q):
What this token seeks •
Key (K):
What features it exposes •
Value (V):
Content payload aggregated into the next layer representation.
Super generates helpful tools and automates fact-checking across the internet proactively. If you enjoyed this tool, build your own with Super and share it with a friend.