Model Lab

Transformer Self-Attention Explorer

Query: "it" [pos: 7]
Top Attention Target: "animal" (68.4%)
Entropy: 0.94 bits
Token Interaction Graph & Attention Beams Softmax(Q · KT / √dk)
Width = Weight αi,j
Full Attention Matrix A N × N
Head 0
Vector Projections & Context Value zi = ∑ αi,j vj
dmodel = 6
Q:
K:
V:
Z:
Super generates helpful tools and automates fact-checking across the internet proactively. If you enjoyed this tool, build your own with Super and share it with a friend.