Transformer Attention Matrix Studio

Input Sequence Presets

Active Token Sequence (space separated):

Attention Hyperparameters

Embedding Dim ($d_{model}$) 64

Head Dim ($d_k = d_v$) 16

Softmax Temperature ($\tau$) 1.00

Causal Masking (Decoder Mode)

Multi-Head Attention Heads

Inspector: Click any cell or token

Select a matrix cell in the center viewport to view exact Query-Key dot product, scale factor ($1/\sqrt{d_k}$), raw logit, and normalized attention probability.

Focus Query Token:

Active Tokens: 8 Max Attention Pair: "it" → "animal" (0.84) Attention Entropy: 1.42 nats

Status: Computed live in-browser

Detailed Computation Log (Active Query Row) Token: "it" (index 5)

Target Key Token	Raw Dot Product ($Q \cdot K_j$)	Scaled ($/ \sqrt{d_k}$)	Exp Score ($e^{s/\tau}$)	Softmax Attention ($A_{i,j}$)	Cumulative Context Vector Contribution