Transformer Attention Head Visualizer

1. Input Sequence

2. Active Query Token (Qᵢ)

3. Scaling Scale (√dₖ) 2.00

0.5 (Sharp) 2.0 (Standard) 5.0 (Diffuse)

Attention Weights & Connection Flow

Click any token badge below to make it the active Query

Step-by-Step Score Computation

Key (Kⱼ)	Dot Product (Q·K)	Scaled (Score/√dₖ)	Softmax Weight (α)

Synthesized Value Vector Output

The output vector zᵢ = Σ (αᵢⱼ · vⱼ) aggregates semantic value projections from all tokens according to their normalized attention weights.

Attention insight: Notice how coreferent pronouns (like "it") assign significant attention probability to their antecedent noun phrase based on query-key compatibility.