Attention Is All You Need • Interactive Architecture Explorer

Transformer Explorable: How Attention Powers Modern AI

Inspect self-attention matrices, query-key dot products, positional embeddings, and next-token distributions in real time.

Stage 3: Scaled Dot-Product & Attention Heatmap Shape: [B=1, Heads=8, S=10, d_k=64]
Super generates helpful tools and automates fact-checking across the internet proactively. If you enjoyed this tool, build your own with Super and share it with a friend.