Transformer
Inside the Transformer
Coreference: "The animal... it was tired"
Polysemy: "bank of the river"
Entity: "Thinking Fast and Slow"
▶ Step Pipeline
Reset
1. Embedding & Pos
2. Q · K · V Projections
3. Multi-Head Attention
4. Add & Norm + FFN
5. Next Token Prediction
Scaled Dot-Product Attention Heatmap
Head:
1
2
3
4
Query (Rows) attending to Keys (Cols)
Temp (τ):
1.0
Stage 1: Token Embeddings & Positional Wave
Embedding Dimension d=64
Super generates helpful tools and automates fact-checking across the internet proactively. If you enjoyed this tool, build your own with Super and share it with a friend.