Foundational Research Lineage Map
Interactive D3 Dependency Graph
Attention Is All You Need
Attention(Q,K,V) = softmax(QK^T / √d_k)V
Replaces recurrent architectures with self-attention mechanisms, enabling parallelization across token sequences.
Architectural Tensor Sandbox
Live Metric Computations
Attention FLOPs
32,768
Total Parameters
198,144
Matrix Dimension
8x64
Gradient Stability
Stable (1.02)
Sequence Length ($S$)
8
Hidden Dim ($D$)
64
Attention Heads ($H$)
4
LayerNorm Type