Token Attention Flow
Click token as Query (Q)Attention Weights Heatmap (A = Softmax(S))
Vector Routing Engine
Token: "it"Attention Distribution for Active Query:
Mathematical Step:
1. $s_{i,j} = \frac{q_i \cdot k_j}{\sqrt{d_k}}$ (Scaled Dot Product)
2. $\alpha_{i,j} = \frac{\exp(s_{i,j})}{\sum_l \exp(s_{i,l})}$ (Normalized Softmax)
3. $z_i = \sum_j \alpha_{i,j} v_j$ (Contextualized Output Embedding)
1. $s_{i,j} = \frac{q_i \cdot k_j}{\sqrt{d_k}}$ (Scaled Dot Product)
2. $\alpha_{i,j} = \frac{\exp(s_{i,j})}{\sum_l \exp(s_{i,l})}$ (Normalized Softmax)
3. $z_i = \sum_j \alpha_{i,j} v_j$ (Contextualized Output Embedding)