LLM Transformer Token Flow & Vector Attention Lab

Tokens:

Model & Sequence Config

Preset Benchmarks

Input Sequence Prompt

Attention Head Inspection

Attention Temperature 1.00

Query Vector Weight Scaling 1.0

Key Vector Weight Scaling 1.0

Architecture Params

Dim ($d_{model}$): 64
Heads ($h$): 4
Head Dim ($d_k$): 16
Layers: 2

Scaled Dot-Product Attention Matrix

Score = Softmax($Q K^T / \sqrt{d_k}$) across all sequence tokens

Matrix Size: 10x10

Sequence Length

Active Heads

Max Attention Weight

0.884

Top Output Token

representation

Compute Latency

12ms