Input Sequence

Prompt / Context (Max 7 words)

Tired Animal Wide Street Robot Vision Quantum States

Softmax Temperature (τ = 1.0)

0.2 (Sharp/Argmax) 1.0 (Standard) 3.0 (Uniform)

Architecture Stages

1 Token & Positional Embeddings

2 Q, K, V Projections

3 Scaled Dot-Product Attention

4 Multi-Head Combination

5 Feed-Forward & Residuals

6 Autoregressive Next Token

Architecture Specs

Embedding Dim (d_model): 64

Attention Heads (h): 4

Head Dim (d_k): 16

FFN Inner Dim (d_ff): 128

Scaled Dot-Product Attention Matrix

Interact with attention scores calculated via Softmax((Q • K^T) / √d_k). Click any token or cell to inspect query-key interactions.

Active Head:

Focuses on pronominal bindings & entity links

Max Attention Pair

it → animal

Attention Entropy

1.42 nats

Active Head Sparsity

28.4%

Top Next Token

because (p=0.41)