Transformer Attention Gating & Entropy Workbench

Query-Key Attention Matrix

Click cell to edit raw Query-Key logit score

Live Softmax

Context Routing Graph & Entropy

Visualizing token signal distribution & token gating

Sequence Scenarios:

Softmax Temperature (τ) 0.70

Lower τ sharpens focus; higher τ diffuses attention.

Attention Mask Rule

Top-K Cutoff 3

Mathematical Gating Logic:

Attention(Q,K,V) = Softmax( Mask( Q · K^T / τ ) ) · V | H(P) = - ∑ p_i log₂(p_i)

Gating Analysis & Sequence Impact

Loading interactive sequence computation...