Read the explanation
This teaching page uses four fixed illustrative score tables, rather than running a trained transformer. Select the White Okra preset and head one. In the Okra query row, scores for White, Okra, Harvest and First are point three five, point five two, point zero eight and point zero five. Exponentiation and division by the sum turn them into probabilities. At temperature one, the weights are approximately point two seven one zero, point three two one three, point two zero six nine and point two zero zero eight. The four bars use the same seven hundred pixels per unit probability. Their heights add to one probability unit. The largest score receives the largest share, but the other tokens keep positive shares. Temperature divides every score before exponentiation. Keep the same illustrative scores and slide temperature from point two to three. At point two, the Okra weight is about point six one two one. At three, it moves closer to one quarter. All four bars share the same scale as before. Increasing temperature flattens this particular distribution; it does not change the stored score table. The head pills switch among four illustrative tables. The all-head option averages their raw scores first and then computes softmax. That is the precise operation this tool demonstrates, rather than the concatenation and output projection of a complete multi-head attention layer. Now use causal masking. A query at the second position may attend only to the first and second positions. In this numerical example, replace the two future scores by negative nine hundred ninety nine, exactly as the page does. Their exponentials underflow to zero at temperature one. The remaining scores are renormalized, so White receives about point four five seven six and Okra about point five four two four. These are the masked version of our same example scores, not the different stored scores of the page causal preset. On the actual page, click any matrix cell to select its query and key, inspect the row probabilities, then export the exact selected state as JSON or the drawing as PNG. The export makes this small illustrative computation inspectable; it is not evidence about a real language model.