How Do Transformers Work?

This page computes attention-like weights from an authored score rule for one selected query. It does not run a trained transformer or compute query, key and value vectors. Its first native sentence contains twelve tokens, including the separate words did and not. Its third sentence contains nine. Choosing the actual sentence option therefore changes the number of token buttons and key weight rows from twelve to nine. Each held tile here identifies one actual source token. There is no fourteen-token option and no fractional token count. The public page displays one query distribution, not a complete square matrix. The explainer therefore counts the native rows that actually exist rather than claiming it renders one hundred forty four cells. These counts teach the difference between sentence length and a selected query row; they do not measure a transformer implementation. The prose in the product mentions value-vector mixing, but the source code does not implement that operation. Hold the first sentence and select it as the query, at position nine. Changing the native head from coref to syntax changes the source score rule. In coref, the animal key has raw score five point seven and street has one point seven. In syntax, animal has zero point eight while street, two positions from the query, has three point zero. The code divides these assigned scores by one point five, exponentiates them and normalizes over all twelve keys. Every held bar here is the complete actual normalized row, with a separate value label. On the common seven hundred pixels per weight-unit scale, the corresponding widths reflect those exact values. Animal to street has a weight ratio of about fourteen point three nine in coref and zero point two three zero seven in syntax. Syntax does not make these two weights equal. The distance-two rule explains the change. This is a causal comparison of a genuine native head selection, not learned linguistic evidence. The page does not train these rules, apply a temperature slider, or produce a value-vector output. Keep the first sentence and the coref head. The native query menu or token buttons can change the selected query from it, position nine, to animal, position two. With it selected, the source has special assigned boosts for animal, street and tired. With animal selected, those it-specific branches do not apply. The animal key instead receives the self score of two point five, and other keys use the distance-based fallback unless a different explicit query branch applies. All twelve scores are exponentiated after division by one point five and normalized together. The two complete held rows show this genuine query action on the same seven hundred pixels per normalized weight-unit scale. Every key stays individually labeled, including repeated the tokens identified by position. Both sums are one because the code normalizes each calculated row. This is an authored heuristic teaching model. Its global head uses a cosine rule and is not actually uniform. No trained query-key dot product, learned semantic claim or value mixing is inferred from these bars. The original controls and source mathematics are preserved.