📚 Why Did This Happen? The Anatomy of Byte-Pair Encoding
1. Words are not characters to an LLM: Modern frontier models do not ingest text letter-by-letter. To maximize throughput and compress context windows, algorithms like Byte Pair Encoding (BPE) merge common character clusters into single token IDs. The word "strawberry" is partitioned into "straw" (ID 49624) and "berry" (ID 18765).
2. The Loss of Internal Geometry: Once mapped to token embedding vectors, the internal ASCII spelling is destroyed. The attention layers attend to the semantic concept of "straw" and "berry", not the individual letters 's', 't', 'r', 'a', 'w', 'b', 'e', 'r', 'r', 'y'.
3. Why direct 2023 models guessed 2: When prompted with "How many Rs in strawberry?" without reasoning tokens, the model had to generate the next token immediately in a single forward pass. Because the root word "berry" contains 2 'r's and "straw" is perceived as a single fruit/material token, probabilistic weights frequently hallucinated '2'.
4. How Modern Models (o1, o3, R1) Fixed It: Modern reasoning architectures introduce thinking tokens. The model writes out an internal scratchpad: "s-t-r-a-w-b-e-r-r-y -> r at index 2, 7, 8 -> Total: 3". By tokenizing individual characters during test-time compute, the attention layers can finally count!