An interactive explainer
Large language models feel like magic, but under the hood they run a surprisingly understandable loop. Scroll through five interactive steps — inspired by a viral visual breakdown — and watch each stage of the pipeline come alive.
Step 1
The AI doesn't read whole words. It chops your text into tokens — subword chunks like "un", "believ", "able" — and assigns each a numeric ID. Type below and watch the tokenizer split your text in real time.
Step 2
Each token ID is mapped to an embedding: a long vector of numbers. Words with similar meanings end up near each other in this space. This is a 2D slice of that geometry — hover or tap points to inspect them.
Notice: king, queen, prince cluster together; cat, dog, puppy form another island; run, walk, jump a third.
Step 3
Inside the transformer, self-attention lets each token weigh how relevant every other token is to it. That is how "it" figures out what it refers to. Hover or tap a token in the sentence below to see where its attention flows.
Try "it" — the model attends strongly to "cat" and "mat".
Step 4
After all the layers, the model outputs a probability distribution over its entire vocabulary for the single next token. Pick a prompt and see how the odds shift.
Step 5
Generation is just this prediction repeated: sample a token, append it, predict again. Temperature controls how adventurous the sampling is — low is predictable, high is chaotic. Run the loop yourself.
Each Generate press samples 12 more tokens from a tiny toy model, one at a time — exactly the loop a real LLM runs at scale.
This page does not run a language model, tokenizer vocabulary or trained embeddings. It slices each non-whitespace word into chunks of four UTF-sixteen code units while more than four remain, then keeps the remainder. Unbelievable gives u-n-b-e, l-i-e-v and a-b-l-e: lengths four, four and four, twelve total. Bars at fifty display pixels per code unit are two hundred pixels each. The displayed identifier is a hand-written hash modulo fifty thousand two hundred fifty-seven, not a learned vocabulary index. Thirteen fixed two-dimensional points draw word clusters; a label suggesting four thousand ninety-four additional dimensions does not mean those dimensions exist in the source. For the selected word it, the source assigns cat fifty-five percent, mat thirty percent and soft ten percent. Those three displayed weights sum to ninety-five percent. Other cells use a point zero four fallback and the selected token uses one, so this display is not a normalized learned attention row. Bars at five pixels per percentage point have lengths two hundred seventy-five, one hundred fifty and fifty. The next-token prompt cat sat on the shows six fixed probabilities totaling ninety-three percent, leaving seven percent unrepresented. The other two prompt totals are ninety-nine point five and eighty-seven percent. These are illustrative lookup tables, not measured outputs from a real transformer. The first three toy transition weights are point five, point three and point two. At temperature one they retain fifty, thirty and twenty percent. At temperature one half, squaring gives point two five, point zero nine and point zero four; dividing by their point three eight sum gives about sixty-five point seven nine, twenty-three point six eight and ten point five three percent. Bars at five hundred pixels per unit probability are about three hundred twenty-eight point nine five, one hundred eighteen point four two and fifty-two point six three pixels. Generate schedules up to twelve more local samples, one at a time. If a token has no following table, it appends a period and stops early. Repeated clicks can start overlapping loops, and Reset does not cancel an already scheduled callback; neither is a trained model or a safe asynchronous cancellation mechanism.