An interactive explainer
Large language models feel like magic, but under the hood they run a surprisingly understandable loop. Scroll through five interactive steps — inspired by a viral visual breakdown — and watch each stage of the pipeline come alive.
Step 1
The AI doesn't read whole words. It chops your text into tokens — subword chunks like "un", "believ", "able" — and assigns each a numeric ID. Type below and watch the tokenizer split your text in real time.
Step 2
Each token ID is mapped to an embedding: a long vector of numbers. Words with similar meanings end up near each other in this space. This is a 2D slice of that geometry — hover or tap points to inspect them.
Notice: king, queen, prince cluster together; cat, dog, puppy form another island; run, walk, jump a third.
Step 3
Inside the transformer, self-attention lets each token weigh how relevant every other token is to it. That is how "it" figures out what it refers to. Hover or tap a token in the sentence below to see where its attention flows.
Try "it" — the model attends strongly to "cat" and "mat".
Step 4
After all the layers, the model outputs a probability distribution over its entire vocabulary for the single next token. Pick a prompt and see how the odds shift.
Step 5
Generation is just this prediction repeated: sample a token, append it, predict again. Temperature controls how adventurous the sampling is — low is predictable, high is chaotic. Run the loop yourself.
Each Generate press samples 12 more tokens from a tiny toy model, one at a time — exactly the loop a real LLM runs at scale.