How LLMs Actually Work — An Animated Walkthrough

Most explainers of large language models jump straight to equations. This one starts with a moving picture: press play, and watch the sentence "The cat sat on the mat because it was" travel all the way from raw text to the next predicted token. Then, below the animation, each stage gets a short, honest explanation — what really happens, and where the popular metaphors lie.

The one-sentence version

A large language model is a function that takes a sequence of tokens and returns a probability distribution over the next token. That's the entire product. Everything else — the apparent reasoning, the style, the facts it recites — is what emerges when you fit that one function on enough text with enough parameters and then sample from it over and over.

If you internalize that sentence, the rest of this page is just mechanical detail about how the function is built.

Stage by stage

From letters to IDs

The model never operates on characters or words — it operates on tokens, the units a tokenizer produces. Modern models (GPT-style, Qwen, Llama) use byte-level BPE: frequent strings like " cat" stay one token, rare compounds get split into pieces the vocabulary actually contains. The tokenizer itself is learned from the training corpus — which is why non-English text often costs more tokens per word than English.

One consequence that surprises people: the model has no built-in concept of a "word". Spelling tricks, homoglyphs, or unusual spacing can change the token stream — and therefore the computation — even when the meaning for a human is identical.

From IDs to geometry

Each token ID indexes a row of the embedding matrix: a vector of thousands of numbers. After training, distances between these vectors carry structure — "cat" and "dog" sit closer together than "cat" and "invoice", because the training process pushed them there to predict their shared contexts better.

Position gets encoded too (modern models use rotary embeddings, RoPE), because "dog bites man" and "man bites dog" must compute differently even though they contain the same tokens.

Attention: the actual idea

Attention is the load-bearing concept. For every token, the model computes three projections of its vector — a query, a key, and a value — and then:

1. the query of token i is dotted against the keys of all tokens it may look at (in a decoder, all earlier ones),
2. the scores are scaled and softmaxed into weights that sum to 1,
3. the output is the weighted mixture of the values.

That's literally a soft lookup table: "given what I'm asking (query), how relevant is each earlier token (key), and what should I bring back from the relevant ones (value)?" In the animation, the thickest line goes from "it" to "cat" — resolving that pronoun is precisely the kind of thing attention weights implement.

The quadratic cost of step 1 — every token against every earlier token — is why long context is expensive, and why techniques like grouped-query attention (GQA), sliding windows, and KV-cache quantization exist: they all attack this term.

Why depth beats width

One attention layer plus a feed-forward network is a block. Stacking 30–90 of them is what turns pattern matching into something that composes. Empirically, shallow blocks learn syntax and local patterns; deeper blocks implement longer-range dependencies, factual recall, even in-context algorithms like induction heads. Each block also gets a residual connection — a direct highway for the token's original vector — so the stack edits representations progressively instead of replacing them.

This is also where the parameters live: the block's matrices scale with the square of the hidden size, which is why going from a 4B to a 70B model multiplies weight memory roughly 17×, not linearly. The LLM inference math guide covers the resulting memory and compute budgets, and the Qwen3-Next notes show how sparse MoE activation bends the rule.

Sampling: where "creativity" hides

The final hidden vector of the last position is projected onto the vocabulary (one number per possible token), and softmax converts scores into probabilities. The model itself is fully deterministic given a seed; the apparent variability comes from sampling out of that distribution:

• Temperature rescales the logits before softmax. Low (0.2) → near-greedy, repetitive but safe. High (1.2) → flatter distribution, more surprising — and more wrong — picks.
• Top-p / top-k truncate the distribution before sampling, cutting the long tail of nonsense tokens.

The animation's stage 6 shows exactly this: the same context, a distribution over candidate next tokens, and one sampled winner.

Training: predict, correct, repeat

Pre-training is embarrassingly simple at heart: show the model a chunk of text, ask it to predict each next token, measure how wrong it was (cross-entropy), and nudge all the matrices a tiny step in the direction that reduces the error (backpropagation + an optimizer). Repeat over trillions of tokens. There is no teacher that explains why — only the raw pressure of the next token.

What modern labs add on top: supervised fine-tuning on curated instruction/response pairs, then preference training (RLHF or DPO) so the model's outputs follow the distribution of helpful, human-preferred continuations rather than merely likely ones. The post-trained model answers questions; the base model just continues text.

What the animation leaves out

Honesty pass — the parts the pretty version skips:

• KV cache & decode mechanics. Real generation caches keys and values so each new token doesn't reprocess the whole prompt; that cache dominates serving memory at long context. The inference calculator lets you compute it.
• MoE routing. Many current models (including Qwen3-Next) route each token through a subset of experts; the six stages stay identical, only the feed-forward step becomes sparse.
• Multimodality. Images, audio and video enter as more tokens; from stage 3 onward the pipeline is the same.
• Reasoning models. "Thinking" tokens are still the same next-token loop — just run longer before answering. The machinery in this article doesn't change.

If you want the performance side — how all this maps to FLOPS, bandwidth, and euros — continue with the inference economics page, which builds on the same pipeline from the hardware's point of view.