How Large Language Models Actually Work, in Plain English

A large language model is, at its core, a system that predicts the next piece of text. Give it the start of a sentence and it calculates which word fragment is most likely to come next, adds it, and repeats, thousands of times a second. Everything a modern chatbot appears to do, from drafting a cover letter to explaining a tax form, is built on that single trick, scaled up to an enormous degree. Understanding the trick makes the tools far easier to use well, and far easier to distrust at the right moments.

Step one: turning words into tokens

Computers do not read words; they read numbers. So the first thing a language model does is split text into tokens, which are common chunks of characters. A short, frequent word like "the" is usually one token. A longer or rarer word, such as "unconstitutional," may be broken into several pieces. Punctuation and spaces count too.

Each token is mapped to an ID number, and each ID is mapped to a long list of numbers called an embedding. You can think of an embedding as coordinates in a space with hundreds or thousands of dimensions. Tokens that are used in similar ways end up near each other: "doctor" sits closer to "nurse" than to "bicycle." The model does not know what a doctor is in the way a person does, but it knows, numerically, which contexts the word tends to appear in.

Tokens matter to ordinary users for two practical reasons. Many AI services price their APIs per token, and every model has a limit on how many tokens it can consider at once.

Step two: training on a mountain of text

A model starts as a huge network of numbers, called parameters or weights, set more or less at random. Training adjusts them. The process, called pre-training, works like an endless fill-in-the-blank exercise:

1. Take a passage of text from the training data.
2. Hide the next token and ask the model to predict it.
3. Compare the prediction to the real answer.
4. Nudge millions or billions of parameters slightly so the right answer becomes a little more likely next time.

Repeat that across an immense collection of books, websites, code and other text, and the parameters gradually encode grammar, facts, writing styles, reasoning patterns and the structure of programming languages. Nobody writes these rules in by hand. They emerge because predicting text well requires capturing them.

This is also where many of the field's legal and ethical debates begin. What went into the training data, who owns it and whether it was licensed are live questions in American courts and at the US Copyright Office.

Step three: the transformer and "attention"

The architecture behind nearly every major language model is the transformer, introduced by Google researchers in a 2017 paper titled "Attention Is All You Need." Its key idea is attention: when the model processes a token, it looks back at every other token in the passage and decides how much each one matters.

Consider the sentence "The trophy didn't fit in the suitcase because it was too big." To predict what comes after, the model needs to know that "it" refers to the trophy, not the suitcase. Attention lets the model weigh "trophy" heavily when interpreting "it." Stack dozens of these attention layers on top of each other, and the model can track who did what to whom across long documents.

Transformers also train efficiently on the specialized chips used in data centers, which is a big reason models grew so quickly in size and capability after 2017.

Step four: turning a predictor into an assistant

A freshly pre-trained model is good at continuing text but not at being helpful. Ask it a question and it might respond with three more questions, because that is a plausible continuation of a list of questions online. Turning it into a useful assistant takes further training:

• Instruction tuning (a form of fine-tuning) trains the model on examples of instructions paired with good responses, so it learns the shape of a helpful answer.
• Reinforcement learning from human feedback (RLHF) and related methods have people, or other models guided by written principles, compare candidate answers and mark which is better. The model is then adjusted to prefer the kinds of answers that rate well: accurate, clear, polite and, ideally, honest about uncertainty.
• Safety training teaches the model to decline certain requests and to handle sensitive topics carefully.

This stage is why two products built on similar underlying technology can feel quite different. Their personalities, caution levels and formatting habits are largely set here.

What happens when you type a question

Once deployed, the model answers in a loop called inference:

The amount of text the model can consider in one go is its context window, measured in tokens. Anything outside it, such as the start of a very long conversation, effectively disappears. That is why long chats sometimes "forget" instructions you gave early on, and why pasting a fresh summary into a new chat can work better than continuing an old one.

Why models get things wrong

The design explains the most common failure, often called hallucination. The model is optimized to produce text that is plausible, not text that is verified. When it has seen a fact many times in training, plausible and true usually coincide. When it has not, such as with an obscure court case, a niche statistic or a recent event, it may still produce a fluent, confident answer, because a confident answer is what the pattern of the conversation calls for.

Other limits follow from the same mechanics:

• Knowledge cutoff. A model knows little about events after its training data was collected, unless the product adds a search step.
• Arithmetic and counting. Models see tokens, not individual digits or letters, so precise math and character counting can trip them up. Many products now route calculations to a separate tool.
• Sensitivity to wording. Because the output depends on every token in the input, small changes in phrasing can change the answer.

Add-ons that make models more reliable

Product makers wrap models in extra machinery to cover these gaps:

• Retrieval-augmented generation (RAG) searches a set of documents, such as a company's policies or the web, and inserts the relevant passages into the prompt so the model answers from them rather than from memory.
• Tool use lets the model call a calculator, run code, query a database or check a calendar, then incorporate the result.
• Reasoning modes have the model work through intermediate steps before giving a final answer, which tends to help on multi-step problems at the cost of speed.

These additions are why the same underlying model can be noticeably more accurate inside one product than another.

What this means for everyday use

Knowing the mechanics suggests a few habits. Give the model the source material when accuracy matters, rather than asking it to recall facts. Ask for citations, and click them. Keep important instructions near the end of long prompts, or restate them. Treat fluent prose as a draft, not a verdict, especially for legal, medical or financial questions. For organizations, the federal NIST AI Risk Management Framework offers a vendor-neutral way to think through these risks.

A language model is neither a search engine nor a mind. It is a remarkably capable pattern engine, and it works best when you hand it the right patterns to work from.