Blog · AI guide
How a language model works, explained without jargon
Behind every chatbot is a program that does one thing, over and over: guess which piece of text comes next. Understanding that explains almost everything else, including why it is sometimes wrong with total confidence.
The idea in one sentence
A language model is a program that looks at a piece of text and works out which chunk of text is most likely to come next. It picks one, adds it to the end and calculates again. Chunk by chunk, that is how it writes answers, poems, summaries or code. It doesn’t look anything up in a database of ready-made replies: it builds them on the spot, like someone finishing a sentence that was left hanging.
That sounds far too simple to explain what a modern chatbot can do, but that’s the trick: to guess the next word well across millions of different situations, the model had to soak up a huge number of patterns in language, in facts and in the way people reason in writing.
First, the text is cut into tokens
The model doesn’t read letters or whole words. It reads tokens: chunks of text that can be a full word, part of a word, a punctuation mark or a space with something attached. A common word is usually a single token; a rare or long one gets split into several. Each token becomes a number, and that number becomes a list of values the model knows how to work with.
This explains some quirks you may have noticed. Ask it how many times the letter “r” appears in a word and it can get it wrong, because it never saw the word letter by letter. And languages with less training text available tend to be split into more tokens, which makes them a bit more expensive and slower to process.
Predict the next piece, again and again
Once the tokens are in, the model runs a huge calculation and ends with a list: for every possible token, how likely it is to come next. After “The cat sleeps on the,” “couch” gets a high probability and “Tuesday” a very low one. The program picks one, adds it to the text and runs the whole process again on the slightly longer sentence.
It doesn’t always pick the top choice. If it did, its answers would be repetitive and flat. So a bit of chance is mixed in, controlled by a setting often called “temperature”: low, and the model is more predictable; high, and it’s more varied and more likely to wander off. That’s why the same question can get you two different answers.
This stage, when the model is finished and simply generates text, is called inference. It runs on computers with specialized chips, usually GPUs, because every single token takes millions of multiplications.
How it learns: training
A brand-new model is a neural network with billions of numbers, its parameters, set at random. At first its predictions are garbage. Training means showing it enormous amounts of text, hiding the next word, letting it guess and nudging all those numbers slightly so it does better next time. Repeat that an astronomical number of times and you have deep learning.
Nobody teaches it grammar rules or loads an encyclopedia into it. What it knows ends up spread across its parameters as statistical patterns, not as tidy index cards it can cite. That’s why it can’t tell you which page a fact came from.
That first stage produces a model that completes text, but not necessarily one that holds a good conversation. Next comes fine-tuning on examples of questions and good answers, and often a stage where people rate which answer is better. This technique, known as RLHF (a form of reinforcement learning), was described in an influential 2022 paper on training models to follow instructions. It’s what turns a giant autocomplete into an assistant that answers, declines certain requests and keeps a consistent tone.
Why it sometimes makes things up
When a model confidently states something false, like a quote nobody said or a book that doesn’t exist, it’s called a hallucination. It isn’t a rare glitch or a lie: it follows directly from how the model works. It always produces the text that sounds most likely, and a well-formed made-up answer can sound just as likely as a true one.
How models are graded matters too. A 2025 study on why language models hallucinate argues that many tests reward guessing over saying “I don’t know,” the way a multiple-choice exam with no penalty for wrong answers rewards guessing. Hallucinations can be reduced, for example by giving the model trusted documents to consult (known as RAG), but they can’t be ruled out entirely today. For anything important, check an original source.
The context window: its working memory
The model doesn’t remember your conversation the way a person would. Every time you reply, the system sends it all the relevant text again: the instructions, your messages, its earlier answers and any documents you pasted in. The limit on how much fits is called the context window, and it’s measured in tokens.
If a conversation goes past that limit, something has to be left out, and the model “forgets” the oldest parts or works from a summary. Even within the limit, not everything counts the same: a study called “Lost in the Middle” found that several models make better use of information at the beginning or end of a long text than of information buried in the middle. The practical lesson: put what matters at the top or the end, and say it clearly.
What a prompt really does
A prompt is the text you start with: the question, the instruction and any context you provide. Since the model continues whatever it receives, a prompt isn’t a magic command; it’s the beginning of the text the model is about to complete. That’s why the result changes so much when you say who it’s for, what tone you want, what format you expect, or show an example of what you’re after.
A good prompt looks a lot like a good brief for someone new on the team: explain the goal, give them the information they need, say what you don’t want and ask for things step by step. You don’t need a special language or secret formulas.
What it can’t do (or doesn’t do well)
- Know the latest news. What it learned stops at the date its training data was collected. If it answers about something from this week, it’s because the system gave it a search or a document, not because it knows.
- Calculate exactly. It predicts text; it doesn’t do math like a calculator. Long calculations can go wrong unless it uses a separate tool.
- Check its own claims. It has no built-in way to tell a true memory from an invented one.
- Be neutral by nature. It learns from human writing, prejudices included. Algorithmic bias can be reduced, but it doesn’t go away on its own.
- Have experiences or intentions. It can write “I love that” or “I understand how you feel” because that’s how people write in those situations, not because it feels anything.
Some versions go beyond text: multimodal models also take in images or audio, and reasoning models write out intermediate steps before answering, which tends to help with logic or math problems. Under the hood, though, they are still generating one token after another.
The short version
A language model cuts your text into tokens, works out which piece is most likely to come next, adds it and repeats. It learned to do that by reading enormous amounts of text and adjusting billions of numbers, and was then polished to hold a conversation. That makes it surprisingly useful for drafting, summarizing, translating or thinking out loud, and it also explains its flaws: it can make things up, it forgets what doesn’t fit in its window and it doesn’t know what it never saw. Treat it as a fast, articulate collaborator, and leave the final word to trusted sources. If you want to keep going, the Aster encyclopedia explains every term.