What Is a Large Language Model? A Plain Guide

Chatbots such as ChatGPT, Claude and Gemini have become part of everyday life in only a few years. All of them are built on the same kind of technology: the large language model, or LLM.

Large language models can write, summarise, translate and answer questions with startling fluency, and they can also state falsehoods with complete confidence. Understanding how they work helps explain both. Here is a plain guide, without the jargon.

What an LLM is

A large language model is a machine learning model, usually a neural network, trained on vast amounts of text to perform language tasks. Large refers both to the amount of text it learns from and to the size of the model itself, measured in parameters, the internal numbers adjusted during training.

Some models are enormous. In January 2025 the Chinese company DeepSeek released DeepSeek-R1, an openly available model with 671 billion parameters. Training the biggest models takes huge amounts of computing power and money.

The transformer

The modern wave of LLMs rests on one invention. In 2017 Google researchers published a paper called Attention Is All You Need, which introduced the transformer, a design that lets a model weigh how each word in a passage relates to every other.

Transformers made it practical to train much larger models on much more text. Within a year, two influential families appeared: BERT from Google, built to understand text, and GPT from OpenAI, built to generate it.

Predicting the next word

At the heart of a generative model is a simple task: given some text, predict what comes next. Text is split into pieces called tokens, often words or parts of words, and the model is trained on enormous amounts of text to predict the next token, over and over.

That simple task, repeated across a vast library of writing, forces the model to learn a great deal about grammar, facts, styles and reasoning, because all of those help predict what comes next. When you chat with an LLM, it is generating its reply one token at a time.

From prediction to assistant

A model trained only to predict text is not yet a helpful assistant. So pretrained models are fine-tuned: trained further on examples of good answers, and with methods such as reinforcement learning from human feedback, in which people rate responses, or constitutional AI, which trains a model against a set of written principles.

This second stage is what turns a text predictor into a chatbot that follows instructions, answers questions and declines some requests. It is also where much of the difference between one company's model and another's comes from.

A short history

GPT-2 drew wide attention in 2019, partly because OpenAI at first withheld the full model over fears it could be misused. GPT-3 followed in 2020, available only through an online interface for developers. In late 2022, ChatGPT, built on a fine-tuned model called GPT-3.5, brought LLMs to a mass audience almost overnight.

In 2023 GPT-4 was praised for its accuracy, and many models became multimodal, able to handle images as well as text. In 2024 OpenAI released o1, a reasoning model that works through long chains of thought before answering, an approach many other models have since adopted.

Hallucinations

The best-known weakness of LLMs is that they can produce fluent, plausible text that is simply wrong, or that does not match the source they were given. These errors are called hallucinations. A model can invent a reference, a quotation or a fact with exactly the same confidence as it states a true one.

This follows from how the models work: they are built to produce likely text, not to check it against the world. That is why it is important to verify anything that matters, and why our guide to how to evaluate a claim applies to chatbot answers as much as to anything else.

Do bigger models get smarter?

Researchers disagree about how LLMs gain their abilities. Some argue that new abilities appear suddenly once models reach a certain size. Others, including a study by Schaeffer and colleagues, argue that the abilities improve smoothly and predictably with scale, and only look sudden because of how they are measured.

The question matters for predicting what future models will be able to do, and how quickly. It is one of the open debates in a field that changes month by month.

Do they think?

LLMs are now good enough to pass versions of the famous Turing test, which asks whether a machine can pass for human in conversation, as we explain in the Turing test explained. That does not settle whether they understand anything, and researchers continue to argue about it.

What is clear is that they are powerful tools with real limits. Used well, they save time; trusted blindly, they spread mistakes. Our piece on AI in archaeology shows how to judge claims about what AI has achieved.