Definition
The context window is the maximum amount of text, measured in tokens, that a large language model can take into account in a single request. It includes the system prompt, your question, any documents you attach, the conversation so far and the answer the model is producing. Anything outside the context window does not exist for the model. It is the working memory of the AI, not its long-term knowledge.
In practice, the context window decides what you can do in one call. A small window forces you to summarise or split documents. A large window lets you paste an entire contract, a codebase or a year of support tickets and ask questions across all of it. Companies deal with limits through retrieval-augmented generation, which fetches only the relevant pieces, and through summaries that carry the important facts from one step to the next.
Frontier models in 2026 offer context windows of hundreds of thousands of tokens, with some reaching a million or more, enough for whole books. But a bigger window is not free: cost and latency grow with the amount of text, and models still attend unevenly to very long inputs. The misconception is that a larger context window makes retrieval unnecessary. For large, changing document sets, retrieval remains cheaper and more accurate.
In practice
A legal team drops a 300-page due diligence data room into a single request and asks for every clause on change of control. With a large context window the model reads it all; with a small one, the team would need a retrieval pipeline.
Why it matters
The context window sets the ceiling on what an AI can reason about in one go and drives the cost per request. It shapes whether a use case is trivial, needs engineering, or is not yet feasible.
Frequently asked questions
- What happens when the context window is full?
- The model cannot accept more text. Depending on the tool, it either returns an error or silently drops the oldest parts of the conversation, which is why long chats sometimes forget earlier instructions. Good applications summarise or retrieve to stay within the limit.
- Is a bigger context window always better?
- Not always. Larger inputs cost more, respond more slowly, and models can still miss details buried in very long text. For a single large document, a big window is convenient. For a growing knowledge base, retrieval-augmented generation is usually cheaper and more accurate.