Everything the model uses to answer must fit inside the context window: system instructions, your messages, earlier replies, uploaded files and the new output. Anything that does not fit is cut or summarised. A bigger window allows longer documents and conversations, but not unlimited memory or perfect recall.
How it works
Language models read text as tokens, chunks of characters averaging roughly three-quarters of an English word. A context window is the maximum number of tokens the model can process in one pass. Early large language models handled a few thousand tokens. Leading models now commonly offer windows of hundreds of thousands of tokens, and some reach a million or more, which can hold several long books. Inside the window, the transformer's attention mechanism lets each token relate to every other token, which is why the model can link a question to a detail far back in a document. Attention cost grows quickly with length, so long contexts are slower and more expensive, and API pricing is usually per token. In chat apps, the whole conversation is resent with each turn; when it exceeds the window, older messages are dropped or summarised. Research has also found that models can miss details buried in the middle of very long inputs, a pattern sometimes called 'lost in the middle'.
Why it matters
The context window determines what a model can work with at once. For creators, a large window lets you paste a full transcript and ask for chapters, clips and posts in one go. Professionals can compare long contracts or reports without splitting them. Businesses building on APIs must balance window size against cost and speed. Knowing the limit explains why a long chat 'forgets' early instructions and when to start fresh or use retrieval instead.
Where you will see it
- Summarising a full podcast transcript into chapters and show notes
- Comparing two long contracts clause by clause
- Asking questions across an entire codebase or set of reports
- Keeping a long brainstorming session's instructions in view
Limits and common misconceptions
- A larger window is not perfect memory; models can overlook details in very long inputs.
- Filling the window raises cost and response time, especially through APIs.
- The context window resets per conversation; it is different from saved memory features or training.
How it differs from related terms
- Context window vs Token Tokens are the units of text; the context window is how many of them the model can handle at once.
- Context window vs RAG (retrieval-augmented generation) RAG fetches only the most relevant passages into the context window, instead of relying on fitting everything in.
Tools that use it
Frequently asked questions
What happens when you exceed the context window?
The model cannot see the excess. In chat apps, older messages are usually dropped or summarised, so the model may forget early instructions. Through an API, an over-long request may be rejected with an error or truncated. Starting a new chat with a short summary often helps.
How many words fit in a context window?
It depends on the model and the text. As a rough rule for English, one token averages about three-quarters of a word, so 100,000 tokens is around 75,000 words. Code, numbers and other languages can use more tokens per word.
Is the context window the same as memory?
No. The context window is short-term working space for one conversation or request. Memory features in some chat apps save selected facts separately and add them back into the context later. The model's general knowledge comes from training, which is a different thing again.
Related terms
- Chatbot Software that holds a conversation in natural language, by text or voice, to answer questions, complete tasks or provide information.
- Computer vision The field of AI that enables computers to interpret images and video, identifying objects, people, text, movement and scenes.
- Copilot An AI assistant embedded inside another app that uses what you are working on as context to suggest, draft or complete tasks.
- Deep learning Machine learning that trains neural networks with many stacked layers to learn patterns directly from raw data such as text, pixels or sound.
- Deepfake Synthetic video, image or audio, made with AI, that realistically shows a real person saying or doing something they never said or did.