Short answer
A context window is the maximum span of tokens a model can consider at once—including system prompts, history, retrieved docs, and the new reply.
Why it matters
Context length shapes architecture choices between RAG, summarization, and long-context models—and directly drives cost.
Article
Practice
Long pastes can push out earlier instructions; keep important rules near the top and summarize history.
Examples
Contract review session
A lawyer’s assistant can fit a long agreement in-window, but the best prompts point the model to specific clauses and ask for excerpted quotes to verify.
Coding repo map
An IDE assistant retrieves only the auth module and failing test instead of pasting the entire monorepo into one request.
FAQ
Does the context window include the model’s answer?
Usually yes—output tokens consume part of the available context budget. Check provider accounting rules for each model.
Why did the assistant forget what I said earlier?
Earlier turns may have dropped out of the active window, or the product summarized them lossily. External memory features differ by app.
Is a larger context window always better?
It can help for long documents, but costs rise and irrelevant tokens can hurt quality. Prefer focused context and retrieval.
Sources
-
Anthropic docs
AnthropicOfficial model documentation describing context limits and usage.
-
OpenAI platform model docs
OpenAIOfficial model pages listing context sizes and pricing units.
-
Google AI Gemini API docs
GoogleOfficial documentation for Gemini models including long-context features.