Glossary
Streaming responses — Glossary
Sending model output tokens to clients incrementally as they are generated.
Full definition
Streaming improves perceived latency for chat UIs. Servers should still enforce timeouts, partial-failure handling, and moderation on the assembled output when required.