Glossary
Latency budget — Glossary
The maximum acceptable response time for an AI feature in a product.
Full definition
Latency budgets drive model choice, caching, streaming UX, and whether heavy reasoning models are acceptable on a path.
Glossary
The maximum acceptable response time for an AI feature in a product.
Latency budgets drive model choice, caching, streaming UX, and whether heavy reasoning models are acceptable on a path.