Glossary
Tokens per second — Glossary
A throughput metric for how quickly a system generates tokens during inference.
Also known as: tok/s
Full definition
Tokens/sec depends on hardware, batching, precision, and implementation kernels. It is a systems metric distinct from answer quality.