Glossary
Inference — Glossary
The process of running a trained model on new inputs to produce predictions or generations, usually without updating model weights.
Full definition
Inference is deployment-time computation: the model transforms inputs into outputs using learned parameters. For classifiers, inference yields scores or labels; for generative models, it includes decoding tokens or denoising steps for images.
Inference engineering focuses on latency, cost, batching, hardware acceleration, caching, and safety filters. Some systems support limited online learning, but most production inference keeps weights fixed between explicit retraining or fine-tuning jobs.