---
title: Inference
description: 'Executing a trained or otherwise specified model on input to produce scores, predictions, decisions, or generated content.'
canonical_url: 'https://darkfactory.dev/glossary/inference'
markdown_url: 'https://darkfactory.dev/glossary/inference.md'
collection: glossary
date_published: '2026-08-03T00:00:00-04:00'
date_modified: '2026-08-26T00:00:00-04:00'
---

# Inference


## Definition

Inference is the execution of a trained or otherwise specified model on input to produce an output. The output may be a score, label, prediction, recommendation, decision, embedding, or generated sequence.

For a classifier, one inference call may map a feature vector to class scores. For an autoregressive language model, inference repeatedly predicts a distribution for the next token, selects or samples a token, appends it to the context, and continues until a stopping condition. One user request may therefore contain many token-level model executions and several application-level calls.

Inference can run online for an interactive request, in batches over stored records, on a device, or through a remote service. The model may be fixed while the runtime changes quantization, batching, kernels, hardware, or numerical precision.

## Distinguish it from nearby terms

- **Training** adjusts model parameters against data and an objective. Inference uses the resulting model to produce outputs.
- **Serving** is the infrastructure that receives requests, schedules work, executes inference, and returns results. It may add caching, routing, batching, and fallbacks.
- **Logical inference** derives conclusions under a formal reasoning system. Model inference is the broader operational term and may be statistical or generative.
- An **agent run** may combine many inference calls with retrieval, tool actions, state changes, and verification.

## Operational significance

Measure latency, throughput, memory, hardware, precision, batch policy, decoding settings, and model version. A change in the inference backend can alter cost and, through numerical or scheduling differences, may alter behavior. Reproducibility requires more than a model name.

## Check your understanding

A coding agent reads a task, calls a model, searches the repository, calls the model again, edits files, and requests review. Each model execution is inference. The full sequence is an agent run.

## Also called

model inference, serving

## Related factory areas

- [Model selection, routing & budgets](https://darkfactory.dev/factory/model-routing-budgets)

## Evidence and further reading

- [Google Machine Learning Glossary](https://developers.google.com/machine-learning/glossary/)