---
title: Training
description: "The process of adjusting a model's parameters to reduce a defined objective on training data, with the aim of producing useful behavior on new inputs."
canonical_url: 'https://darkfactory.dev/glossary/training'
markdown_url: 'https://darkfactory.dev/glossary/training.md'
collection: glossary
date_published: '2026-08-03T00:00:00-04:00'
date_modified: '2026-08-26T00:00:00-04:00'
---

# Training


## Definition

Training is the process of adjusting a model's parameters to reduce a defined objective on training data. A typical neural-network training step selects a batch of examples, runs them through the model, measures a loss, computes gradients, and uses an optimizer to update the parameters. Repeating this process produces a checkpoint that can be evaluated or used for inference.

Reducing training loss is the mechanism, not the final goal. The useful question is whether the learned behavior generalizes to new examples and operating conditions. A model can memorize its training data, optimize a poor proxy, or exploit leakage while appearing to improve against the training objective.

"Training" covers several stages. Pretraining learns broad patterns from a large corpus. Fine-tuning continues training for a narrower domain or behavior. Post-training is an umbrella for later methods such as supervised instruction tuning, preference optimization, and reinforcement learning. The boundaries vary across organizations, so name the stage and objective instead of treating training as one undifferentiated event.

## Distinguish it from nearby terms

- **Inference** uses a trained model to produce an output. It does not ordinarily update the model's parameters.
- **Prompting** changes the runtime input. It can change behavior without training the weights.
- **Evaluation** measures behavior on held-out examples or real tasks. Evaluation data should not silently become training data.
- **Online learning** updates a model as new data arrives. A live system is not doing online learning merely because its context changes between requests.
- **Retrieval-augmented generation** adds external information at inference time. It changes context, not the base model's parameters.

## Operational significance

Record the data version, objective, initialization, optimizer settings, random seeds, checkpoints, and evaluation protocol. Separate training, validation, and test data by the unit that can leak, such as customer, document, repository, or time period. Monitor subgroup and out-of-distribution performance because one aggregate loss can hide important failures.

Training also creates governance obligations. Data rights, personal information, harmful content, labor used for labeling, energy use, and the ability to reproduce or roll back a checkpoint belong in the training record.

## Check your understanding

A team adds three examples to a system prompt and sees better answers. That is in-context adaptation. If the team runs an optimizer that changes the stored model parameters, it is training.

## Also called

model training

## Related terms

- [Inference](https://darkfactory.dev/glossary/inference)
- [Training data](https://darkfactory.dev/glossary/training-data)
- [Objective function](https://darkfactory.dev/glossary/objective-function)

## Evidence and further reading

- [Google Machine Learning Glossary](https://developers.google.com/machine-learning/glossary/)
- [Deep Learning](https://www.deeplearningbook.org/)