Definition
A parameter is a numerical value learned during training that helps determine how a model transforms inputs into outputs. In a linear model, parameters include coefficients and an intercept. In a neural network, they usually include weights and biases stored in matrices or higher-dimensional tensors.
Training begins with initial parameter values, measures performance against an objective, and updates those values through an optimization procedure. Inference normally uses the learned values without changing them. Further training, fine-tuning, or some forms of adaptation can update them again.
When a model is described as having 7 billion parameters, the number usually counts individual learned scalar values. It is a rough measure of storage and computation, not a direct measure of capability. Parameter sharing, sparsity, numerical precision, mixture-of-experts routing, architecture, and training data can make equal counts behave very differently.
Distinguish it from nearby terms
- A hyperparameter is selected or tuned outside the ordinary learning update. Examples include learning rate, batch size, and some architecture choices.
- An activation is a value produced while processing a particular input. It is temporary model state, not usually a learned parameter.
- A checkpoint stores parameters and may also store optimizer state, metadata, or other training information.
- A feature is an input attribute or learned representation presented to a model. A parameter controls how the model uses it.
Operational significance
Parameter count is useful for estimating memory, transfer size, and some compute requirements. It does not reveal data quality, benchmark performance, context handling, calibration, or system reliability. When comparing models, check active parameters per inference, precision, architecture, task results, and deployment cost instead of ranking them by count alone.
Check your understanding
A 70-billion-parameter model can underperform a smaller model on a specific task. The larger count describes learned capacity and resource demands, not guaranteed fitness for the task.