Definition
A hyperparameter is a configuration choice outside ordinary model-weight learning. Examples include learning rate, batch size, weight-decay strength, network depth, LoRA rank, and the number of training epochs. Some are fixed before a run. Others follow a configured schedule or are selected by a tuning system.
The boundary depends on the procedure. A value is a model parameter when the training objective learns it through the ordinary update rule. The same kind of value may be a hyperparameter in another system if a person, search process, or outer optimization loop selects it. Runtime controls such as sampling temperature are often called inference parameters or inference hyperparameters because they affect generation without changing learned weights.
Hyperparameter search consumes evaluation data. Repeatedly selecting settings against the test set turns that test set into part of development and weakens the final performance claim. Sound tuning separates training data, validation data used for selection, and a final test set reserved for evaluation.
Distinguish it from nearby terms
Model parameters are learned state such as weights. Hyperparameters configure the architecture, training, or inference procedure. Ordinary software configuration may control logging or deployment without affecting the learned function, so not every configuration value is a machine-learning hyperparameter.
Check your understanding
You run 200 learning-rate and batch-size combinations, choose the best result on the test set, and report that same score. Which values are hyperparameters, and why is the reported test score now biased?