Definition
Regression estimates a numeric response or the relationship between a response and one or more inputs. A house-price model, demand forecast, and delivery-time estimate are common machine-learning examples. The result may be a point estimate, an interval, a quantile, or a full predictive distribution.
The model does not have to be a straight line. Linear regression, decision-tree ensembles, neural networks, and many other model families can perform regression. What makes the task regression in the usual machine-learning sense is the numeric target and the objective used to estimate it.
The loss function defines which errors training emphasizes. Mean squared error penalizes large misses strongly. Mean absolute error is less sensitive to outliers. Quantile loss can estimate an upper or lower conditional quantile when planning for asymmetric risk matters more than predicting the average.
Where the name came from
Francis Galton used "regression" in his 1886 paper on hereditary stature. He observed that relatives of unusually tall or short people tended, on average, to be closer to the population mean. The statistical term comes from that specific idea of regression toward the mean. Modern regression methods estimate many kinds of relationships and do not require that pattern.
Distinguish it from nearby terms
- Classification predicts a category or probabilities over categories. Regression usually estimates a quantity on a numeric scale.
- Logistic regression is historically named as regression because it models log odds or probabilities, though it is commonly used for classification.
- Correlation summarizes association. Regression specifies how a response changes with inputs and can support prediction, but neither one alone establishes causation.
- Regression to the mean is a sampling phenomenon in repeated measurements. It is not the same thing as fitting a regression model.
Operational significance
Specify the target unit, forecast horizon, acceptable error, and cost of overprediction versus underprediction. Report performance across the range of values, since one average metric can hide severe misses for rare or high-value cases. Test on future or otherwise held-out data when the deployment setting involves change over time.
Check your understanding
A staffing model predicts call volume. A five-call error may be harmless during a quiet hour and costly during a surge, so the evaluation should expose scale, time, and asymmetric business consequences.