Definition
The degree to which stated probabilities or confidence levels correspond to observed frequencies of correctness.
Distinguish it from nearby terms
Calibration measures whether confidence is statistically meaningful; accuracy measures how often predictions are correct.
Check your understanding
A confident natural-language rationale is not a calibrated probability.