Definition
An optimizer converts gradients and accumulated training state into updates to model parameters.
Distinguish it from nearby terms
The loss defines what to improve; backpropagation finds gradients; the optimizer determines the update rule.
Check your understanding
Name the state the optimizer retains and how it changes a raw gradient.