Definition
The learning rate sets the base scale of parameter updates during optimization. In basic gradient descent, the update equals the negative gradient multiplied by the learning rate. A larger value moves farther along that direction; a smaller value moves less.
The effective step can differ from the configured number. Momentum, adaptive optimizers, gradient clipping, parameter scale, batch size, and distributed aggregation all alter the update that reaches a weight. A schedule may warm the learning rate up, decay it, restart it, or change it after a plateau.
If the learning rate is too high, loss may oscillate, jump over useful regions, or diverge. If it is too low, progress can be needlessly slow or stall in poorly conditioned regions. The best value often changes over a run, which is why a reported training setup should include both the starting value and its schedule.
Distinguish it from nearby terms
The learning rate is not data arrival rate, throughput, or examples per second. It is an optimizer hyperparameter. Adaptive methods may compute different effective steps for different parameters, but they still use a configured base learning rate.
Check your understanding
Training loss falls for several steps, then alternates between two large values without settling. Which learning-rate failure does that suggest, and what evidence would distinguish it from bad data or numerical overflow?