Gradient Descent
The loss landscape
Training moves the "ball" toward lower-loss regions, one step at a time.
In more detail
Imagine the loss as a function of all the parameters, a "landscape". The gradient points in the direction in which the loss increases fastest. To reduce the loss, the parameters take a small step in the opposite direction:
new value = old value − learning rate × gradient
The learning rate sets the step size: too small and training is slow; too large and it overshoots or becomes unstable.
Real training uses small random groups of examples (mini-batches, hence "stochastic" gradient descent), usually with adaptive optimizers such as Adam or AdamW and a schedule that changes the learning rate over time.
The ball rolling down a hill is a metaphor with limits. Real loss landscapes have billions of dimensions and do not look like a single smooth bowl.