Planned chapter | Dan Hollick
Gradient descent and backpropagation.
Training works by measuring error, tracing responsibility backward through the model and nudging each parameter toward a better result.
The source chapter is still planned. This route preserves the collection and offers an original conceptual preview.
Training works by measuring error, tracing responsibility backward through the model and nudging each parameter toward a better result. The apparent simplicity comes from a set of carefully chosen representations, transformations and physical assumptions working together.
A loss function
One number estimates how far the model’s output is from the desired answer.
This is one part of a longer chain: prediction becomes loss becomes gradients becomes updated weights. The useful abstraction hides the physical work, but the underlying constraints still shape the software built above it.
Backpropagation
The chain rule distributes that error backward through every differentiable operation.
The implementation is full of compromises. Precision, speed, storage and energy rarely improve together, so practical systems choose the errors people are least likely to notice.
Gradient descent
An optimiser applies small updates while balancing speed, noise and stability.
Once this layer is visible, familiar design conventions stop looking arbitrary. They are accumulated responses to the capabilities and limits of the machinery below.
A visual study based on the original chapter. Text is condensed and rewritten.