Gradient descent and a neural net by hand

A loss function measures how wrong a model's predictions are. Its gradient with respect to a parameter tells you which direction increases the loss — so training moves the parameter the opposite way, a little at a time: param -= learning_rate * gradient. Repeat that over many examples and the loss goes down.

# fit y = w*x + b by gradient descent, no framework
w, b, lr = 0.0, 0.0, 0.01
for epoch in range(200):
    dw = sum(2 * (w * x + b - y) * x for x, y in data) / len(data)
    db = sum(2 * (w * x + b - y) for x, y in data) / len(data)
    w -= lr * dw
    b -= lr * db

This is literally a one-neuron network: it learns a line, y = wx + b, entirely by repeated gradient steps, no ML framework involved. Every larger neural net is the same idea — more parameters, more layers, the same core update rule. If you want to go deeper on this, Andrej Karpathy's "Neural Networks: Zero to Hero" builds up from exactly this point.