Feed the network
Train it — real gradient descent
| x₁ | x₂ | target | prediction | verdict |
|---|
Neurons & weights
A neuron is embarrassingly simple: it multiplies each input by a weight, adds a bias, then squashes the result through an activation function:
Weights are the network's memory — everything it "knows" lives in these numbers. This little network has just 37 parameters (weights + biases). GPT-class models have hundreds of billions, but the neuron math is the same.
Activation functions
Without a nonlinearity, stacked layers collapse into one big linear function — no curve-fitting power. The classics:
ReLU dominates deep learning because its gradient doesn't vanish for positive z — try it above and compare how fast the loss falls (you may need a lower η; ReLU can be twitchy on tiny nets).
Training = backpropagation
Each Train step does exactly this, no magic: 1) forward pass on all 4 XOR examples; 2) measure error with mean-squared-error loss L = ½(ŷ−y)²; 3) use the chain rule to compute ∂L/∂w for every weight — walking the error backwards layer by layer; 4) nudge every weight downhill:
XOR is the famous test case: it's not linearly separable, so a network needs a hidden layer to solve it — the 1969 observation that stalled neural nets until backprop revived them in 1986.