Predict first. Then inspect how one learning rate turns the gradient into convergence, a shrinking zig-zag, or escape.
This tool teaches eight exact gradient updates on loss x squared plus four y squared. The gradient is two x, eight y. Starting at four, three with learning rate zero point one, each update multiplies x by zero point eight and y by zero point two. Both multipliers are positive and below one. The blue x curve and yellow y curve begin at four and three on the same coordinate scale. Advance through the eight source updates. Both coordinates stay positive and approach zero. Each vertex is one exact source state. The moving teaching pointers connect neighboring states; those connecting positions are illustrative interpolation, not additional optimizer steps. At step eight, x is about zero point six seven one one and y is zero point zero zero zero zero zero seven six eight. Loss falls from fifty two to about zero point four five zero four. Positive contracting multipliers explain smooth convergence on this specific quadratic. Now hold the same loss, starting coordinates, and eight updates, but choose learning rate zero point two. The source x multiplier is zero point six; the y multiplier is minus zero point six. Both magnitudes are below one. A negative multiplier changes the sign without preventing contraction. The two coordinate tracks again start at four and three. Advance the exact eight updates. The yellow y vertices alternate above and below the zero line. Their distance from zero shrinks at every step. Blue x remains positive and shrinks. The lines connect discrete states for explanation; they are not a continuous time solution. At step eight, both signs are positive: x is about zero point zero six seven two and y about zero point zero five zero four. The final loss is about zero point zero one four seven. The source calls this stable zig zag convergence: sign changes and convergence can coexist. For the final contrast, learning rate zero point three gives x multiplier zero point four and y multiplier minus one point four. The x coordinate still contracts. The y multiplier has magnitude greater than one, so its sign flips while its magnitude grows. The vertical axis now explicitly expands to minus forty five through forty five to show those larger source states. Advance the same eight exact updates. The yellow y vertices cross zero and move farther away. The blue x vertices shrink toward zero on this shared expanded scale. This is the source fixed rate recurrence, not a neural network benchmark or a physical trajectory. At step eight, x is about zero point zero zero two six two and y about forty four point two seven three seven. Loss grows from fifty two to about seven thousand eight hundred forty point six. Source stability requires both multiplier magnitudes below one. Zero point three violates that condition in y; the expanding sign changes explain divergence.