2D Gradient Contour Surface
Saddle Point
Step: 0
Adam
SGD + Momentum
RMSprop
SGD
Adam
Active
Loss: 0.00
Oscillation: 0.00
⚠ Overshooting Minima
SGD + Momentum
Active
Loss: 0.00
Oscillation: 0.00
⚠ Momentum Explosion
RMSprop
Active
Loss: 0.00
Oscillation: 0.00
⚠ Path Divergence
SGD
Active
Loss: 0.00
Oscillation: 0.00
⚠ Stagnated in Ravine
Why "Throwing Adam At It" Fails
On anisotropic or non-stationary landscapes, Adam's second-moment estimation ($v_t$) can cause step scaling to overshoot narrow valleys or diverge when gradients shift faster than moments decay. Click anywhere on the canvas above to drop comparison trajectory probes!