Gradient Descent Lab Memorization vs Generalization

Training Loss (Empirical)
Test Loss (Generalization)
Optimizer Trajectory (Adam/SGD)
Train Loss
0.000
Empirical empirical risk
Test Loss
0.000
Out-of-distribution loss
Generalization Gap
0.000
Test - Train gap
Gradient Norm ||g||
0.000
Local steepness
Weight Norm ||w||
0.000
Parameter magnitude
Optimization Status: Click canvas or hit Play to start optimization trajectory.

The optimization lab separates its synthetic basin from regularization

Read the explanation

The source wide basin is one half times the sum of squared coordinate offsets from one and negative point five. At fixed vertical center, horizontal offsets one and two give basin values point five and two. Bars share two hundred pixels per synthetic loss unit. This is a deliberately constructed landscape, not a measured neural-network loss surface. The source adds an L two penalty of one half times weight decay times x squared plus y squared. With squared coordinate magnitude five, decay point zero one adds point zero two five, while decay point one adds point two five. Bars share sixteen hundred pixels per synthetic penalty unit. Other landscape terms remain separate and the illustration is not a guarantee about generalization. The plain SGD branch sets coordinate change to negative learning rate times the computed gradient. A hypothetical gradient two gives step magnitude point one at learning rate point zero five and point two at point one. Bars share eighteen hundred pixels per coordinate-step magnitude. Direction is opposite the gradient; momentum and Adam use different branches. Original full history JSON, genuine reset, p five buffers and four-width first-footer placement remain checked.

Super generates helpful tools and automates fact-checking across the internet proactively. If you enjoyed this tool, build your own with Super and share it with a friend.