Calculus & Linear Algebra
- Sheldon Axler. Proof-first vector spaces, eigenvalues, spectral theorem.
- Michael Spivak. Rigorous single-variable calculus and real proofs.
- Gilbert Strang. Intuition, four fundamental subspaces, SVD.
Learn the math, not just the API
Understand why gradient descent works, watch it fail when the learning rate is wrong, and follow a hand-curated reading path through the classic mathematics that deep learning is built on.
Gradient descent repeats the update x ← x − η·f′(x). Pick a loss surface, choose a starting point and learning rate η, then step or run. Watch convergence, oscillation, escape from local minima, and outright divergence.
Ready. Press Run to optimize.
Try η = 1.05 on the convex bowl to see oscillating divergence, or start the double well at x₀ = 0.1 to see which basin captures the iterate. On the wavy surface, small η gets trapped in local minima while a well-chosen larger η can hop over ridges.
If f has L-Lipschitz gradients, then f(x−η∇f) ≤ f(x) − η(1−Lη/2)‖∇f‖². Choosing η ≤ 1/L guarantees every step decreases the loss. That inequality is the whole reason the algorithm works.
Convex + smooth gives O(1/k) error after k steps; strong convexity upgrades this to a geometric (linear) rate. These are provable limits, not folklore, and they come straight from the analysis in convex optimization texts below.
Too large η on curvature L makes updates overshoot and diverge; nonconvex surfaces admit local minima and saddle points. You can reproduce every one of these behaviors in the playground above.
A hand-curated path through the classics. Check items off as you finish them; progress is saved in your browser.
For the source convex bowl x squared, gradient is two x. At starting x four point two and learning rate point one, subtracting rate times gradient gives three point three six. Bars share eighty pixels per x unit. This educational browser computes the update directly, not through a training API. Those x values have loss seventeen point six four and eleven point two eight nine six. Bars share twenty pixels per loss unit. The update moves opposite the positive gradient and lowers loss for this example. Lower loss after one step is evidence for this specific surface and rate, not a universal convergence guarantee. For the x squared bowl, each update multiplies x by one minus twice the learning rate. Rates point one and point three give multipliers point eight and point four. Bars share four hundred pixels per multiplier magnitude. Larger rates may overshoot or diverge; other source surfaces have different curvature. Genuine native Step and Reset, full actual metric states, all canvas buffers, native resize and first-footer checks preserve the educational tool. Clipboard trace and automatic-run timing remain separately unverified; no native download exists.