ML Technical Interview Interactive Theory Lab

Standard Interview Question

"Derive and explain the Bias-Variance Tradeoff. How do model complexity and dataset size shift the sweet spot?"

Expected Answer Strategy

• High Bias (Underfitting): Model makes restrictive assumptions (e.g. degree 1 line fitting sine). Cannot capture true underlying pattern. High train & high test error.
• High Variance (Overfitting): Model fits sample noise (high degree polynomials oscillating wildly). Low train error but poor generalization.
• Irreducible Error (σ²): Noise in measurement/target that no model can eliminate.
• Dataset Scaling: Increasing sample size (N) lowers the variance curve, shifting the optimal complexity rightward.

Senior Follow-Up: Mitigations

To reduce variance: Bagging / Ensembling (Random Forests average uncorrelated estimators, dividing variance by ~M), L1/L2 regularization, or collecting more training samples. To reduce bias: Increase capacity, add interaction features, or boost (GBM).

"Why does L1 induce true sparsity while L2 only shrinks weights?"

Visual & Mathematical Defense

• Diamond Corners (L1): The L1 ball {w : |w₁| + |w₂| ≤ C} has sharp vertices located exactly on coordinate axes where wᵢ = 0.
• First Point of Contact: As loss ellipses expand from the OLS center, they tangentially touch the diamond vertex with very high probability, setting coefficients to strict zero.
• Spherical Smoothness (L2): The L2 ball {w : w₁² + w₂² ≤ C²} has no corners; gradients are continuous everywhere. The tangent point will almost never lie on an axis, shrinking weights smoothly without strict zeros.
• Subgradient View: L1 derivative is sgn(w) × λ (constant push towards 0). L2 derivative is 2λw (push diminishes as w → 0).

Elastic Net Follow-Up

When features are highly correlated (ρ > 0.8), Lasso randomly selects one feature. Elastic Net combines L1 + L2 penalties: L1 produces sparsity while L2 retains grouped feature selection.

"How do you troubleshoot a model that is not converging or oscillating?"

Candidate Defense Framework

• Too Small η: Sluggish convergence, gets trapped in plateau or shallow local minima; excessive training epochs required.
• Too Large η: Oscillates perpendicular to ravine walls or causes exploding gradients (∇L → ∞).
• Ill-Conditioned Curvature: In ravines (λ_max / λ_min >> 1), standard SGD bounces violently across steep walls instead of moving down the gentle ravine floor.
• Remedies: Momentum β damps cross-canyon oscillations while accelerating floor velocity. Adaptive optimizers (Adam, RMSProp) scale learning rates per coordinate by historical second moments.

"When do you optimize ROC-AUC vs PR-AUC in production?"

Interview Talking Points

• Severe Class Imbalance: Switch to PR curve. ROC curve uses FPR = FP / (FP + TN). When TN is in millions (e.g. ad click / fraud), FP can balloon by 10,000× without visibly altering FPR, making ROC deceptively optimistic.
• Precision Sensitivity: Precision directly penalizes FP because denominator is (TP + FP). PR curves reveal severe precision collapse in rare-event detection.
• Threshold Tuning: In medical diagnosis (cost of FN is lethal), choose low τ for high Recall. In spam filtering (cost of FP drops important emails), choose high τ for high Precision.