Models have zero real-world intuition. Explore how raw feature scale disparity, isolated leverage outliers, and careless imputation dismantle gradient descent, distance metrics, and production generalization.
Because Income ($100,000) is numerically 2,000× larger than Age (50), the gradient step and Euclidean distance treat a minuscule $500 raise as vastly more consequential than an entire 40-year career difference. Feature 1 is virtually ignored.
Distance-based models like KNN, SVM with RBF kernels, and gradient-descent algorithms (Logistic Regression, Neural Networks) rely on Euclidean norms: d = √((x₁-y₁)² + (x₂-y₂)²).
When Feature 2 spans $10,000 to $250,000 while Feature 1 spans 18 to 75, the distance metric becomes almost 100% dependent on Feature 2. Normalization or Z-score standardization scales both dimensions to identical variance, restoring geometric equilibrium.
Convex optimization algorithms optimize Mean Squared Error (MSE) or Log-Loss. Because errors are squared, a single point at coordinates (x = 10, y = 10,000) generates an astronomical penalty.
The optimizer tilts the entire hyperplane away from the underlying ground truth just to appease that single extreme outlier. Applying IQR winsorization caps leverage points before weights are fit.
Filling missing values with arbitrary zeros (e.g., zero for missing salary or missing age) shifts the expected value E[X] and creates multimodal spike artifacts that never exist in production.
Median imputation preserves monotonic rank order while resisting extreme outliers, maintaining consistent class distributions across training and production serving.