Polynomial Regression

The one idea: When a scatterplot curves, a straight line will systematically miss — underpredicting in the middle or at the ends, in a pattern you can see. Polynomial regression fixes this by adding powers of your input (x², x³...) as new features, then fits an ordinary straight-line regression on top of that expanded feature set. It's still "linear regression" mathematically — the curve comes from the features, not from a new kind of model.

⚠️ The trap: A high-degree polynomial can hit every single training point almost exactly — training R² near 1.0 — while making wild, physically absurd predictions the moment you step slightly outside the training range or even between training points. A polynomial fit that looks like a perfect line through your training scatterplot is a red flag, not a win — check test-set error, not train-set R², before trusting the fit.

When a straight line isn't enough

Plain linear regression assumes a constant slope: each unit of x moves y by the same amount everywhere. Plenty of real relationships don't work that way — diminishing returns on marketing spend, an exponential-then-flattening dose-response curve, a U-shaped relationship between age and error rate. If you fit a straight line to any of those, the residual plot (predicted vs. actual error) shows a clear curve instead of random scatter — the model's mistakes aren't random noise, they're a leftover pattern it failed to capture.

import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score

np.random.seed(0)
x = np.linspace(0, 10, 60)
# a genuinely curved relationship: y rises, peaks, and falls
y = 5 + 8*x…