Regression in Practice: GLMs and Beyond
Most researchers run regressions they could not defend under questioning: a line fit by habit, diagnostics skipped, a logistic bolted on when the outcome turns binary. This path does the linear model seriously, then crosses into generalised linear models, where links, deviance, counts, and mixed effects live, and ends where prediction meets honesty: regularisation, calibration, and knowing when the model is the wrong tool.
You can read an OLS fit as a projection, name which assumption broke from the diagnostic panels, tell an outlier from a leverage point from an influential one, choose a link and family for a binary or count outcome, read an odds ratio without overclaiming, catch overdispersion before it fakes your standard errors, say what a mixed-effects model shrinks and why, and state plainly when a coefficient is not a causal effect and when a problem does not deserve machine learning at all.
The line, done seriously
Least squares is a projection, and the hat matrix says how hard each observation pulls on its own fit; every diagnostic that follows is a reading of this one picture.
The Gauss-Markov assumptions are a checklist of failure modes, and each failure has a distinct cost: biased coefficients, faked standard errors, or both. Knowing which is which decides whether the fix is the model or the errors.
Outlier, leverage, and influence are three different properties that software defaults happily conflate; the dangerous point is the one with both leverage and residual, and Cook's distance is how you catch it.
Fanning residuals fake your standard errors and outliers bend the fit itself; GLS, sandwich errors, and bounded-influence estimators are the working repairs, each with a price you should be able to name.
Interactions are where regressions earn their keep and where main effects stop meaning what they seem to; AIC, BIC, and cross-validation are how you choose among candidate models without fooling yourself.
A coefficient is a conditional-expectation slope, not an effect; confounding and reverse causation produce the same slope as a true cause, and no sample size fixes that.
Beyond the line
The GLM is one machine with two dials, a link and a family, and every regression you have ever run is a setting of it; choosing the dials for the outcome you actually have is the whole move.
Logistic regression is the most used GLM after OLS, and the odds ratio is its most misread number; fitting one by IRLS and watching the deviance drop is how the machinery stops being a black box.
Poisson regression assumes the variance equals the mean, and real counts almost never oblige; overdispersion silently shrinks your standard errors, and negative binomial is the honest repair.
Raw residuals stop working the moment the variance depends on the mean; deviance and its residuals are the GLM's native diagnostics, the same discipline as the OLS panels in a new currency.
Grouped data breaks the independence assumption every fixed model relies on; random effects pool information across groups and shrink noisy group estimates toward the grand mean, which is exactly what you want and must be able to say.
Zero inflation, nonlinearity, and non-exponential-family outcomes are where the GLM's honesty runs out; knowing the named exits, GAMs, zero-inflated models, quantile regression, is what separates a practitioner from a formula user.
Prediction and honesty
Prediction and explanation are different games with different scoreboards, and ridge and lasso are the same linear model replayed as a loss machine; the ML chapter shows that machine at work on geoscience data, gradient descent, cross-entropy and all.
A model that ranks well can still lie about probabilities, and decisions ride on the probabilities; reliability curves and ECE are how you check, and recalibration is how you fix it without refitting.
The final regression skill is refusal: small samples, shifted deployment, high-stakes unexplained decisions, and data-quality bottlenecks are all cases where the honest answer is a simpler model or no model.