Regression glossary

Clear, one-line definitions of the Regression terms used across the OgbonLab textbooks. Each entry links to the interactive sections where the idea is taught.

adjusted r-squared

R² penalised for the number of predictors; corrects the inflation of R² as more variables are added.

f-test

An ANOVA-style test for whether a group of coefficients are jointly zero; uses an F-distributed statistic.

See: t-tests, χ², F-tests done by hand

glm

Generalised linear model: g(E[Y | X]) = Xβ for a link function g and an exponential-family response distribution.

hat matrix

H = X(XᵀX)⁻¹Xᵀ; the projection matrix mapping y to its OLS fitted values ŷ = Hy.

heteroscedasticity

Non-constant residual variance Var(eᵢ | X) ≠ σ²; invalidates classical OLS standard errors but not point estimates.

See: Heteroscedasticity, GLS, and weighted regression

lasso

Penalised OLS using an L₁ penalty λ||β||₁; performs simultaneous shrinkage and variable selection.

leverage

hᵢᵢ, the diagonal of the hat matrix H = X(XᵀX)⁻¹Xᵀ; measures how much yᵢ pulls its own fitted value.

See: Diagnostics: residuals, leverage, influence

multicollinearity

Strong linear dependence among predictors; inflates SE(β̂) and destabilises coefficient estimates.

ols

Ordinary least squares: the estimator β̂ minimising Σ(yᵢ − xᵢᵀβ)²; closed form β̂ = (XᵀX)⁻¹Xᵀy.

poisson regression

A GLM with log link for count data; log(E[Y | X]) = Xβ, often with an offset for exposure.

quantile regression

Models conditional quantiles Q_τ(Y | X) rather than the mean; robust to outliers and skew.

See: Quantile regression and distribution-free CIs

r-squared

R² = 1 − SS_res/SS_tot; the fraction of variance in y explained by the model.

ridge regression

Penalised OLS: β̂ minimises ||y − Xβ||² + λ||β||²; shrinks coefficients toward 0, useful with collinearity.

robust regression

Estimation that downweights outliers (e.g., Huber M-estimator, Tukey biweight) to stabilise inference.

See: Robust regression

vif

Variance inflation factor: VIFⱼ = 1/(1 − Rⱼ²) where Rⱼ² regresses Xⱼ on the other predictors; VIF > 10 flags multicollinearity.