From linear to generalised: link and family

Linear regression assumes the response Y is conditionally Normal with constant variance. This breaks for binary outcomes (Y in {0,1}), count outcomes (Y in 0,1,2,...), positive-skewed continuous outcomes (lifetimes, costs), and proportions. Generalised Linear Models extend the OLS framework to these cases, keeping the linear-in-parameters predictor while replacing Normality with a more general exponential-family distribution.

The three GLM components

• Random component: Y_i has a distribution in the exponential family, Normal, Binomial, Poisson, Gamma, Inverse Gaussian, Negative Binomial, Multinomial, etc. With mean $\mu_i = E[Y_i|X_i]$ and variance $\mathrm{Var}(Y_i|X_i) = \phi V(\mu_i)$ where $V$ is the variance function and $\phi$ the dispersion parameter.
• Systematic component: linear predictor $\eta_i = \mathbf{x}_i^T \boldsymbol{\beta}$.
• Link function: invertible $g$ such that $g(\mu_i) = \eta_i$. The link CONSTRAINS μ to a valid range.

Canonical links

• Normal: identity link, $\mu = \eta$. Recovers OLS.
• Binomial (binary, proportions): logit link, $\log!\frac{\mu}{1-\mu} = \eta$. Logistic regression.
• Poisson (counts): log link, $\log \mu = \eta$. Poisson regression.
• Gamma (positive continuous): inverse link, $1/\mu = \eta$ (more common in practice: log link).

The "canonical" link is the one for which the linear predictor equals the natural parameter of the exponential-family distribution. Using the canonical link gives the nicest properties, sufficient statistics in closed form, IRLS converges quickly, but ANY invertible link with the right range can work.

IRLS fitting

GLMs are fitted by Iteratively Reweighted Least Squares: at each iteration, form a "working response" $z_i$ and a working weight $w_i$, then run weighted least squares. Converges quadratically for canonical links. Implemented in R's glm() and Python's statsmodels GLM.

Visualising the four canonical links

The same linear predictor η = β₀ + β₁·x flows through four different inverse-link functions to produce μ in four very different ranges. Move the sliders, note that:

• Normal · identity permits any μ, including negative values (which are absurd for counts, proportions, or lifetimes).
• Binomial · logit squashes η ∈ ℝ into (0, 1), large positive η ⇒ μ → 1, large negative η ⇒ μ → 0.
• Poisson · log maps η ∈ ℝ to (0, ∞), small η changes have multiplicative effects on μ.
• Gamma · inverse is only well-defined for η > 0; outside that range μ is undefined.

Try it

• Set β₁ = 0 and slide β₀ from -3 to +3. Watch how the Binomial μ moves from near 0 to near 1 along an S-curve, while the Poisson μ scales from ≈ 0.05 to ≈ 20. Same η range, totally different μ behaviour.
• Set β₀ = -2, β₁ = 0.5. For which x-values is the Gamma panel UNDEFINED? Convince yourself that the inverse link is unsuitable when η can cross zero, log-link Gamma is the workhorse alternative.
• Set β₁ = 1.5. In the Binomial panel, how steep is the sigmoid near η = 0? Now flatten it: β₁ = 0.2. The same one-unit change in x produces a much smaller change in μ when β₁ is small (logistic regression's "marginal effect" depends on where you are on the curve).
• Crank n samples up to 200 and look at the Poisson panel: do the dots span the full y-range or hug the lower portion? At η = -2, μ ≈ 0.14, almost every sample is 0. This is why Poisson regression on rare events needs lots of data.
• Try β₀ = 3, β₁ = 0. The Normal panel happily predicts μ = 3 for all x. The Gamma panel correctly gives μ = 1/3 ≈ 0.33. The Binomial panel saturates at μ ≈ 0.95. The Poisson panel gives μ ≈ 20. Four totally different stories from the same η.

If your response is a measured proportion (e.g., germination rate across 200 trials), and you accidentally fit OLS instead of logistic regression, what TWO concrete things go wrong? Hint: think about predicted values that exceed (0, 1), and variance that is wrongly assumed constant when it actually depends on μ.

What you now know

GLMs extend OLS to non-Normal responses via the family + link framework. The link is not a stylistic choice, it CONSTRAINS μ to a valid range and ties the variance structure to the mean. §5.2-5.4 cover the two most important non-Normal cases (logistic and Poisson) and the GLM-specific diagnostics. §5.5 introduces mixed-effects extensions for clustered data. §5.6 closes Part 5 with honest scope: what GLM cannot do.

Mark section complete →

References

• Nelder, J.A., Wedderburn, R.W.M. (1972). "Generalized linear models." J. Roy. Stat. Soc. A 135(3), 370-384. (The foundational GLM paper.)
• McCullagh, P., Nelder, J.A. (1989). Generalized Linear Models, 2nd ed. Chapman & Hall. (The canonical book.)
• Agresti, A. (2015). Foundations of Linear and Generalized Linear Models. Wiley.
• Dobson, A.J., Barnett, A.G. (2018). An Introduction to Generalized Linear Models, 4th ed. Chapman & Hall.
• Wood, S.N. (2017). Generalized Additive Models: An Introduction with R, 2nd ed.