Quantile regression and distribution-free CIs

OLS regression models the CONDITIONAL MEAN $E[Y \mid X]$. But the mean isn't always the most informative summary — when the conditional distribution is skewed or its scale varies with X, the mean misses important structure. QUANTILE REGRESSION (Koenker & Bassett 1978) directly models conditional QUANTILES — the median, the 10th percentile, the 90th percentile, etc. — by minimising an asymmetric loss function.

The check loss

For a chosen quantile $\tau \in (0, 1)$, the τ-th quantile regression solves

The loss function $\rho_\tau$ is the ASYMMETRIC CHECK FUNCTION: positive residuals are weighted by $\tau$, negative residuals by $1 - \tau$. For $\tau = 0.5$, the loss is the absolute residual, and the solution is the conditional MEDIAN. For $\tau = 0.1$, larger positive residuals are heavily penalized — the fit pulls down, becoming the 10th percentile. The choice of τ picks which percentile we're estimating.

Why quantile regression matters

• Robust to outliers: τ = 0.5 (median) is robust; ALL quantile regressions are robust to outliers in y.
• Distribution-free: no assumption about the residual distribution. Works for skewed, heavy-tailed, multimodal residuals.
• Reveals heteroscedasticity: if quantile slopes (β_{1,τ} across τ) differ, the conditional distribution's SHAPE varies with X. OLS shows only the central trend; QR exposes the full conditional structure.
• Application-specific: in finance, Value-at-Risk is a quantile estimator. In medicine, growth charts use percentile curves. In policy analysis, the conditional 10th percentile reveals "worst case" outcomes that the mean ignores.

Computation

The check-loss minimization is a LINEAR PROGRAM. Modern QR software uses:

• Simplex methods: classical linear programming, exact solution.
• Interior-point methods: scale better for large N.
• Iterative reweighted least squares: pseudo-weights based on residual sign; converges in 20-50 iterations.

R's quantreg::rq() (Koenker) is the gold-standard implementation. Python: statsmodels.QuantReg or scikit-learn via gradient boosting trees with quantile loss.

Standard errors and inference

QR coefficients have asymptotic Normality under regularity conditions, with covariance that depends on the conditional density of residuals at zero. Estimating that density is itself nonparametric, so standard SE estimates can be unstable. The standard practical fix: BOOTSTRAP the SEs. Resample observations with replacement, refit QR, repeat B times. The bootstrap variance is robust and easy to compute.

Multiple quantile curves

The full picture of $F(y \mid x)$ emerges from fitting QR at MANY quantiles $\tau \in {0.05, 0.10, \ldots, 0.95}$. The resulting fan of quantile curves displays the entire conditional distribution as a function of x — far richer than the single OLS mean. R's quantreg automates this.

Censored quantile regression

For censored outcomes (e.g., survival times, top-coded incomes), Powell (1986) developed censored QR. The objective generalises to: minimize $\sum \rho_\tau(\min(y_i, c_i) - \min(x_i^T \beta, c_i))$. Useful in econometrics for top-coded wages, in survival analysis for right-censored times.

Distribution-free CIs from QR

The conditional 5th and 95th quantiles, computed via QR at τ = 0.05 and 0.95, give a distribution-free 90% PREDICTION INTERVAL: the band between the two quantile curves contains 90% of new outcomes (conditional on x). No distributional assumption required. Compare to the OLS prediction interval, which assumes Normal residuals.

Quantile regression vs OLS: when to use which

• OLS: when residuals are roughly homoscedastic Normal, mean is the parameter of interest, and N is moderate.
• QR (median, τ = 0.5): robust regression default; immune to outliers in y; competitive with OLS even under Normal data.
• QR at extreme τ: when you care about tail outcomes (VaR, worst-case rates, top earners).
• Multiple QR: for heteroscedasticity diagnosis and shape-of-distribution reporting.

Try it

• Default: N = 120, heteroscedasticity = 0.4. OLS mean line (red, dashed) tracks the central trend. The three QR lines (τ = 0.1, 0.5, 0.9) fan out as x grows — wider spread at higher x — because variance grows with x. OLS doesn't show this.
• Drag hetero down to 0.0 (homoscedastic). The three QR lines become PARALLEL, with constant gaps between them. No fanning, no heteroscedasticity.
• Drag hetero up to 1.2 (strong heteroscedasticity). The fanning is dramatic. At x = 3, the gap between the 10th and 90th percentile is enormous; at x = -3, it's tiny.
• Smaller N (e.g., 30) gives noisier QR slopes. Re-sample several times to see the variability. QR statistical noise is higher than OLS at small N for the same reason that quantile statistics are noisier than mean statistics.
• Compare OLS slope (red) to QR median slope (green): they should be similar when the data are symmetric. Under skewed conditional distributions, they diverge — OLS is pulled by extreme values; QR median is not.

A financial analyst wants to estimate Value-at-Risk for a portfolio at the 1% level: the threshold below which only 1% of daily returns fall. Why is quantile regression the natural tool?

What you now know

Quantile regression (Koenker-Bassett 1978) estimates conditional quantiles via asymmetric check loss. Robust to outliers, distribution-free, naturally reveals heteroscedasticity by fitting many τ. Computed via linear programming or IRLS. Inference via bootstrap. Applications: VaR, growth charts, censored data, distribution-free prediction intervals. Part 8 closes here. Part 9 next: machine learning for researchers — when ML helps and when it misleads.

Mark section complete →

References

• Koenker, R., Bassett, G. (1978). "Regression quantiles." Econometrica 46(1), 33–50. (The original.)
• Koenker, R. (2005). Quantile Regression. Cambridge University Press. (The definitive monograph.)
• Powell, J.L. (1986). "Censored regression quantiles." J. Econometrics 32(1), 143–155.
• Yu, K., Lu, Z., Stander, J. (2003). "Quantile regression: applications and current research areas." The Statistician 52(3), 331–350.
• Hao, L., Naiman, D.Q. (2007). Quantile Regression. Sage. (Applied undergraduate-level introduction.)