Bootstrap, deeper: parametric vs nonparametric

Part 8 — Resampling and nonparametrics

Learning objectives

Distinguish PARAMETRIC bootstrap (resample from a fitted parametric model) from NONPARAMETRIC bootstrap (resample from the empirical distribution)
Compute the PERCENTILE bootstrap CI and recognise its limitations under skewness or bias
Apply the BCa (bias-corrected and accelerated) bootstrap for second-order correct CIs
Recognise when bootstrap CONSISTENCY holds and when it fails (extreme quantiles, dependent data, parameters on the boundary)
Apply BLOCK BOOTSTRAP for time-series data and CLUSTER BOOTSTRAP for clustered data

Part 1 introduced the bootstrap as a way to get standard errors and CIs from any statistic computable from a sample. §8.1 develops the bootstrap into a comprehensive resampling framework: parametric vs nonparametric, the BCa correction for skewness and bias, and the consistency conditions that determine when the bootstrap works.

The bootstrap principle

You have data $y_1, \ldots, y_N \sim F$ and a statistic $\hat{\theta} = T(y)$ . The bootstrap principle: the unknown sampling distribution of $\hat{\theta}$ around $\theta$ can be approximated by the simulated distribution of $\hat{\theta}^$ $\hat{θ}^{*}$ around $\hat{\theta}$ , where the $\hat{\theta}^$ are computed from resamples. The resamples can come from two sources:

Nonparametric bootstrap (Efron 1979): resample with replacement from the empirical distribution $\hat{F}_N$ . Each bootstrap sample y* is N iid draws from y.
Parametric bootstrap: fit a parametric model $y_i \sim F_\theta$ giving estimate $\hat{\theta}$ ; resample from $F_{\hat{\theta}}$ .

The choice: parametric is more EFFICIENT (uses model structure) when the model is correct; nonparametric is more ROBUST (no model assumed) but loses some power. In practice, nonparametric is the default; reserve parametric for known-correct models or when N is too small for the empirical distribution to be a good estimate of F.

The percentile bootstrap CI

The simplest bootstrap CI: take B bootstrap samples, compute $\hat{\theta}^$ , and report the empirical α/2 and 1−α/2 quantiles of the bootstrap distribution as the CI. Symmetric, conceptually clean, but only first-order accurate (coverage error $O(1/\sqrt{N})$ ) and biased when the underlying distribution is skewed.

Studentized (pivotal) bootstrap CI

For each bootstrap sample, compute the studentized statistic $T^* = (\hat{\theta}^* - \hat{\theta})/\hat{\text{SE}}^$ . Report CI as $\hat{\theta} - \hat{\text{SE}} \cdot q_{1-\alpha/2}^$ $\hat{θ} - \hat{SE} \cdot q_{1 - α /2}^{*}$ to $\hat{\theta} - \hat{\text{SE}} \cdot q_{\alpha/2}^$ where $q^$ $q^{*}$ are quantiles of T*. SECOND-ORDER accurate ( $O(1/N)$ ) when valid.

BCa: bias-corrected and accelerated (Efron 1987)

The BCa method corrects for both BIAS in the bootstrap median and SKEWNESS via an acceleration parameter. The CI is

[\hat{\theta}^*_{(\alpha_1)}, \hat{\theta}^*_{(\alpha_2)}],

where the adjusted quantile levels are:

\alpha_1 = \Phi\left(\hat{z}_0 + \frac{\hat{z}_0 + z_{\alpha/2}}{1 - \hat{a} (\hat{z}_0 + z_{\alpha/2})}\right), \quad \alpha_2 = \Phi\left(\hat{z}_0 + \frac{\hat{z}_0 + z_{1-\alpha/2}}{1 - \hat{a} (\hat{z}_0 + z_{1-\alpha/2})}\right).

$\hat{z}$ is the bias correction. $\hat{a}$ is computed from the jackknife: $\hat{a} = \sum (\bar{\theta}$ {(\cdot)} - \hat{\theta}{(i)})^3 / (6 [\sum (\bar{\theta}{(\cdot)} - \hat{\theta}_{(i)})^2]^{3/2}) $\overset{a}{^} = \sum (\overset{ˉ}{θ}_{(\cdot)} - \hat{θ}_{(i)})^{3} / (6 [\sum (\overset{ˉ}{θ}_{(\cdot)} - \hat{θ}_{(i)})^{2}]^{3/2})$ . The result is a second-order correct CI that handles skewness and bias automatically.

The R package boot (Davison & Hinkley 1997) implements BCa as boot.ci(type = "bca"). Modern best practice for nonparametric CIs.

Bootstrap consistency

The bootstrap is CONSISTENT — its sampling distribution converges to the true sampling distribution — under mild conditions: T is a smooth functional of F, and E[|y_i|²] < ∞. The bootstrap FAILS when:

Extreme quantiles: the 99th percentile of a sample. The empirical estimate of an extreme quantile is unstable, and the bootstrap doesn't fix this.
Parameters on the boundary: e.g., bootstrap for variance components in mixed-effects models when the variance estimate is at zero.
Discontinuous functionals: e.g., max, min of a sample don't bootstrap consistently.
Heavy-tailed data with infinite variance: bootstrap fails when the underlying distribution's second moment doesn't exist.

Block bootstrap for time series

The vanilla bootstrap assumes IID data. For TIME SERIES or DEPENDENT data, resampling individual observations destroys the dependence structure. The BLOCK BOOTSTRAP (Künsch 1989) resamples CONTIGUOUS BLOCKS of length b: each bootstrap sample is a concatenation of randomly-chosen blocks from the original series. Block size b is the key tuning parameter; $b \sim N^{1/3}$ is asymptotically optimal under typical mixing conditions. Variants: stationary block bootstrap (Politis & Romano 1994), moving block bootstrap.

Cluster bootstrap

For CLUSTERED data (e.g., patients within hospitals, students within schools), resample WHOLE CLUSTERS rather than individual observations. This preserves the within-cluster correlation structure that the dependence creates. Cluster bootstrap is the standard tool for clustered-data SE estimation in econometrics and biostatistics.

Try it

Start with Normal data (default). All four CIs (Normal-z, parametric, nonparametric percentile, BCa) coincide. Each has width roughly 2 × 1.96 × SE(ȳ) ≈ 0.46. Bootstrap doesn't add much for Normal data — the parametric assumption is correct, so all methods agree.
Switch to "Skewed (lognormal)". The data is right-skewed. Look at the CI overlay panel: the parametric bootstrap CI is symmetric (it assumes Normal — wrong); the nonparametric and BCa CIs are ASYMMETRIC (longer right tail), reflecting the data's actual sampling distribution. Sample skewness is reported in the readout.
Stay on skewed. Bump N up to 200. The bootstrap CIs narrow as expected by 1/√N. The asymmetry persists — it's about the shape of the bootstrap distribution, not the precision.
Switch to "Bimodal mixture". Now the data has two modes (3 and 8) with 70/30 mixing. The bootstrap distributions are still concentrated on the sample mean (~4.5), but the histograms show slight bimodality in some resamples that include unusual proportions of each cluster.
Compare BCa to plain percentile: BCa typically shifts the CI slightly to handle bias and acceleration. The shift is small when N is large and the distribution is symmetric; bigger for small N or skewed data — exactly the cases where the correction matters.

A clinical trial reports a small treatment effect with N = 30. The data are right-skewed. Which CI method is most appropriate, and why?

What you now know

Bootstrap turns any statistic into a CI by resampling. Parametric bootstrap inherits model assumptions (efficient if correct); nonparametric is robust. BCa applies second-order corrections for bias and skewness; it's the modern best practice for nonparametric CIs. Bootstrap fails on extreme quantiles, boundary parameters, and IID-violations; block bootstrap rescues time series, cluster bootstrap rescues clustered data. §8.2 next: permutation tests, the canonical resampling-based hypothesis-testing tool.

References

Efron, B. (1979). "Bootstrap methods: Another look at the jackknife." Annals of Statistics 7(1), 1–26. (The original bootstrap.)
Efron, B. (1987). "Better bootstrap confidence intervals." JASA 82(397), 171–185. (BCa method.)
Davison, A.C., Hinkley, D.V. (1997). Bootstrap Methods and Their Application. Cambridge University Press. (The standard reference.)
Künsch, H.R. (1989). "The jackknife and the bootstrap for general stationary observations." Annals of Statistics 17(3), 1217–1241. (Block bootstrap.)
Politis, D.N., Romano, J.P. (1994). "The stationary bootstrap." JASA 89(428), 1303–1313.