Capstone: Meta-analysis with publication-bias correction

The third research capstone walks an end-to-end meta-analysis: synthesise K study-level effect estimates, characterise between-study heterogeneity, diagnose and correct for publication-bias-driven funnel-plot asymmetry. This is the workhorse evidence-synthesis tool of Cochrane reviews, evidence-based medicine, and policy-research syntheses worldwide.

The data-generating model

Random-effects meta-analysis models each study's true effect as a draw from a Normal population:

μ is the population-mean effect (the "synthesis"). τ² is the between-study variance, the heterogeneity. SE_k is the within-study sampling error, typically inversely proportional to study sample size √n_k.

Fixed-effect pool: inverse-variance weighting

The classical pool assumes τ² = 0 (all studies estimate the SAME effect):

Larger studies (small SE) get more weight. Fixed-effect is appropriate when heterogeneity is low (I² < 25 %) and inappropriate otherwise, because it understates uncertainty.

Random-effects pool (DerSimonian-Laird)

Add the between-study variance τ̂² to each weight:

τ̂² is estimated via DerSimonian-Laird from Cochran's Q statistic. RE has wider CIs than FE because it accounts for between-study variation, closer to honest under heterogeneity.

Heterogeneity statistics

• Cochran's Q: $Q = \sum_k w_k (\hat{\theta}$k - \hat{\mu}{FE})^2. Distributed χ²(K-1) under no heterogeneity.
• τ̂²: between-study variance, in the same units as θ. Crucial for interpretation: τ̂ ≈ 0.5 means the typical study's true effect deviates by 0.5 from the mean.
• I² = max(0, (Q - (K-1)) / Q): fraction of total variance due to between-study heterogeneity. < 25 % low; 25-50 % moderate; > 50 % substantial; > 75 % considerable.

Publication bias and funnel-plot asymmetry

Studies showing statistically significant (often positive) results are more likely to be PUBLISHED than null or negative findings. Result: the meta-analytic pool over-represents extreme positive results, biasing μ̂ upward.

Visual check: the FUNNEL PLOT (θ̂_k vs SE_k). Under no selection, points should be symmetrically distributed around μ. Under selection, the lower-left corner (small studies with null / negative results) is empty.

Formal test: EGGER'S REGRESSION (Egger et al. 1997). Regress θ̂_k / SE_k on 1/SE_k. The intercept is the asymmetry parameter, significant ≠ 0 → bias suspected.

Trim-and-fill correction (Duval & Tweedie 2000)

An iterative procedure:

• Compute the pooled estimate.
• Identify the most extreme (right-asymmetric) studies whose removal would symmetrise the funnel.
• Recompute the pooled estimate from the symmetric subset.
• MIRROR the trimmed studies onto the left side around the new pooled estimate (the "fill" step).
• Report the final pooled estimate from the full augmented set as a publication-bias-corrected estimate.

Trim-and-fill is HEURISTIC, not gold-standard. It assumes that the asymmetry is due to selection (not heterogeneity) and that suppressed studies mirror published ones. Modern alternatives: selection models (Vevea & Hedges 1995), p-curve (Simonsohn et al.), p-uniform.

Try it

• Defaults (μ = 0.3, τ = 0.15, K = 20, selZ = 0): no selection. FE and RE both recover μ̂ ≈ 0.3. The funnel plot is symmetric. Egger's test fails to reject.
• Crank selection threshold to z* = 2.0. Now only studies with |θ̂/SE| ≥ 2 survive. Watch the bias: μ̂_FE jumps to ≈ 0.5 (massively over-estimated). Funnel becomes asymmetric, small-SE studies (top of funnel) keep both positive and negative; large-SE studies (bottom) keep only positive. Egger's rejects.
• Now toggle trim-and-fill on (it's automatic in the readout). It imputes 4-6 missing studies (red dots on funnel) and corrects μ̂ downward, often back to ≈ 0.35-0.40, much closer to truth than the uncorrected 0.5.
• Crank τ to 0.3 (high heterogeneity, I² ≈ 50 %). RE's CI widens but its centre is correct. FE's CI is too narrow, RE is appropriate here.
• Reduce K from 20 to 5. Watch all CIs widen and bias-test power drop. Meta-analysis on few studies is unreliable; small K should trigger conservative bias-correction.

A meta-analyst reports I² = 85%, substantial heterogeneity. Should they report the random-effects pooled estimate as the headline number, or do something else? Justify in two sentences.

What you now know

You can run an end-to-end meta-analysis: synthesise K study-level estimates, characterise heterogeneity, diagnose publication bias, and correct for it. The RE estimator is the workhorse for heterogeneous evidence; trim-and-fill is a useful heuristic correction for funnel asymmetry. Reporting standards (PRISMA) require explicit disclosure of all these analyses; sensitivity analyses across different bias-correction methods are best practice.

Mark capstone complete →

References

• DerSimonian, R., Laird, N. (1986). "Meta-analysis in clinical trials." Controlled Clinical Trials 7(3), 177-188.
• Egger, M., Smith, G.D., Schneider, M., Minder, C. (1997). "Bias in meta-analysis detected by a simple, graphical test." BMJ 315, 629-634.
• Duval, S., Tweedie, R. (2000). "Trim and fill: a simple funnel-plot-based method." Biometrics 56(2), 455-463.
• Higgins, J.P.T., Thompson, S.G. (2002). "Quantifying heterogeneity in a meta-analysis." Statistics in Medicine 21(11), 1539-1558.
• Borenstein, M., Hedges, L.V., Higgins, J.P.T., Rothstein, H.R. (2009). Introduction to Meta-Analysis. Wiley.
• Vevea, J.L., Hedges, L.V. (1995). "A general linear model for estimating effect size in the presence of publication bias." Psychometrika 60(3), 419-435.