Experimental Design glossary
Clear, one-line definitions of the Experimental Design terms used across the OgbonLab textbooks. Each entry links to the interactive sections where the idea is taught.
13 terms
- blocking
- Grouping experimental units into homogeneous blocks before randomising treatment within each block; controls nuisance variation.
- See: Blocking Well Logs
- file drawer
- Unpublished null results that languish in researchers' file drawers; a key driver of publication bias.
- harking
- Hypothesising After the Results are Known: presenting post-hoc findings as if predicted; a form of researcher degrees of freedom.
- open science framework
- A free public repository (osf.io) for preregistrations, data, code, and materials supporting transparent and reproducible research.
- p-hacking
- Searching for significance by trying many tests or transformations and reporting only what crosses α; inflates false positives.
- power analysis
- Calculation of the sample size required to detect an effect of a given size with specified power and significance level.
- preregistration
- Publishing analysis plans before observing data; locks in hypotheses, design, and tests to deter p-hacking.
- See: Preregistration and the garden of forking paths
- publication bias
- Systematic preference for statistically significant or novel results in published literature; distorts meta-analyses.
- randomisation
- Random assignment of units to treatments; ensures groups are statistically comparable in expectation and underlies causal claims.
- replication
- Repeating a study with new data and ideally independent investigators; the primary check on a finding's robustness.
- See: The replication crisis and what to actually do
- reproducibility crisis
- The widespread finding (post-2010) that many published effects in psychology, biomedicine, and economics do not replicate.
- sample size
- n, the number of observations collected; controls the precision of estimates and the power of tests.
- stratification
- Sampling or analysing within strata defined by covariates to improve precision and ensure balance.