Bayesian Statistics glossary

Clear, one-line definitions of the Bayesian Statistics terms used across the OgbonLab textbooks. Each entry links to the interactive sections where the idea is taught.

bayes factor

P(data | M₁) / P(data | M₀); the ratio of marginal likelihoods, measuring evidence for one model over another.

See: Model comparison: Bayes factors and WAIC

bic

Bayesian Information Criterion: −2·log L + k·log n; approximates marginal likelihood and penalises model complexity.

conjugate prior

A prior whose posterior lies in the same family, e.g., Beta-Binomial, Gamma-Poisson, Normal-Normal.

See: Conjugate priors and analytic posteriors

credible interval

A Bayesian interval [L, U] with P(θ ∈ [L, U] | data) = 1 − α under the posterior; not the same as a frequentist CI.

empirical bayes

Estimates a prior from the data and then performs Bayesian updating; pragmatic shortcut to fully Bayesian inference.

gibbs sampling

An MCMC scheme that updates each parameter by sampling from its full conditional given the others.

See: Gibbs sampling on a 2D example

hamiltonian monte carlo

An MCMC method using gradient-driven Hamiltonian dynamics for efficient sampling in high-dimensional continuous parameter spaces.

See: Hamiltonian Monte Carlo intuition

hierarchical model

A multi-level Bayesian model in which group-level parameters are drawn from a shared population distribution.

See: Mixed-effects and hierarchical models intro

map

Maximum a posteriori estimator: the value θ̂ that maximises the posterior p(θ | data); MLE plus a prior penalty.

marginal likelihood

P(data) = ∫ P(data|θ)·P(θ) dθ; the normalizing constant in Bayes' rule and the basis of Bayes factors.

mcmc

Markov chain Monte Carlo: simulation methods that draw correlated samples from a posterior by constructing a Markov chain with that stationary distribution.

metropolis-hastings

An MCMC algorithm that proposes moves from a candidate distribution and accepts with probability min(1, α) to target a stationary distribution.

See: Metropolis-Hastings by hand

posterior

The distribution of a parameter given the data, proportional to likelihood × prior; the target of Bayesian inference.

See: Posterior-predictive checks, Conjugate priors and analytic posteriors

prior

A probability distribution expressing belief about a parameter before observing data; updated to a posterior via Bayes.