Probability glossary

Clear, one-line definitions of the Probability terms used across the OgbonLab textbooks. Each entry links to the interactive sections where the idea is taught.

57 terms

addition principle: If A and B are disjoint events or choices, the number of ways to pick from A or B is |A| + |B|.
bayes theorem: P(A|B) = P(B|A)P(A)/P(B); inverts a conditional probability using the prior P(A) and the marginal P(B).
bernoulli: A distribution on {0, 1} with P(X = 1) = p; mean p, variance p(1 − p); the single coin flip.
beta: Beta(α, β): a continuous distribution on [0, 1] used to model proportions; the conjugate prior for the Bernoulli/binomial.
binomial: Bin(n, p): the number of successes in n independent Bernoulli(p) trials; mean np, variance np(1−p).; See: Poisson and negative-binomial counts
central limit theorem: For iid Xᵢ with finite variance σ², √n (X̄ − μ) converges in distribution to N(0, σ²).; See: The central limit theorem, Limit Theorems and the Central Limit Theorem
characteristic function: φ(t) = E[e^{itX}]; always exists, uniquely identifies the distribution, and underlies central limit proofs.; See: Moment-generating and characteristic functions
chi-squared: χ²(k): the distribution of Z₁² + ... + Z_k² for iid standard normals; arises in variance and goodness-of-fit tests.
combination: An unordered selection of k objects from n, counted by C(n,k) = n!/(k!(n-k)!).; See: Attribute combination: RGB blending and classification
complement rule: For an event A in a probability space, P(A^c) = 1 - P(A); the probability A does not occur.
conditional probability: P(A|B) = P(A ∩ B)/P(B), defined when P(B) > 0; the probability of A given that B has occurred.; See: Conditional Probability and Independence
continuous rv: A random variable whose CDF is absolutely continuous; described by a probability density function on ℝ.
convergence in distribution: Xₙ ⇒ X if F_n(x) → F(x) at every continuity point of F; the mode of convergence in the CLT.
convergence in probability: Xₙ →ᵖ X if P(|Xₙ − X| > ε) → 0 for every ε > 0; weaker than almost-sure convergence.
correlation: ρ = Cov(X,Y) / (σ_X·σ_Y) ∈ [−1, 1]; a scale-free measure of linear association.; See: Spatial Correlation and the Variogram
covariance: Cov(X,Y) = E[(X − μ_X)(Y − μ_Y)]; positive when X and Y co-vary, zero under independence.; See: Covariance, correlogram, and variogram, three views of the same thing
discrete: Taking values in a countable set (often ℕ or ℤ) rather than a continuum; e.g. a discrete random variable has a probability mass function.
discrete rv: A random variable taking values in a countable set, fully described by its probability mass function.
event: A measurable subset of the sample space Ω; the collection of events forms a σ-algebra on which the probability measure is defined.
expected value: E[X] = Σ x·P(X=x) (discrete) or ∫ x·f(x)dx (continuous); the long-run average of a random variable.
exponential: Exp(λ): a continuous waiting-time distribution with density λe^{−λx} for x ≥ 0; memoryless.; See: Exponential Functions, Polar and Exponential Form
f distribution: F(d₁, d₂): the distribution of (V₁/d₁)/(V₂/d₂) for independent chi-squareds; underlies the F-test in ANOVA and regression.
gamma: Gamma(k, θ): a flexible continuous distribution on (0, ∞) with density x^{k−1}e^{−x/θ}/(θ^k Γ(k)); generalises the exponential.
gaussian: Synonym for the normal distribution N(μ, σ²); arises as the limit in the central limit theorem.; See: Truncated Gaussian Simulation, Beam / Gaussian-beam migration
geometric: The number of independent Bernoulli(p) trials until the first success; PMF (1−p)^{k−1}·p for k = 1, 2, ...; See: Geometric Series, Anisotropy: range, sill, geometric vs zonal
iid: Independent and identically distributed: a sequence X₁, X₂, ... drawn independently from the same distribution.
inclusion-exclusion: |A ∪ B| = |A| + |B| - |A ∩ B|, generalizing to |⋃A_i| = Σ|A_i| - Σ|A_i ∩ A_j| + ...; corrects multiple counting.
independence: Events A and B are independent when P(A ∩ B) = P(A)·P(B); knowing one tells you nothing about the other.; See: Conditional Probability and Independence
independent: Two events A and B are independent when P(A ∩ B) = P(A)P(B); random variables X, Y are independent when their joint distribution factors.
interval-additivity: The property of a probability measure on ℝ that P([a,c]) = P([a,b]) + P((b,c]) for a < b < c; a special case of countable additivity.
joint distribution: The probability distribution describing two or more random variables together, specifying P(X=x, Y=y) or f(x, y).
kolmogorov axioms: The 1933 foundational axioms, non-negativity, normalization P(Ω)=1, and σ-additivity, that define a probability measure.
kurtosis: E[(X − μ)⁴] / σ⁴; measures tail-heaviness, equal to 3 for the normal distribution.
law of large numbers: For i.i.d. random variables with mean μ, the sample average (X_1+...+X_n)/n converges to μ as n → ∞.; See: The law of large numbers
law of total probability: If {B_i} partitions the sample space, P(A) = Σ P(A|B_i)P(B_i); decomposes A's probability over a partition.
marginal: A distribution obtained by summing or integrating a joint distribution over one or more of its variables.; See: Joint, conditional, marginal
mgf: Moment generating function M(t) = E[e^{tX}]; its derivatives at 0 yield the moments of X when M exists in a neighborhood of 0.
moment: The k-th moment of X is E[X^k]; the k-th central moment is E[(X − μ)^k], measuring shape around the mean.; See: Method of moments, Expectations and moments
multinomial: Generalises the binomial to k categories; counts outcomes in n iid trials over a categorical distribution.
multiplication principle: If a procedure has a steps with n_i choices at step i, the total number of outcomes is n_1 · n_2 · ... · n_a.
multivariate normal: A joint distribution on ℝᵏ with density determined by mean vector μ and positive-definite covariance matrix Σ.
mutual independence: Events A_1, ..., A_n are mutually independent when P(A_{i_1} ∩ ... ∩ A_{i_k}) = P(A_{i_1})···P(A_{i_k}) for every subset; stronger than pairwise independence.
negative binomial: The number of trials (or failures) until r successes in iid Bernoulli(p) trials; generalises the geometric.
normal: N(μ, σ²): the bell-shaped distribution on ℝ with density (1/√(2πσ²))·exp(−(x−μ)²/(2σ²)).; See: Normal-score transform, The Normal-Score Transform
pdf: Probability density function f(x) of a continuous RV; ∫ f(x) dx over an interval gives the probability of landing there.
pmf: Probability mass function: p(x) = P(X = x) for a discrete random variable; sums to 1 over its support.
poisson: Poisson(λ): counts of rare events in a fixed interval; PMF e^{−λ}λ^k/k!; mean = variance = λ.; See: Poisson and negative-binomial counts
probability axioms: Kolmogorov's three axioms: P(A) ≥ 0, P(Ω) = 1, and countable additivity for disjoint events.
random variable: A measurable function X: Ω → ℝ from a probability space; assigns a numerical outcome to each elementary event.; See: Random variables & noise, Random variables and distributions
sample space: The set Ω of all possible outcomes of a random experiment; the universe on which the probability measure is defined.; See: Probability: Sample Spaces and Events
signed-area: The integral ∫f(x)dx interpreted as area above the x-axis counted positive and area below counted negative.
skewness: E[(X − μ)³] / σ³; a dimensionless measure of asymmetry, zero for symmetric distributions.
standard normal: N(0, 1): the normal with mean 0 and variance 1; any normal X = μ + σZ where Z is standard normal.
student t: t(ν): the distribution of Z/√(V/ν) for Z ∼ N(0,1) and V ∼ χ²(ν); heavier-tailed than normal at small ν.
uniform: U(a, b): a continuous distribution with constant density 1/(b−a) on [a, b]; mean (a+b)/2.; See: Uniform Convergence and Its Consequences
union bound: P(A_1 ∪ ... ∪ A_n) ≤ P(A_1) + ... + P(A_n); a simple but widely used upper bound for the probability of a union.
variance: Var(X) = E[(X - E[X])²] = E[X²] - (E[X])²; measures the spread of a random variable around its mean.; See: Calibration of the kriging variance, Random Variables, Expectation, and Variance