mixle.inference.multiple_testing module¶
Multiple-testing correction and evidence combination.
When you run many tests at once, the chance that some cross a fixed threshold by luck grows with the number of tests, so a raw 0.05 cutoff no longer means what it says. Two notions of “control” answer this, trading power for stringency:
Family-wise error rate (FWER) – the probability of any false rejection. Controlled by
bonferroni()(single-step, always valid) and the uniformly more powerful step-wiseholm()(always valid) andhochberg()(valid under independence / positive dependence).False discovery rate (FDR) – the expected fraction of rejections that are false. Less stringent, more power for screening many hypotheses.
benjamini_hochberg()(valid under independence / PRDS) andbenjamini_yekutieli()(valid under arbitrary dependence).
Each returns adjusted p-values (a.k.a. q-values for FDR) in the original order plus the rejection mask
at level alpha; adjust_pvalues() is the unified dispatcher.
For the complementary problem – combining evidence for the same hypothesis across independent
replications/strata – fisher_combine(), stouffer_combine() (optionally weighted), and
tippett_combine() give a single pooled p-value (lightweight fixed-effect meta-analysis).
- bonferroni(pvals, *, alpha=0.05)[source]
Bonferroni single-step FWER control: adjusted
p_i = min(1, m p_i).The simplest and most conservative correction; always valid regardless of dependence.
- holm(pvals, *, alpha=0.05)[source]
Holm step-down FWER control – uniformly more powerful than Bonferroni, equally valid.
Sorts p-values ascending and applies the running factor
m - kwith a cumulative maximum so the adjusted values stay monotone.
- hochberg(pvals, *, alpha=0.05)[source]
Hochberg step-up FWER control (valid under independence / positive dependence).
Same per-step factor as Holm but applied step-up (from the largest p-value down), giving more rejections; requires the independence/PRDS assumption that Holm does not.
- benjamini_hochberg(pvals, *, alpha=0.05)[source]
Benjamini–Hochberg FDR control; adjusted values are q-values.
Controls the expected false-discovery proportion at
alphaunder independence or positive regression dependence (PRDS). The standard choice for screening many hypotheses.
- benjamini_yekutieli(pvals, *, alpha=0.05)[source]
Benjamini–Yekutieli FDR control, valid under arbitrary dependence.
Like
benjamini_hochberg()but inflated by the harmonic factorc(m) = sum_{i=1}^m 1/i, so it holds for any dependence structure at the cost of power.
- adjust_pvalues(pvals, *, method='bh', alpha=0.05)[source]
Unified dispatcher over the correction methods.
- Parameters:
- Returns:
{'reject', 'pvals_adjusted', 'n_reject', 'alpha'}.- Return type:
- fisher_combine(pvals)[source]
Fisher’s method: combine independent p-values via
-2 sum log p ~ chi^2_{2k}.Sensitive to a few very small p-values. For combining evidence for the same hypothesis across independent tests.
- stouffer_combine(pvals, *, weights=None)[source]
Stouffer’s Z method: combine p-values on the z-scale, optionally weighted.
Z = sum w_i Phi^{-1}(1 - p_i) / sqrt(sum w_i^2). Weights let more-precise studies (e.g. larger samples) count more; equal weights recover the unweighted combination.