mixle.experimental.pac_bayes module

PAC-Bayes certificates over an explicit finite hypothesis space.

The posterior and prior accepted here are probability distributions over fixed predictors/hypotheses—not observation-density mixture components. The prior must be selected independently of the certified sample; the posterior may depend on that sample. Callers provide one bounded loss per hypothesis and sample, and an explicit PACBayesAssumptions receipt. The certificate uses the finite-space KL(Q || P) and the McAllester bounded-loss inequality.

gaussian_kl remains available only as a mathematical primitive for genuine Gaussian distributions over parameters. It is not used by certify_generalization() and does not convert fitted observation distributions into a hypothesis posterior.

class PACBayesAssumptions(hypothesis_space_id, prior_id, sample_iid, hypotheses_fixed_before_sample, prior_fixed_before_sample, loss_fixed_before_sample, loss_lower=0.0, loss_upper=1.0)

Bases: object

Caller-attested premises required by the theorem.

Parameters:

• hypothesis_space_id (str)

• prior_id (str)

• sample_iid (bool)

• hypotheses_fixed_before_sample (bool)

• prior_fixed_before_sample (bool)

• loss_fixed_before_sample (bool)

• loss_lower (float)

• loss_upper (float)

class GeneralizationCertificate(empirical_loss, empirical_loss_by_hypothesis, kl, hypothesis_kl_terms, n, delta, raw_bound, bound, vacuous, posterior, prior, theorem, assumptions)

Bases: object

A theorem-matched finite-hypothesis PAC-Bayes certificate.

Parameters:

• empirical_loss (float)

• empirical_loss_by_hypothesis (tuple[float, ...])

• kl (float)

• hypothesis_kl_terms (tuple[float, ...])

• n (int)

• delta (float)

• raw_bound (float)

• bound (float)

• vacuous (bool)

• posterior (tuple[float, ...])

• prior (tuple[float, ...])

• theorem (str)

• assumptions (PACBayesAssumptions)

largest_kl_term()

Index of the largest signed summand in the exact finite-space KL decomposition.

Return type:

int

gaussian_kl(mu_q, s2_q, mu_p, s2_p)

KL(N(mu_q, s2_q) || N(mu_p, s2_p)) for genuine parameter laws.

Parameters:

• mu_q (float)

• s2_q (float)

• mu_p (float)

• s2_p (float)

Return type:

float

categorical_kl(posterior, prior)

Finite-hypothesis KL(Q || P) and its exact per-hypothesis summands.

Parameters:

• posterior (Any)

• prior (Any)

Return type:

tuple[float, tuple[float, …]]

mcallester_bound(empirical_loss, kl, n, *, delta=0.05)

Return the raw McAllester upper bound for a loss in [0, 1].

Parameters:

• empirical_loss (float)

• kl (float)

• n (int)

• delta (float)

Return type:

float

certify_generalization(losses, posterior, prior, *, assumptions, delta=0.05)

Certify the Gibbs risk of posterior over a fixed finite hypothesis class.

Parameters:

• losses (Any) – Matrix with shape (n_hypotheses, n_samples) and values in [0, 1].

• posterior (Any) – Sample-dependent probability vector Q over the hypothesis rows.

• prior (Any) – Data-independent probability vector P over the same rows.

• assumptions (PACBayesAssumptions) – Explicit theorem-premise receipt; any false premise fails closed.

• delta (float) – Failure probability in (0, 1).

Return type:

GeneralizationCertificate