mixle.stats.compute.posterior module

The Posterior hierarchy: a uniform, samplable object for any model-derived distribution.

Posterior is the shared base contract – inference produces posteriors; you draw from them through one interface. Every realization answers the same questions where they are defined:

sample(rng)        -> a single draw
samples(n, rng)    -> n draws (loop by default; vectorized override where cheaper)
mean() / mode()    -> the posterior mean / MAP configuration
marginals()        -> per-component marginals (e.g. the EM E-step responsibilities)
entropy()          -> H[q]                       (the ELBO entropy term)
interval(level)    -> a credible interval

This base lives in the compute layer next to the sampler contracts (DistributionSampler / ConditionalSampler in mixle.stats.compute.pdist) so both mixle.stats and mixle.inference can build on it without a layering inversion. The richer realizations that need inference machinery – parameter posteriors (conjugate / MCMC) and the posterior-predictive, plus the posterior(model, ...) factory – live in mixle.inference.posterior.

LatentPosterior is the latent q(z | x) subtype implemented here: each latent model handles its hidden variables implicitly inside EM (the E-step returns raw responsibility arrays); LatentPosterior makes q(z | x) a single object – exact for mixtures/HMMs, mean-field for LDA/VMP – so the EM E-step (marginals), latent sampling (sample), and the ELBO entropy term are methods on it. Mean-field realizations additionally provide expected_complete_ll(dist) / update(dist) / elbo(dist); for exact posteriors those are not needed.

class Posterior

Bases: ABC

A model-derived distribution exposing one uniform interface for draws and summaries.

Only sample() (a single draw) is required. samples() loops it by default; vectorized subtypes override it. mean / mode / marginals / entropy / interval raise NotImplementedError unless a subtype defines them, so each realization implements exactly the summaries that are meaningful for it.

abstractmethod sample(rng=None)

Draw a single sample from the posterior (rng is a seed, RandomState, or None).

Parameters:

rng (Any)

Return type:

Any

samples(n, rng=None)

Draw n samples; loops sample() by default (override for a vectorized draw).

Parameters:

• n (int)

• rng (Any)

Return type:

Any

mean()

The posterior mean E[.] (not defined for every realization).

Return type:

Any

mode()

The maximum-a-posteriori configuration (not defined for every realization).

Return type:

Any

marginals()

Per-component marginals – e.g. the EM E-step responsibilities (not always defined).

Return type:

Any

entropy()

The entropy H[q] (not always defined).

Return type:

Any

interval(level=0.9)

A central credible interval at the given level (not always defined).

Parameters:

level (float)

Return type:

Any

class LatentPosterior

Bases: Posterior

The posterior q(z | x) over a model’s latent variables (exact or mean-field).

abstractmethod marginals()

Per-latent marginal responsibilities – the quantity the EM M-step consumes.

Return type:

Any

abstractmethod sample(rng=None)

Draw the latent variables z ~ q(z | x) (rng is a seed, RandomState, or None).

Parameters:

rng (Any)

Return type:

Any

abstractmethod mode()

The maximum-a-posteriori latent configuration.

Return type:

Any

abstractmethod entropy()

The entropy H[q] (per latent / per observation).

Return type:

Any

class CategoricalLatentPosterior(responsibilities, support=None)

Bases: LatentPosterior

Independent categorical latents q(z) = prod_i Cat(z_i; r_i).

The exact posterior for a finite mixture’s component labels (and the per-token topic factor of an LDA document). responsibilities is the row-stochastic (N, K) matrix r_ik = q(z_i = k | x_i); support maps column k to its latent label (default 0..K-1).

Parameters:

• responsibilities (ndarray)

• support (Any)

marginals()

The (N, K) responsibility matrix.

Return type:

ndarray

sample(rng=None)

Draw one latent label per observation; returns an (N,) array of support labels.

Parameters:

rng (Any)

Return type:

ndarray

mode()

The most-probable latent label per observation, (N,).

Return type:

ndarray

entropy()

Per-observation entropy -sum_k r_ik log r_ik, (N,).

Return type:

ndarray

class MarkovChainLatentPosterior(log_pi, log_A, log_b)

Bases: LatentPosterior

Chain-structured latents q(z_1..z_T | x) for an HMM – exact, via forward-backward.

Built from the log initial distribution log_pi (K,), the log transition matrix log_A (K, K) (row j -> column k), and the per-position emission log-likelihoods log_b (T, K). The latents are coupled (a Markov chain), so:

marginals() -> the (T, K) forward-backward smoothing probabilities q(z_t = k | x) sample(rng) -> a full state path (T,) by forward-filter / backward-sample (FFBS) mode() -> the Viterbi (max-product) path (T,) entropy() -> the exact scalar chain entropy H[q(z_1..z_T | x)]

Parameters:

• log_pi (ndarray)

• log_A (ndarray)

• log_b (ndarray)

log_likelihood()

The sequence log-likelihood log p(x) (the forward normalizer).

Return type:

float

marginals()

The (T, K) smoothing probabilities q(z_t = k | x).

Return type:

ndarray

sample(rng=None)

Draw a state path z ~ q(z | x) via FFBS; returns (T,) state indices.

Parameters:

rng (Any)

Return type:

ndarray

mode()

The Viterbi (max-product) MAP path (T,).

Return type:

ndarray

entropy()

Exact scalar chain entropy via the FFBS factorization q = q(z_T) prod_t q(z_t|z_{t+1}).

Return type:

float

class MeanFieldLDAPosterior(gamma, phi, counts)

Bases: LatentPosterior

Mean-field variational posterior for one LDA document: q(theta, z) = Dir(theta; gamma) prod_n Cat(z_n; phi_n).

The Blei-Ng-Jordan variational factorization made into an object instead of loose gamma/phi arrays. gamma (K,) is the document’s variational Dirichlet parameter (q(theta)); phi (W, K) the per-distinct-word topic responsibilities (q(z_n), rows sum to 1); counts (W,) the word counts. Note the latents are heterogeneous (continuous theta + discrete z), so sample returns the pair (theta, z) and entropy is a scalar.

Parameters:

• gamma (ndarray)

• phi (ndarray)

• counts (ndarray)

topic_proportions()

The mean document-topic distribution E_q[theta] = gamma / sum(gamma) (K,).

Return type:

ndarray

marginals()

The (W, K) per-distinct-word topic responsibilities q(z_n).

Return type:

ndarray

sample(rng=None)

Draw the full latent (theta, z): theta ~ Dir(gamma) and per-token topics z from phi.

Parameters:

rng (Any)

Return type:

tuple[ndarray, ndarray]

mode()

The MAP topic per distinct word, argmax_k phi_wk (W,).

Return type:

ndarray

entropy()

Mean-field entropy H[q(theta)] + sum_w count_w H[Cat(phi_w)] (scalar).

Return type:

float