mixle.stats.latent.sparse_mixture module

Sparse mixture scoring with a CERTIFIED tail bound – the DSA ‘top-k + bound the rest’ idea.

DeepSeek Sparse Attention scores all candidates with a cheap indexer, keeps the top-k, computes the expensive thing only on those, and accepts a (bounded) error. The mixture analogue: each component has a cheap, x-independent upper bound log w_k + sup_x log p_k(x) (weight times peak density). Rank by it, score the exact log w_k + log p_k(x) for only the top max_components, and bound the dropped tail by the sum of the remaining upper bounds. That yields a certified bracket [lower, upper] provably containing the true log p(x) – unlike DSA, the error is certified, not just hoped small.

The bound is valid only where every contributing component’s density is BOUNDED (a finite peak): the design review’s scoping. Families with an unbounded density (Gamma shape<1, Beta with a<1 or b<1, …) return None from log_density_sup(), and scoring falls back to exact (no certification) for those.

log_density_sup(dist)[source]

Maximum log p(x) of a single distribution, or None if unbounded / unknown.

A finite value certifies the component can contribute at most this much, which is what makes the dropped-tail bound sound. None (unbounded density or an unrecognized family) forces the exact path.

Parameters:

dist (Any)

Return type:

float | None

class SparseScore(lower, upper, exact, n_scored)[source]

Bases: object

A certified bracket on a mixture’s log p(x) from scoring only the top components.

lower <= log p(x) <= upper always holds; exact is True when the bracket collapsed (all components scored, or max_components covered the support). n_scored components were exactly evaluated. When exact, lower == upper == log p(x).

Parameters:
lower: float
upper: float
exact: bool
n_scored: int
sparse_mixture_score(mixture, x, max_components)[source]

Score mixture.log_density(x) exactly on the top max_components and certify the rest.

Ranks components by the cheap upper bound log w_k + sup_k (x-independent), scores the exact contribution of only the top ones, and bounds the dropped tail by the remaining upper bounds. Returns a SparseScore bracket. If any positive-weight component’s density is unbounded (log_density_sup is None), falls back to exact full scoring (lower == upper, no speedup).

Parameters:
Return type:

SparseScore

collapse_identical(mixture)[source]

Merge components that are identical (same family + parameters) by summing their weights – EXACT.

A fitted huge mixture often carries duplicate components; pooling exact duplicates leaves log p(x) unchanged while shrinking K. Identity is keyed on the component’s string form (which encodes its parameters). ‘Blend a mixture of closed forms’ with zero approximation.

Parameters:

mixture (Any)

Return type:

Any

collapse_gaussian_mixture(mixture, max_components)[source]

Reduce an all-Gaussian mixture to <= max_components by moment-matching the nearest pair.

Greedily merges the two closest components (by mean) into the single Gaussian preserving their combined weight, mean, and variance, until max_components remain. Approximate (it widens), but it preserves the OVERALL mixture mean and variance exactly – the analytic ‘collapse a huge mixture of closed forms’ that lets scoring/sampling scale.

Parameters:

  • mixture (Any)

  • max_components (int)

Return type:

Any