mixle.ops module

Operations — the verbs that transform distributions, and how they change capabilities.

The missing third axis of mixle: alongside objects (distributions) and capabilities (what they support), the operations that move an object from one capability set to another. Each verb documents its capability signature — what it requires of its input and what the output gains — so “what can I do to a distribution, and what does it become?” has one home.

quantize(dist, bits) : any Distribution -> FiniteSupport · Enumerable · RankableByIndex truncate(dist, …) : Distribution -> Distribution (Enumerable preserved) condition(dist, observed) : Conditionable -> Distribution marginalize(dist, keep) : Marginalizable -> Distribution mixture(dists, w) : Distributions -> LatentStructured transform(dist, f) : Distribution + inv. f -> Distribution (Jacobian-corrected) tilt(dist, theta) : ExponentialFamily -> Distribution project(source, target) : Sampleable x Fittable -> Distribution (forward-KL M-projection) product_of_experts(dists) : tractable family -> Distribution (geometric/log-linear pool)

See docs/ARCHITECTURE.md and the operation table in docs/CAPABILITIES.md.

quantize(dist, bits=8, *, lo_q=1e-3, hi_q=1.0 - 1e-3, n_samples=200_000, seed=0)[source]

Discretize a continuous dist into a finite distribution over 2**bits bins.

Capability signature: any Distribution -> FiniteSupport · Enumerable · RankableByIndex. The result is a CategoricalDistribution over bin midpoints; the bin masses are exact (via the base cdf when available) or sampled otherwise. This is the concrete answer to “if I quantize a distribution, what does it become and how” — describe(quantize(g)) shows the gained capabilities.

Target measure. [lo_q, hi_q] brackets where the bins are placed, not which mass survives: the whole distribution is quantized, with mass below the bracket assigned to the first bin and mass above it to the last – the clipped image of dist under the bin map. The exact CDF route used to instead drop both tails and renormalize the interior to one, i.e. silently return the distribution conditional on the bracket, while the sampling fallback clipped its draws into the edge bins. The two advertised implementations therefore targeted different distributions: with lo_q=.25, hi_q=.75, bits=2 on a uniform, the CDF route returned four masses of .25 where the clipped answer is .375/.125/.125/.375. Both routes now agree on the clipped measure.

Parameters:
truncate(dist, *, allowed=None, forbidden=None)[source]

Restrict dist to (allowed) or away from (forbidden) a finite set; mass renormalizes.

Parameters:
condition(dist, observed)[source]

Condition dist on a subset of coordinates. Requires the Conditionable capability.

Parameters:
marginalize(dist, keep)[source]

Marginalize dist to the kept coordinates. Requires the Marginalizable capability.

Parameters:
mixture(dists, w=None)[source]

Weighted mixture of dists — a latent-variable model (LatentStructured).

Capability signature: Distributions -> LatentStructured. dists is materialized once (a one-shot iterator is consumed exactly once, not walked again for the weight count), must contain at least one component, and w — when given — must supply exactly one weight per component.

Raises:

  • CapabilityError – if dists is empty. A mixture of nothing is not a distribution, and the default uniform weight 1 / len(dists) used to compute it before anything checked, so mixture([]) reported an incidental ZeroDivisionError from the façade’s own arithmetic rather than the contract error this boundary declares (compare product_of_experts(), which already refuses an empty expert list here).

  • ValueError – if w has a different length than dists — caught at the façade, where the caller’s own two arguments are still in view, rather than inside the component layer.

Parameters:
transform(dist, f)[source]

Change of variables y = f(x) with a Jacobian-corrected density (f is a Transform).

Parameters:
tilt(dist, theta)[source]

Exponentially tilt dist by theta. Requires the ExponentialFamily capability.

Parameters:
project(source, target, *, n_samples=20_000, seed=0, max_its=50, init=None, delta=1.0e-6)[source]

Variationally project source onto the family of target (the sample-based M-projection).

Capability signature: Sampleable source x Fittable target -> Distribution. Draws n_samples from source and fits target to them by maximum likelihood. Works for any source exposing a sampler (a distribution, mixture, GP, or a trained neural model) onto any fittable target family, so e.g. a neural sequence model can be distilled onto an HMM.

This is the empirical M-projection, not the exact one. Maximum likelihood on the population would minimize the forward divergence KL(source || target_family) exactly; what runs here is maximum likelihood on one finite n_samples-sized Monte Carlo draw, optimized by an iterative fitter that is capped at max_its and, for latent-variable families, only guaranteed to reach a local optimum. The result therefore carries sampling error (of order 1/sqrt(n_samples), not estimated or reported here), optimization error, and whatever misspecification the target family imposes. It converges to the exact projection as n_samples grows, and nothing in the returned object certifies how close this particular fit got. Treat it as an approximation and validate it (e.g. score held-out source draws under the result) before claiming a KL guarantee.

target may be a distribution (its estimator() supplies the family) or an estimator directly. The returned model is a member of the target family; describe(project(...)) shows its capabilities.

init is the member of the target family EM starts from. Without it the fit starts from the estimator’s own random initialization at seed, which for a latent family means one restart at whichever optimum that draw reaches: a mixture-reduction benchmark read as “closed-form beats EM by 5x” was measuring a single EM start stuck at a degenerate optimum, and the same call warm started from the closed-form solution improved monotonically past it (P10-F03). Passing the closed-form projection, or any other good guess, is the usual way to use this.

delta is the fitter’s convergence tolerance, or None to remove the convergence stop so the fit may use all max_its iterations – what a budget-matched comparison between two projections wants, since a run that stopped early and one that used its whole budget are not comparable on wall-clock. None does not guarantee the whole budget runs: a rejected update (a non-improving or non-finite step) still ends the loop, and the fit then warns with how many of the max_its iterations ran. The mixture-reduction benchmark’s own delta=None projections stopped well short of their 50 this way (Q10-F03), so compare runs by fit_provenance().iterations rather than by the budget that was requested.

Parameters:
product_of_experts(dists, weights=None)[source]

Geometric (log-linear) pooling of densities — a Product of Experts.

Capability signature: tractable family -> Distribution. Multiplies the expert densities, p(x) ∝ ∏_k p_k(x)**w_k i.e. log p(x) = Σ_k w_k·log p_k(x) − log Z, with raw exponent weights w_k (default 1.0 each — not normalized to sum to one, since PoE weights are exponents). Only the cases with a tractable normalizer are constructed exactly; the general continuous case raises CapabilityError.

  • Categorical / finite shared support (the LLM-vocab fusion case): exact over the intersection of the weighted experts’ supports — new_pmap[x] ∝ ∏_k p_k(x)**w_k for x with positive mass under every expert carrying a nonzero exponent, then renormalized. Returns a CategoricalDistribution.

  • Gaussian (closed form): a Gaussian with precision-weighted combination 1/σ² = Σ_k w_k/σ_k² and μ = σ²·Σ_k w_k·μ_k/σ_k². Returns a GaussianDistribution.

A w_k = 0 expert contributes the constant p_k(x)**0 = 1 and so is not in the pool at all; it is dropped before the support intersection is taken, exactly as it drops out of the Gaussian route’s precision sum. (It used to be intersected first: an expert supported only on {"a"} at weight zero deleted every other label from a weight-one expert’s answer.)

Parameters:

  • dists (Any) – the experts to pool (an iterable of distributions, all of the same tractable kind).

  • weights (Any) – per-expert exponents w_k; None (default) uses 1.0 for each expert. Each must be a finite real number; negative exponents are permitted (a “divide by this expert” pool) but are the caller’s responsibility to keep proper.

Raises:

CapabilityError – if the experts have no shared finite support and are not all Gaussian — the PoE normalizer Z = ∫ ∏_k p_k(x)**w_k dx is then intractable in closed form — or if every exponent is zero, which leaves the constant function 1 rather than a distribution.