mixle.stats.compute.pdist module

Defines abstract classes for SequenceEncodableProbabilityDistribution, SequenceEncodableStatisticAccumulator, ProbabilityDistribution, StatisticAccumulator, StatisticAccumulatorFactory, DataSequenceEncoder, ParameterEstimator, ConditionalSampler, and DistributionSampler for classes of the mixle.stats.

class DensitySemantics(*values)

Bases: Enum

What a distribution’s log_density returns relative to the true log-density.

The default contract is EXACT; override density_semantics() on models whose log_density is a variational bound or an approximation, so callers can tell an exact likelihood from one (e.g. LDA’s per-document ELBO). Surfaced via the ExactDensity capability and describe.

every_factor_is_composable(children)

Whether a combinator over children may itself serve as a latent-model component.

A combinator whose density is a PRODUCT of its children’s (a composite over fields, a sequence law over elements and a length) differs from a law by a positive scale exactly when each of its children does – and a mixture absorbs that scale into the component’s own mixing weight, leaving responsibilities unchanged. Without this delegation a combinator over an admissible factor was refused where the bare factor was accepted, and the library’s own EM produces exactly that state: a component that wins no responsibility re-estimates to an empty categorical, so rebuilding the model from its components failed with advice about len_estimator= in a run that already had one (P10-F13).

A child that declines – a scoring_only factor – makes the whole product decline.

Return type:

bool

join_density_semantics(semantics)

Combine child density semantics for a combinator whose log_density is monotone in its children.

A combinator whose score rises with each child’s log_density – a mixture’s logsumexp, a composite’s sum – inherits: a lower bound if any child is a lower bound, an upper bound if any is an upper bound, exactness only if all children are exact, and an undirected ESTIMATE if bounds of both directions (or any estimate) are mixed.

Return type:

DensitySemantics

class FitProvenance(algorithm, estimator, objective, iterations, max_iterations, converged, delta=None, final_objective=None, objective_gain=None, n_observations=None, repairs=(), seed=None, last_accepted_objective=None)

Bases: object

How a fitted distribution was produced – the estimator-side half of MXR-080-1190/1202.

density_semantics() already tells a caller whether a score is an exact density, a bound, or an unnormalized factor. It says nothing about fitting: a GaussianDistribution returned by optimize() used to be byte-identical to one written by hand, so a consumer could not tell a converged fit from one that hit the iteration cap, nor learn that a covariance was repaired on the way. This record carries that, and is_approximate() folds it into the one question most callers actually ask.

None from ProbabilityDistribution.fit_provenance() means the object was constructed directly and no fit produced it – which is different from a fit that produced no receipt.

Parameters:

• algorithm (str)

• estimator (str)

• objective (str)

• iterations (int)

• max_iterations (int)

• converged (bool)

• delta (float | None)

• final_objective (float | None)

• objective_gain (float | None)

• n_observations (int | None)

• repairs (tuple[str, ...])

• seed (int | None)

• last_accepted_objective (float | None)

algorithm

the fitting procedure, e.g. "em", "block-em", "fused-em".

Type:

str

estimator

class name of the ParameterEstimator that drove it.

Type:

str

objective

which objective was maximized – "mle", "map", "vb".

Type:

str

iterations

iterations actually run (not the cap).

Type:

int

max_iterations

the cap that was in force.

Type:

int

converged

True only if the objective gain fell below delta before the cap. False means the run stopped without the gain ever falling below delta – either at the max_iterations cap with the objective still improving, or earlier (iterations < max_iterations) because the family’s next update failed the monotone acceptance gate. In that second case more iterations would not help; the initialization (init_p, rng) is what determines the returned parameters. Like iterations and objective_gain this describes the RUN, not the returned model: under validation-based best-model selection the returned model may be an earlier iterate than the step that converged (its value is final_objective; the trajectory’s is last_accepted_objective).

Type:

bool

delta

the convergence threshold, or None when the caller asked for a fixed iteration count. A fixed-count request (delta=None) can still see iterations < max_iterations: the monotone acceptance gate runs regardless of delta and a rejected step ends the loop early exactly as it would with a real delta. optimize() raises a UserWarning when that happens; compare iterations to max_iterations here to check yourself.

Type:

float | None

final_objective

the objective value of the model this receipt is attached to – the model the fit actually returned. When selection returns an earlier iterate than the trajectory’s last accepted step (validation-based best-model selection, or monotone=False), this is that returned iterate’s value; the trajectory’s own last value is then last_accepted_objective.

Type:

float | None

objective_gain

the last accepted improvement; compare against delta.

Type:

float | None

last_accepted_objective

the objective value of the trajectory’s LAST accepted step, recorded by loops that may return a different (better) iterate. Equal to final_objective for a monotone run that returned its last step; None when the loop recorded no separate trajectory value (the returned model is the last accepted step by construction).

Type:

float | None

n_observations

rows the fit consumed.

Type:

int | None

repairs

numerical repairs applied during the fit, e.g. ("min_covar-clamped",). Empty means none were recorded – by a fit that records them.

Type:

tuple[str, …]

seed

the seed governing any stochastic initialization, when one was set.

Type:

int | None

is_approximate()

Whether this fit is a stopped-early or repaired approximation rather than a clean optimum.

True when the run hit its iteration cap without converging, or when a numerical repair changed the parameters. Both mean the returned law is not simply “the estimator’s answer on this data”.

Return type:

bool

classmethod from_dict(value)

Rebuild a receipt from as_dict() output, re-validating every field.

Fields are passed through as they were recorded rather than coerced. The coercions here used to be lossy in exactly the direction that matters: bool("false") is True, so a persisted record whose convergence verdict had been corrupted to a string was rebuilt as a CONVERGED fit, and int("3")/float("nan") accepted text where a measurement belongs (MXR-080-1190/1202). __post_init__ now type-checks each field, so handing it the recorded value makes a malformed record fail loudly instead of being silently reinterpreted.

Parameters:

value (dict[str, Any])

Return type:

FitProvenance

as_dict()

A JSON-compatible record, for provenance ledgers and receipts.

Return type:

dict[str, Any]

exception EnumerationError(dist, path='', reason='')

Bases: NotImplementedError

Raised when a distribution (or a child of a combinator) cannot enumerate its support.

The path argument identifies the offending child within a combinator, e.g. ‘CompositeDistribution.dists[1]’.

Parameters:

• dist (Any)

• path (str)

• reason (str)

Return type:

None

exception KeyValidationError

Bases: ValueError

Raised when keyed sufficient-statistic sites are incompatible.

A key denotes an equality constraint across model sites. Sites sharing a key must therefore have the same accumulator family and compatible estimator settings before their sufficient statistics are pooled.

exception ContractError(path, expected, actual, fix='')

Bases: ValueError

Raised when malformed data trips a combinator’s seq_encode/estimate contract boundary.

Mirrors EnumerationError’s path-composition convention: path names the full field path through nested combinators (composed with " -> ", outermost first), e.g. "CompositeDistribution.dists[2] -> SequenceDistribution.entries". Every ContractError also carries what was expected, what actually arrived, and (when there is one) a concrete, non-generic suggestion for the likely fix – so a caller reading only the message, not the traceback, knows what to change.

Parameters:

• path (str)

• expected (str)

• actual (str)

• fix (str)

Return type:

None

prefix_contract_error(prefix, err)

Return a new ContractError with prefix prepended to err’s field path.

Used by a combinator to annotate a ContractError raised deep inside a child’s seq_encode/estimate with the outer field position, so the final message names the FULL path down to where the failure actually occurred (e.g. a mixture-of-composites-of- sequences error names every level, not just the outermost combinator).

Parameters:

• prefix (str)

• err (ContractError)

Return type:

ContractError

child_enumerator(child, path)

Construct child.enumerator(), annotating EnumerationError with the child’s path.

Combinator enumerators use this so a failure deep in a nested model reports the full path to the offending leaf, e.g. ‘CompositeDistribution.dists[1] -> MixtureDistribution.components[0] -> GaussianDistribution …’.

Parameters:

• child (ProbabilityDistribution)

• path (str)

Return type:

DistributionEnumerator

class FitProvenanceCarrier

Bases: object

Mixin giving a fitted object the three provenance accessors.

Split out of ProbabilityDistribution because not every fitted result inherits from it: learn_structure returns a DependencyTreeDistribution, whose MRO is (cls, object), so a public fitting entry point produced a model that could not report how it was fitted at all (MXR-080-1190/1202). The methods are plain attribute lookups with no other base-class dependency, so any fitted type can carry them.

fit_provenance()

How this object was fitted, or None if it was constructed directly.

A distribution is a value: nothing about its parameters says whether they came from a converged EM run, a run that hit its iteration cap, or a keyboard. optimize() attaches a FitProvenance so that difference is machine-readable (MXR-080-1190/1202).

Return type:

FitProvenance | None

numerical_repairs()

Numerical repairs applied while building this object – () when none were recorded.

Repairs recorded on child distributions are reported too, under the child’s label, so a container’s receipt says what its leaves had to repair (P02-F05).

Return type:

tuple[str, …]

with_fit_provenance(provenance)

Record provenance on this object and return it.

Parameters:

provenance (FitProvenance)

Return type:

FitProvenanceCarrier

class ProbabilityDistribution

Bases: FitProvenanceCarrier, ABC

Base class for all probability distributions in mixle.stats.

A distribution evaluates the (log-)density of a single observation of its data type, creates a DistributionSampler for drawing observations, and creates a ParameterEstimator for re-estimating itself from data. Discrete distributions may additionally provide a DistributionEnumerator over their support.

fit_provenance()

How this object was fitted, or None if it was constructed directly.

A distribution is a value: nothing about its parameters says whether they came from a converged EM run, a run that hit its iteration cap, or a keyboard. optimize() attaches a FitProvenance so that difference is machine-readable (MXR-080-1190/1202).

Return type:

FitProvenance | None

numerical_repairs()

Numerical repairs applied while building this object – () when none were recorded.

A repair means the parameters are not exactly what the estimator computed: a covariance that was jitter-healed back to positive-definite, a variance clamped to a floor. Reported here so fit_provenance() can carry it, since a repaired fit is an approximation whether or not the objective converged.

An empty tuple means no repair was recorded, which for a family that records none is not the same as a proof that none occurred.

Repairs recorded on child distributions are reported too, under the child’s label, so a container’s receipt says what its leaves had to repair (P02-F05).

Return type:

tuple[str, …]

with_fit_provenance(provenance)

Record provenance on this object and return it.

Mutates in place and returns self rather than copying: fitted models are large, the caller is the fitting routine that just produced this object, and no one else holds a reference yet.

Parameters:

provenance (FitProvenance)

Return type:

ProbabilityDistribution

to_dict()

Return a safe JSON-compatible representation of this distribution.

Return type:

dict[str, Any]

classmethod from_dict(payload)

Reconstruct a distribution from to_dict output.

Parameters:

payload (dict[str, Any])

Return type:

ProbabilityDistribution

to_json(**kwargs)

Serialize this distribution as safe strict JSON.

Read back through from_json before returning, the same check mixle.stats.dump_models() (verify=True) performs: encoding alone only proves the registry accepted this object’s __dict__, not that the state reconstructs through the class’s own constructor. A family whose estimator pins extra state onto the fitted object (e.g. VonMisesDistribution’s fit_metadata) encodes cleanly and then refuses to load, and without this check that failure would surface later, on a different object or in a different process, instead of here where the object is still live and recoverable another way (to_dict(), or pickling).

Parameters:

kwargs (Any)

Return type:

str

classmethod from_json(text)

Deserialize a distribution from to_json output.

Parameters:

text (str)

Return type:

ProbabilityDistribution

density(x)

Return the probability density or mass at a single observation.

Concrete default: exponentiate log_density (the abstract method subclasses must provide). Leaves with a cheaper closed form may override this.

Parameters:

x (Any)

Return type:

float

capabilities()

Return the capability names this distribution supports (see mixle.capability).

Feature detection by behaviour rather than class — e.g. "Enumerable", "Conditionable", "ExponentialFamily", "RankableByIndex". Equivalent to mixle.capabilities(self); combinators report the set their children jointly preserve.

Return type:

frozenset[str]

density_semantics()

What log_density returns relative to the true log-density (default: exact).

Override to declare that this distribution’s log_density is a variational lower bound (ELBO), an upper bound, or an approximation rather than the exact log p(x). This is surfaced as the ExactDensity capability and noted in mixle.describe(), so code that needs an exact likelihood can require(x, ExactDensity) instead of silently trusting a bound.

Return type:

DensitySemantics

tropical_displacement_bits()

Worst-case gap (in bits) between this law’s structural count cost and its true log-density.

The structural count-DP (and the seek built on it) bins each value by a cost that is EXACT for decomposable families – composites/sequences/markov chains whose log p(x) is a sum of independent per-factor terms – so this returns 0.0 by default.

For a marginal family the latent index is summed out and the count index bins by the tropical (dominant-component/path) cost M(x) instead of the true log p(x). Because M(x) <= log p(x) <= M(x) + log N for an N-way logsumexp, the two costs differ by at most log2(N) bits. mixle.enumeration.density_rank.marginal_seek() widens its rank bracket by exactly this many bits so the bracket is a guaranteed bound on the true marginal rank (not merely the tropical rank); override to return log2(N) for such a family.

Return type:

float

abstractmethod log_density(x)

Return the log-density or log-mass at a single observation.

Parameters:

x (Any)

Return type:

float

abstractmethod sampler(seed=None)

Return a sampler for drawing observations from this distribution.

Parameters:

seed (int | None)

Return type:

DistributionSampler

abstractmethod estimator(pseudo_count=None)

Return an estimator for fitting this distribution from data.

Parameters:

pseudo_count (float | None)

Return type:

ParameterEstimator

to_fisher(**kwargs)

Return a Fisher-geometry view of this distribution.

The default view is accumulator-backed, so distributions inherit a generic sufficient-statistic/Fisher-vector interface. Each distribution owns its Fisher view by overriding this method in its own module; families not yet migrated to a per-file hook are resolved by the transitional type-name dispatch in mixle.inference.fisher._legacy_to_fisher().

to_exponential_family(engine=None)

Return the canonical exponential-family view, or None.

The canonical form is p(x) = h(x) * exp(<eta, T(x)> - A(eta)). The default reads declaration_for(self).exponential_family (the per-family ExponentialFamilySpec) and wraps it in an ExponentialFamilyForm; it returns None when this family is not a (single) exponential family. There is no type switch – adding a family is a matter of providing its spec.

Parameters:

engine (Any)

get_prior()

Return the conjugate/parameter prior carried by this distribution, if any.

A distribution participates in the Bayesian (variational) protocol by carrying a prior over its parameters. The default returns whatever was stored on the prior attribute (None for a plain point model), so frequentist distributions answer None and behave as MLE models.

Return type:

ProbabilityDistribution | None

set_prior(prior)

Attach a parameter prior to this distribution.

The default just records the prior; conjugate families override this to precompute the variational expected natural parameters used by expected_log_density.

Parameters:

prior (ProbabilityDistribution | None)

Return type:

None

has_conjugate_prior()

Return whether this family supports a closed-form conjugate Bayesian update.

Uniform, family-level signal backed by the single conjugate_posterior registry: True means mixle.stats.bayes.conjugate_posterior(self, data) returns an exact closed-form posterior; False means Bayesian inference must go through the numerical fitters (MAP / Laplace / MCMC / VI). This is the top tier of the inference-capability ladder (see mixle.capability.ConjugateUpdatable). Distinct from the per-instance has_conj_prior flag, which records whether a prior is currently attached.

Return type:

bool

expected_log_density(x)

Return the variational expectation E_q[log p(x | theta)].

When the distribution carries a conjugate parameter posterior q this is the Bayesian E-step term; for a plain point model (no prior) it degenerates to the plug-in log_density(x). Conjugate families override this with their closed form.

Parameters:

x (Any)

Return type:

float

enumerator()

Return a DistributionEnumerator over this distribution’s support.

Distributions with an enumerable (discrete) support override this; the default raises EnumerationError.

Return type:

DistributionEnumerator

support_size()

Return the number of distinct support points, or None if infinite/unknown.

This is the cardinality primitive for bounding a truncated descending-probability sum: after enumerating the top k items (whose smallest probability is p_k), every un-enumerated item has probability <= p_k, so the remaining mass is <= (support_size - k) * p_k (see mixle.enumeration.density_rank.truncated_sum_bound()). Finite discrete leaves return their cardinality; decomposable combinators compose it structurally; an upper bound is acceptable (it only loosens the tail bound). Infinite or continuous supports return None.

Return type:

int | None

support_is_finite()

Return whether the support has a known finite cardinality.

Return type:

bool

quantized_index(max_bits, bin_width_bits=1.0)

Build a bounded bit-quantized index over this distribution’s support.

This is a convenience wrapper around self.enumerator().quantized_index. Non-enumerable distributions raise EnumerationError through enumerator().

Parameters:

• max_bits (float) – Maximum information content in bits to index.

• bin_width_bits (float) – Width of each quantized probability bin in bits.

Returns:

mixle.enumeration.algorithms.QuantizedEnumerationIndex.

density_quantile(q, n_samples=20000, seed=None, *, invalid_score='raise')

Return a representative value at cumulative-density index q (descending-density order).

The arbitrary-index / inverse of the probability-ordered cumulative that mixle.enumeration.density_rank.density_rank() computes (G(x) = P(p(Y) >= p(x))): q = 0 is the mode, q -> 1 walks into the tail. Families with a closed form override this (univariate continuous leaves expose the spatial quantile; multivariate Gaussians and von Mises-Fisher override density_quantile exactly); the default here is the universal Monte-Carlo representative for any samplable family whose support is uncountable or coupled (parameter priors, continuous mixtures, …): draw n_samples points, order them by descending density, and return the one at fractional rank q. Stochastic and approximate – use the exact methods (enumerator/count_dp_seek for discrete) where available.

Parameters:

• q (float) – Cumulative-density index in [0, 1].

• n_samples (int) – Monte-Carlo sample budget.

• seed (Optional[int]) – Sampler seed (reproducible).

• invalid_score (str) – "raise" for non-finite sampled scores, or "omit" to discard them explicitly.

Return type:

Any

density_enumeration(num_points, n_samples=20000, seed=None, *, dedup_key=None, invalid_score='raise')

Return num_points representative (value, log_density) pairs in descending density.

The continuous analogue of enumerator() (which enumerates a countable support exactly): for an uncountable or coupled support there is no exact element enumeration, so this returns a Monte-Carlo representative sweep – n_samples draws ordered by descending density, keeping the num_points most probable (distinct) representatives, i.e. “the support, most-probable region first”. Stochastic and approximate; prefer enumerator() where the support is countable.

Parameters:

• num_points (int) – Number of representatives to return.

• n_samples (int) – Monte-Carlo sample budget.

• seed (Optional[int]) – Sampler seed (reproducible).

• dedup_key (Optional[Callable]) – Stable equality key for values not supported by the default canonical freeze operation.

• invalid_score (str) – "raise" for non-finite sampled scores, or "omit" to discard them explicitly.

Return type:

list[tuple[Any, float]]

quantized_count_index(quantizer, max_fine_bucket)

Build a structural CountIndex over this distribution’s support, bounded by depth.

This is the count-semiring counterpart of quantized_index: it returns per-fine-bucket counts of the complete model probability together with a structural unranker, so the support can be indexed without being enumerated. The default builds a leaf index from the exact enumerator() truncated at max_fine_bucket (efficient for closed-form/small-support families); exponential-support composers (Composite/Sequence/MarkovChain) override this with a dynamic program over the model’s likelihood recursion.

Parameters:

• quantizer (mixle.enumeration.quantization.Quantizer) – Fine/coarse bucketing.

• max_fine_bucket (int) – Inclusive depth bound on indexed fine buckets.

Returns:

Tuple (CountIndex, truncated) – truncated is True when in-support values were dropped because they fell beyond the depth bound.

count_budget_index(budget_bits, bin_width_bits=1.0, oversample=8, num_workers=None)

Build a budget-bounded quantized seek index covering the top 2**budget_bits values.

Computes per-bin counts structurally (never enumerating the domain) and accumulates coarse bins in descending-probability order until the cumulative count reaches the budget. The returned LazyQuantizedEnumerationIndex supports arbitrary-rank seek/unranking; each unranked value carries its exact log_density.

Parameters:

• budget_bits (float) – Index into the top 2**budget_bits most probable values.

• bin_width_bits (float) – Coarse output bin width in bits.

• oversample (int) – Fine buckets per coarse bin (accumulation resolution).

• num_workers (int | None)

Returns:

mixle.enumeration.algorithms.LazyQuantizedEnumerationIndex.

count_budget_distinct(budget_bits, bin_width_bits=1.0, oversample=8, dedup='canonical', start=0, stop=None, max_entries=1 << 16, num_workers=None)

Iterate DISTINCT (value, exact_log_prob) over the count-budget index, approx descending.

For exact-count families this equals the ordered index stream. For the over-counting MARGINAL families (Mixture/HMM) it removes the component/path duplicates by one of two modes:

• dedup='canonical' (default): a STATELESS predicate (is_canonical_copy) keeps a value only at its dominant copy (best-weighted component / min-cost path), via the value’s structural fine bucket (structural_fine_bucket – the SAME sum-of-floored sub-buckets the count index used, so nested composite/sequence values are binned consistently and never dropped). O(1) memory and random-accessible: start/stop select an arbitrary STRUCTURAL rank range, so you can begin anywhere and the work partitions across workers with no shared state. GUARANTEE: every distinct in-budget value is emitted at least once (completeness). It is NOT strictly once – a value is emitted once per component/path that ties within its minimal 1-bit coarse bin; that residue is inherent to a stateless (value, coarse_bin) rule (the tied copies are indistinguishable to it) and is bounded by the number of components/contributing paths (not reduced by oversample, which only refines the intermediate fine bucket). For a strictly-once stream, use dedup='window' or de-duplicate downstream.

• dedup='window': a bounded max_entries LRU over the stream (catches every duplicate within the window regardless of dominance, but is sequential – start must be 0).

Note: start/stop index the STRUCTURAL enumeration, not the distinct rank. Jumping to the k-th distinct value in O(1) is not possible – it needs exact distinct per-bin counts, which require materializing the component/path overlap structure.

Parameters:

• budget_bits (float)

• bin_width_bits (float)

• oversample (int)

• dedup (str)

• start (int)

• stop (int | None)

• max_entries (int)

• num_workers (int | None)

is_canonical_copy(value, coarse_bin, quantizer)

Return True if coarse_bin is value’s dominant (canonical) bin in the count index.

Stateless deduplication hook for the over-counting MARGINAL families: a value that the structural index emits once per component / state-path is kept only at the copy whose bin is the minimal (most probable) one. The default returns True – exact-count families (Composite/Sequence/MarkovChain) never duplicate, so every copy is canonical.

Parameters:

coarse_bin (int)

Return type:

bool

structural_fine_bucket(value, quantizer)

Minimum fine bucket where value is placed by this distribution’s count index.

Mirrors quantized_count_index exactly so that stateless canonical-copy dedup can predict the bucket the index actually used. The count DP bins a composite/nested value by a SUM of floored per-factor buckets (a convolution), which differs from a single floor of the exact joint log-density by up to the number of factors – so a canonical check that used fine_bucket(log_density(value)) would mispredict and silently drop nested values. The leaf default is that single floor (correct for atomic families); combinators (Composite/Sequence/Mixture) override to recurse the same way their count index composes.

Return type:

int

quantized_multi_cross_index(others, max_bits, bin_width_bits=1.0)

Build an aligned bounded cross-bin view against other distributions.

The generic implementation is a bounded candidate join: it unions the bounded quantized indexes of all participating distributions, then evaluates every candidate under every distribution. Structured distributions can override this to build the same aligned rows from support algebra instead.

Parameters:

• others (list[ProbabilityDistribution])

• bin_width_bits (float)

quantized_cross_index(other, max_bits, bin_width_bits=1.0)

Build an aligned bounded cross-bin view against another distribution.

Parameters:

• other (ProbabilityDistribution)

• bin_width_bits (float)

class SequenceEncodableProbabilityDistribution

Bases: ProbabilityDistribution

ProbabilityDistribution with vectorized log-density evaluation on encoded data.

dist_to_encoder() returns a DataSequenceEncoder whose seq_encode() output is consumed by seq_log_density() (and by the matching accumulator’s seq_update / seq_initialize), enabling fast vectorized estimation over iid sequences.

supported_engines()

Return engine names this distribution can evaluate on directly.

Return type:

tuple[str, …]

supports_engine(engine)

Return True when the distribution can safely use engine.

Parameters:

engine (Any)

Return type:

bool

decomposition()

Return how this distribution may be split across devices (model parallelism).

Defaults to Decomposition.atomic() – not split, replicated. Combinators / latent families override this to declare their component / factor / state axis. See mixle.stats.compute.decomposition.

Return type:

Decomposition

seq_ld_lambda()

Return vectorized log-density callables for encoded data.

seq_log_density(x)

Return vectorized log-density values for sequence-encoded observations.

Parameters:

x (Any)

Return type:

ndarray

seq_expected_log_density(x)

Vectorized expected_log_density over sequence-encoded observations.

Degenerates to seq_log_density for a plain point model; conjugate families override with a closed form. See expected_log_density.

Parameters:

x (Any)

Return type:

ndarray

seq_log_density_lambda()

Return vectorized log-density callables for encoded data.

kernel(engine=None, estimator=None)

Return an engine-aware evaluation kernel for this distribution.

Parameters:

estimator (ParameterEstimator | None)

abstractmethod dist_to_encoder()

Return the data encoder used by this distribution for vectorized methods.

Return type:

DataSequenceEncoder

class DistributionSampler(dist, seed=None, *, rng=None)

Bases: ABC

Draws iid observations from a distribution using a seeded RandomState.

sample(size=None) returns a single observation of the distribution’s data type; sample(size=n) returns a length-n collection of observations.

Parameters:

• dist (SequenceEncodableProbabilityDistribution)

• seed (int | None)

• rng (RandomState | None)

new_seed()

Return a fresh random seed drawn from this sampler’s RandomState.

Return type:

int

abstractmethod sample(size=None, *, batched=True)

Draw observations.

Combinator samplers (mixture/sequence/…) accept batched. With batched=True (the default) each child stream is drawn in one vectorized call instead of a per-draw Python loop – far faster. Because every child sampler owns an independent RandomState, batching consumes each stream in the same order as the loop, so the draws are identical to the legacy path. batched=False forces that legacy per-draw loop as a guaranteed- stable reference. Leaf samplers are already vectorized and ignore the flag.

Parameters:

• size (int | None)

• batched (bool)

Return type:

Any

class NucleusResult(items, *, target_probability, cumulative_probability, reached_target, capped, exhausted)

Bases: list[tuple[Any, float]]

List-compatible nucleus prefix with explicit termination provenance.

Parameters:

• items (list[tuple[Any, float]])

• target_probability (float)

• cumulative_probability (float)

• reached_target (bool)

• capped (bool)

• exhausted (bool)

class DistributionEnumerator(dist)

Bases: ABC

Lazy iterator over the support of dist in non-increasing probability order.

Yields (value, log_prob) pairs, possibly infinitely many. Contract:

• Each support value is yielded exactly once (deduplication is the enumerator’s responsibility).

• log_prob equals dist.log_density(value) up to float round-off (~1e-10), and the sequence of log_probs is non-increasing up to the same tolerance.

• Values with zero probability are skipped, never yielded.

• Ties are broken deterministically by insertion order; no further guarantee.

Parameters:

dist (SequenceEncodableProbabilityDistribution)

top_k(k)

Return the k most probable (value, log_prob) pairs (fewer if the support is smaller).

Parameters:

k (int)

Return type:

list[tuple[Any, float]]

top_p(p, max_items=None)

Return the smallest descending-probability prefix whose total probability reaches p.

The nucleus / minimal high-probability set: because values are yielded in non-increasing probability order, the returned prefix is a minimum-size set of outcomes whose summed mass is >= p (e.g. p=0.95 gives a 95%-coverage support set – the discrete analogue of nucleus sampling). Accumulation stops as soon as the cumulative probability reaches p.

max_items caps how many values are pulled so an infinite or heavy-tailed support cannot run away. A cap is mandatory when the support has no known finite cardinality. The returned list-compatible NucleusResult reports whether the target was reached, the accumulated mass, and whether a cap stopped the search.

Parameters:

• p (float) – Target cumulative probability in [0, 1].

• max_items (Optional[int]) – Hard cap on the number of values pulled.

Returns:

A list-compatible nucleus result with termination provenance.

Return type:

NucleusResult

quantized_index(max_bits, bin_width_bits=1.0)

Precompute a bounded bit-quantized index over this enumeration.

The index groups values by floor((-log2 p(x)) / bin_width_bits), includes only values with -log2 p(x) <= max_bits, and returns exact log probabilities for indexed values. Building the index consumes this enumerator.

Parameters:

• max_bits (float) – Maximum information content in bits to index.

• bin_width_bits (float) – Width of each quantized probability bin in bits.

Returns:

mixle.enumeration.algorithms.QuantizedEnumerationIndex.

rank(value)

Rank and cumulative probability of value in the descending-probability order.

Returns a DensityRankResult with .rank (0-based count of strictly-more-probable outcomes; None if only the sampling estimate was used) and .cumulative_probability (G(value) = P(p(Y) >= p(value))). Exact head enumeration where the support is countable, with a Monte-Carlo fallback for the deep tail.

Parameters:

value (Any)

seek(index)

The value at descending-probability index (0-based) – the inverse of rank().

Returns a CountDPSeekResult carrying the value and a provable [rank_lower, rank_upper] bracket. Uses the structural count-DP for decomposable families, so arbitrarily deep indices are reachable without enumerating the prefix.

Parameters:

index (int)

seek_certified(index)

The value at descending index with a GUARANTEED bracket on its TRUE marginal rank.

Unlike seek() – whose bracket bounds only the tropical rank for a marginal family (mixture/HMM) – this widens the rank window by the family’s tropical_displacement_bits and divides out the component over-count, so the returned MarginalSeekResult [true_rank_lower, true_rank_upper] provably contains #{u : log p(u) > log p(value)}. It pins the rank exactly (.exact) for decomposable / provably-disjoint families and for shallow indices, and otherwise returns the certified provable envelope. For a decomposable family it agrees with seek().

Parameters:

index (int)

cumulative(value)

G(value) = P(p(Y) >= p(value)) – total mass of outcomes at least as probable as value.

Returns a CumulativeProbabilityResult with .probability plus explicit certification (.exact, .dropped_upper) of what that figure rests on.

Parameters:

value (Any)

nucleus_size(p)

Size of top_p() (the minimal >= p-mass set) WITHOUT materializing it.

Returns a CountDPTopPResult with a provable size bracket, from the structural count-DP – usable when the nucleus is far too large to list.

Parameters:

p (float)

from_index(start, stop=None)

Iterate (value, log_prob) in descending-probability order starting at structural start.

Yields the same stream as iterating a fresh enumerator but beginning at index start (and ending before stop if given). A fresh underlying enumeration is used, so this does not consume self. (Decomposable families admit a direct structural jump via the count-budget index; the current implementation skips the best-first prefix – the structural fast path is a WS-3 performance follow-up.)

Parameters:

• start (int)

• stop (int | None)

class ConditionalSampler

Bases: ABC

Sampler mixin for conditional draws: sample_given(x) draws from P(. | x).

abstractmethod sample_given(x)

Draw a sample from the conditional distribution given x.

class StatisticAccumulator

Bases: ABC, Generic[SS]

Accumulates weighted sufficient statistics of type SS from observations.

update(x, weight, estimate) adds one observation (estimate is the previous model, used for E-step posteriors; it may be None during initialization). Accumulators merge across partitions via combine(suff_stat) / value() / from_value(), and key_merge / key_replace pool statistics shared across model components through a stats_dict keyed by the accumulator’s key.

update(x, weight, estimate)

Accumulate one weighted observation under an optional current estimate.

Parameters:

• x (Any)

• weight (float)

Return type:

None

initialize(x, weight, rng)

Initialize sufficient statistics from one weighted observation.

Parameters:

• x (Any)

• weight (float)

• rng (RandomState)

Return type:

None

abstractmethod combine(suff_stat)

Merge serialized sufficient statistics into this accumulator.

Parameters:

suff_stat (SS)

Return type:

StatisticAccumulator

abstractmethod value()

Return this accumulator’s serialized sufficient statistics.

Return type:

SS

abstractmethod from_value(x)

Restore this accumulator from serialized sufficient statistics.

Parameters:

x (SS)

Return type:

SequenceEncodableStatisticAccumulator

scale(c)

Scale linear sufficient statistics in-place by c.

The structural default is correct for ordinary weighted sums, nested tuples/lists/dicts, and numeric arrays. Families whose value() payload includes non-linear metadata such as support bounds must override this method and leave that metadata unscaled.

Parameters:

c (float)

Return type:

StatisticAccumulator

key_merge(stats_dict)

Pool this accumulator’s statistics into stats_dict under its merge key.

The structural default implements the common single-key pattern: store the accumulator under self.keys the first time the key is seen, else combine into the one already there. Accumulators whose key does not cover their whole state (several named keys, a wrapped child accumulator, a non-accumulator payload) override this – and ONLY those: as of 2026-07-14 every whole-state family uses this default, after a sweep found sixteen hand-rolled copies that had drifted into three broken shapes. If you are tempted to reimplement this per family, don’t; extend the base or override with super() calls. A keys of None (the default) is a no-op.

Why this exact shape is correct – and the drifted shapes are not: the POOL must live in the dict. Combining INTO the dict-held accumulator (below) mutates the pool in place, so after all sites merge, the dict entry holds everyone’s statistics and key_replace hands that pool to every site. The observed broken variants each violate one half: pulling the dict’s stats into self without writing back (the dict keeps the FIRST site only – later sites’ data silently discarded); merging only when the key is already present (nobody ever inserts: both passes are no-ops and tying does nothing); and replacing by pushing self INTO the dict (the LAST site overwrites the pool). The keyed-protocol sweep (keyed_protocol_sweep_test.py) enforces this behaviorally for every catalog family.

Parameters:

stats_dict (dict[str, Any])

Return type:

None

key_replace(stats_dict)

Replace this accumulator’s statistics from the pooled stats_dict entry (see key_merge).

Parameters:

stats_dict (dict[str, Any])

Return type:

None

class SequenceEncodableStatisticAccumulator

Bases: StatisticAccumulator[SS]

StatisticAccumulator with vectorized updates on encoded data sequences.

seq_update / seq_initialize consume the output of the matching DataSequenceEncoder’s seq_encode() (obtained via acc_to_encoder()) together with a per-observation weight vector.

get_seq_lambda()

Return optional low-level sequence-update kernels used by generated code.

supported_rows(x)

Encoded rows this accumulator can be fitted on, or None when every row qualifies.

Support-limited laws admit out-of-support observations at encode time so that a mixture can encode one batch against every component (P02-F03) – the component that does not own a value scores it -inf, and the E-step gives it no responsibility for that row. Initialization draws responsibilities before any model exists, so a latent model asks each component for this mask first and never seeds a component from a row it could not be fitted on.

Return type:

ndarray | None

abstractmethod seq_update(x, weights, estimate)

Accumulate weighted sufficient statistics from sequence-encoded observations.

Parameters:

weights (ndarray)

Return type:

None

abstractmethod seq_initialize(x, weights, rng)

Initialize sufficient statistics from sequence-encoded observations.

Parameters:

• weights (ndarray)

• rng (RandomState)

Return type:

None

abstractmethod acc_to_encoder()

Return a sequence encoder compatible with this accumulator.

Return type:

DataSequenceEncoder

scale_suff_stat(x, c)

Return x with numeric sufficient-statistic leaves multiplied by c.

Parameters:

• x (Any)

• c (float)

Return type:

Any

class StatisticAccumulatorFactory

Bases: ABC

Factory whose make() returns a fresh, zeroed accumulator for one estimator.

abstractmethod make()

Create a fresh accumulator instance.

Return type:

SequenceEncodableStatisticAccumulator

class ParameterEstimator

Bases: ABC, Generic[SS]

Estimates a distribution from accumulated sufficient statistics.

accumulator_factory() supplies accumulators that gather sufficient statistics of type SS, and estimate(nobs, suff_stat) maps those statistics (plus optional regularization configured on the estimator) to a new distribution.

to_dict()

Return a safe JSON-compatible representation of this estimator.

Return type:

dict[str, Any]

classmethod from_dict(payload)

Reconstruct an estimator from to_dict output.

Parameters:

payload (dict[str, Any])

Return type:

ParameterEstimator

to_json(**kwargs)

Serialize this estimator as safe strict JSON.

Parameters:

kwargs (Any)

Return type:

str

classmethod from_json(text)

Deserialize an estimator from to_json output.

Parameters:

text (str)

Return type:

ParameterEstimator

abstractmethod estimate(nobs, suff_stat)

Estimate a distribution from accumulated sufficient statistics.

Parameters:

• nobs (float | None)

• suff_stat (SS)

Return type:

SequenceEncodableProbabilityDistribution

abstractmethod accumulator_factory()

Return the accumulator factory used to collect this estimator’s sufficient statistics.

Return type:

StatisticAccumulatorFactory

resident_accumulation_supported()

Return whether engine-resident (fixed-width) sufficient statistics suffice for estimate.

Most exponential-family M-steps consume only the resident sufficient statistics, so the default is True. Estimators whose M-step needs more than that (e.g. a full count histogram for the negative-binomial dispersion solve) override this to False so stacked/generated kernels fall back to the host accumulator, keeping every backend’s fixed point identical.

Return type:

bool

get_prior()

Return the parameter prior configured on this estimator, if any.

The unified estimation contract treats the prior as the single regularization concept: None gives maximum likelihood, a conjugate prior gives the Bayesian posterior update inside estimate. The default reads the prior attribute (None when unset).

Return type:

ProbabilityDistribution | None

model_log_density(model)

Return the prior log-density of model’s parameters (the ELBO global term).

Used by the variational/MAP objective in fit. The default is 0.0 (no prior); conjugate estimators override this to evaluate their prior at the model’s parameters, mapping to the prior’s parameterization first.

Parameters:

model (ProbabilityDistribution)

Return type:

float

class DataSequenceEncoder

Bases: ABC

Encodes an iid data sequence into the vectorized form used by seq_* methods.

seq_encode(x) transforms a sequence of observations into the encoding consumed by seq_log_density / seq_update / seq_initialize. Encoders must define __eq__ (so equivalent encoders are interchangeable when batching) and a readable __str__.

seq_encode(x)

Encode the iid observation sequence x for vectorized evaluation.

Parameters:

x (Any)

Return type:

Any

nbytes(x)

Return the approximate in-memory byte size of an encoded payload.

Parameters:

x (Any)

Return type:

int

row_count(x)

Return the number of observations represented by x.

The default covers simple array/list encodings and structurally aligned tuples/dicts. Encoders with flattened, ragged, or otherwise specialized layouts must override this method so metadata wrappers can verify counts instead of trusting a caller assertion.

Parameters:

x (Any)

Return type:

int

encoded_nbytes(x)

Return an approximate byte size for nested encoded array payloads.

Encoders mostly return arrays or tuples/lists/dicts of arrays. The helper keeps accounting structural and deterministic; Python object overhead is included only for scalar leaves where no array-native byte count exists.

Parameters:

x (Any)

Return type:

int

validate_estimator_keys(estimator)

Validate keyed estimator and accumulator sites before EM folds stats.

The validator catches the classic keying footgun: two different families, or two sites with incompatible estimator settings, accidentally sharing the same key string. Validation is intentionally protocol-level and best-effort; a family can still perform stricter checks in its own factory if needed.

Parameters:

estimator (ParameterEstimator)

Return type:

None

estimator_has_keys(estimator)

Whether any site in the estimator tree carries a tying key.

Keyed sites require the merge_accumulator_keys pass after accumulation; an EM driver that cannot run it (or whose update decomposition conflicts with pooling, e.g. block-EM’s sparse per-component M-steps) must refuse or reroute keyed estimators rather than silently untie them.

Parameters:

estimator (ParameterEstimator)

Return type:

bool

validate_accumulator_keys(accumulator)

Validate keyed sites in an already-created accumulator tree.

Parameters:

accumulator (StatisticAccumulator)

Return type:

None

merge_accumulator_keys(accumulator)

Pool keyed statistics across accumulator’s tree – the parameter-tying pass.

Runs the key_merge/key_replace pair every EM driver applies exactly once after accumulation (see mixle.stats.compute.sequence.seq_estimate()), so sites sharing a key estimate from the pooled statistics. A no-op when no site in the tree carries a key. Call it on the fully-combined accumulator, never per shard: pooling twice would double-count the shared statistics on the second combine.

Parameters:

accumulator (StatisticAccumulator)

Return type:

None