mixle.stats.latent.mixture module¶
Create, estimate, and sample from a mixture distribution with homogenous components.
Defines the MixtureDistribution, MixtureSampler, MixtureAccumulatorFactory, MixtureAccumulator, MixtureEstimator, and the MixtureDataEncoder classes for use with mixle.
MixtureDistribution is defined by the density of the form,
P(Y) = sum_{k=1}^{K} P(Y|Z=k)*P(Z=k),
where P(Z=k) is a mixture weight for component k, and P(Y|Z=k) is defined as a the k^{th} component distribution.
If component distribution P(Y|Z=k) has data type (T), then the Mixture distribution has data type (T) as well.
- mixture_prior(weight_prior, component_priors)[source]
Build the joint mixture prior: a weight prior plus one prior per component.
- Parameters:
weight_prior (SequenceEncodableProbabilityDistribution) – Prior on the mixture weights (a
DirichletDistributionorSymmetricDirichletDistribution).component_priors (Sequence[SequenceEncodableProbabilityDistribution]) – Sequence of one conjugate prior per component.
- Returns:
A
(weight_prior, tuple(component_priors))pair consumed byMixtureDistribution/MixtureEstimatorset_prior.- Return type:
tuple[SequenceEncodableProbabilityDistribution, tuple[SequenceEncodableProbabilityDistribution, …]]
- class MixtureDistribution(components, w=MISSING, name=None, weights=MISSING, prior=None)[source]
Bases:
SequenceEncodableProbabilityDistributionMixtureDistribution object defined by component distributions and weights.
The args components (Sequence[SequenceEncodableProbabilityDistribution]) define the component distributions of the mixture distribution as well as the data type. The data type of the MixtureDistribution object is taken to be the data type (T) of the component distributions (all must be the same subclass of SequenceEncodableProbabilityDistribution super class).
- Parameters:
components (Sequence[SequenceEncodableProbabilityDistribution]) – Set component distributions. Must be same subclass of SequenceEncodableProbabilityDistribution super class with type T.
w (ndarray[float]) – Mixture weights, must sum to 1.0.
name (Optional[str]) – Assign string name to MixtureDistribution object.
prior (SequenceEncodableProbabilityDistribution | None)
- components
List of component distributions (data type T).
- Type:
List[SequenceEncodableProbabilityDistribution]
- w
Mixture weights assigned from args (w).
- Type:
ndarray[float]
- name
String name to MixtureDistribution object.
- Type:
Optional[str]
- zw
True if a weight is 0.0, else False.
- Type:
ndarray[bool]
- log_w
Log of weights (w). set to -np.inf, where zw is True.
- Type:
ndarray[float]
- num_components
Number of components in MixtureDistribution instance.
- Type:
- compute_capabilities()[source]
- compute_declaration()[source]
- get_prior()[source]
Return the joint mixture prior, or
Nonefor a plain point model.When a weight prior is attached the joint prior is the
(weight_prior, tuple(component priors))pair produced bymixture_prior(); otherwiseNone.- Return type:
SequenceEncodableProbabilityDistribution | None
- set_prior(prior)[source]
Attach a weight prior (and optional per-component priors), caching weight expectations.
With a (symmetric) Dirichlet weight prior this caches the variational weight expectations
E[log w_k] = digamma(alpha_k) - digamma(sum_j alpha_j)used byexpected_log_density. Component priors, when supplied, are delegated to each component viacomponent.set_prior.prior=None(the default) leaves the mixture a plain point model (byte-identical MLE behaviour).- Parameters:
prior (SequenceEncodableProbabilityDistribution | None)
- Return type:
None
- expected_log_density(x)[source]
Variational expected log-density at observation x.
Uses
E[log w_k]under the (symmetric) Dirichlet weight prior together with each component’sexpected_log_density. Falls back to the plug-inlog_density(x)when no conjugate weight prior is attached.- Parameters:
x (T)
- Return type:
- seq_expected_log_density(x)[source]
Vectorized variational expected log-density at sequence-encoded input x.
Falls back to
seq_log_density(x)when no conjugate weight prior is attached.- Parameters:
x (T1)
- Return type:
- density(x)[source]
Evaluate density of Mixture distribution at observation x.
See log_density() for details.
- Parameters:
x (T) – (T): Single observation from mixture distribution. T is data type of components.
- Returns:
Density at x.
- Return type:
- density_semantics()[source]
What
log_densityreturns relative to the true log-density (default: exact).Override to declare that this distribution’s
log_densityis a variational lower bound (ELBO), an upper bound, or an approximation rather than the exactlog p(x). This is surfaced as theExactDensitycapability and noted inmixle.describe(), so code that needs an exact likelihood canrequire(x, ExactDensity)instead of silently trusting a bound.
- log_density(x)[source]
Evaluate log-density of Mixture distribution at observation x.
A K-component Mixture has log-density,
log(P(x)) = log(sum_{z=k}^{K} P(x|z=k)*P(z=k)),
where P(x|z=k) is component-k log-density at x, and P(z=k) = w[k]. A log-sum-exp is used to evaluate the sum inside the log of the right-hand side above. (See mixle.utils.vector.log_sum() for details).
- Parameters:
x (T) – (T): Single observation from mixture distribution. T is data type of components.
- Returns:
Log-density at x.
- Return type:
- conditional(observed)[source]
Return the conditional mixture over the unobserved coordinates given
observed.The conditional of a mixture is itself a mixture: for
sum_k w_k f_kobservingx_o,P(x_u | x_o) = sum_k w’_k f_k(x_u | x_o), w’_k proportional to w_k f_k.marginal(x_o)(x_o),
i.e. the component responsibilities are updated by how well each component explains the observed coordinates and each component is replaced by its own conditional. Because the result is a full
MixtureDistributionyou can both score it and.sampler(seed).sample()from it – the latter isgiven=-style conditional sampling that first draws a component from the posterior responsibilities, then draws the unobserved coordinates from that component’s conditional.Requires each component to support
marginal(indices)andcondition(observed)(e.g. the multivariate Gaussian / Student-t).observedmaps coordinate index to its fixed value.
- component_log_density(x)[source]
Evaluate component-wise log-density of Mixture distribution at observation x.
A K-component Mixture has log-density, log(P(x|z=k)) for the K-th component.
- Parameters:
x (T) – (T): Single observation from mixture distribution. T is data type of components.
- Returns:
Numpy array of floats containing component-wise log-density at x.
- Return type:
- posterior(x)[source]
Obtain the posterior distribution for each mixture component at observation x.
The posterior distribution of component ‘k’ at observation x is given by,
p_mat(Z=k|x) = p_mat(x|Z=k)*p_mat(z=k) / p_mat(x),
where
p_mat(x) = sum_{k=1}^{K} p_mat(x|Z=k)*p_mat(z=k) = sum_{k=1}^{K} p_mat(x|Z=k)*w[k].
This function returns an ndarray[float] of length K, containing p_mat(Z=k|x) as its k^{th} entry.
- Parameters:
x (T) – (T): Single observation from mixture distribution. T is data type of components.
- Returns:
Numpy array of floats containing posterior distribution at observation x.
- Return type:
- seq_component_log_density(x)[source]
Vectorized evaluation of component-wise log-density for encoded sequence x.
Arg x must be a sequence encoded from MixtureDataEncoder.seq_encode(data) with data type Sequence[T] for data, or results from an equivalent encoding from a DataSequenceEncoder object for the components. The resulting encoded sequence is assumed to be data type T1.
Creates a 2-d numpy array of floats with vectorized evaluations of component_log_density() stored in the rows corresponding to an observation in encoded sequence x.
The returned value is an ndarray[float] with shape (sz,K), where K is the number of mixture components, and sz is the number of iid observations in the encoded sequence x.
- Parameters:
x (T1) – See above for details.
- Returns:
2-d numpy array of floats having shape (sz,K), where sz is the number of iid obs in encoded sequence x, and K is the number of mixture components.
- Return type:
- seq_log_density(x)[source]
Vectorized evaluation of log-density for encoded sequence x.
Arg x must be a sequence encoded from MixtureDataEncoder.seq_encode(data) with data type Sequence[T] for data, or results from an equivalent encoding from a DataSequenceEncoder object for the components. The resulting encoded sequence is assumed to be data type T1.
Evaluates the log-density of each observation in the encoded sequence x (see log_density() for details).
The returned value is an ndarray[float] with shape (sz,K), where K is the number of mixture components, and sz is the number of iid observations in the encoded sequence x.
- Note: A row-wise log-sum-exp is performed for numerical stability. If a row contains a log-density value of,
-np.inf is returned for the corresponding observation value in the encoded sequence x.
- Parameters:
x (T1) – See above for details.
- Returns:
Numpy array of floats containing the log_density of each observation in encoded sequence.
- Return type:
- backend_seq_component_log_density(x, engine)[source]
Engine-neutral component log densities for encoded data.
- backend_seq_log_density(x, engine)[source]
Engine-neutral mixture log-density for encoded data.
- gradient_fit_state(engine, torch, leaves, recurse, tensor_param)[source]
Return distribution-owned state for autograd fitting.
- seq_posterior(x)[source]
Vectorized evaluation of posterior of MixtureDistribution for encoded sequence x.
Arg x must be a sequence encoded from MixtureDataEncoder.seq_encode(data) with data type Sequence[T] for data, or results from an equivalent encoding from a DataSequenceEncoder object for the components. The resulting encoded sequence is assumed to be data type T1.
Vectorized evaluation the posterior of each observation in the encoded sequence x (see posterior() for details).
The returned value is an ndarray[float] with shape (sz,K), where K is the number of mixture components, and sz is the number of iid observations in the encoded sequence x. Each row contains the posterior of the corresponding encoded observation.
- Note: A row-wise log-sum-exp is performed for numerical stability. If a row contains a log-density value of,
-np.inf is returned for the corresponding observation value in the encoded sequence x.
- Parameters:
x (T1) – See above for details.
- Returns:
Numpy array of floats containing the posterior of each observation in encoded sequence.
- Return type:
- latent_posterior(x)[source]
Return the latent posterior
q(z | x)over component labels for raw observationsx.q(z)is the exact independent-categorical posterior whose marginals are the EM responsibilities. The returnedCategoricalLatentPosteriorcan.marginals()(the responsibilities),.sample(rng)component labels,.mode()(the MAP labels), or.entropy().- Parameters:
x (Sequence[T])
- Return type:
CategoricalLatentPosterior
- posterior_predictive(x, seed=None)[source]
Draw posterior-predictive observations conditioned on
x.For each observed
x_ithe component is sampled from the latent posteriorq(z_i | x_i)and a fresh observation is emitted from that component – i.e. “given I sawx_i, draw a new point from the same mixture component it likely came from”. Returns a list the length ofx. Draws are grouped by component and scattered (vectorized) via the shared sampling helper.
- support_size()[source]
Upper bound on distinct support points: the sum over components (union <= sum).
- Return type:
int | None
- tropical_displacement_bits()[source]
log2(#positive-weight components)– the tropical-vs-marginal cost gap (in bits).The marginal
log p(x) = logsumexp_k (log w_k + log p_k(x))is bounded by its largest termM(x) = max_k (log w_k + log p_k(x))viaM(x) <= log p(x) <= M(x) + log K, whereKis the number of components that can contribute (positive weight). The structural seek bins by the tropical costM(x);mixle.enumeration.density_rank.marginal_seek()widens its smear window by this many bits so the reported rank bracket provably contains the TRUE marginal rank.K <= 1means the marginal is a single term ->0.0(the seek is then exact). When the component supports are provably disjoint every value lands in one component, soM(x)equals the marginal and there is likewise no displacement ->0.0(the seek is exact and tight).- Return type:
- to_fisher(**kwargs)[source]
Structural Fisher view for the mixture.
- sampler(seed=None)[source]
Create MixtureSampler for sampling from MixtureDistribution instance.
- Parameters:
seed (Optional[int]) – Seed to set for sampling with RandomState.
- Returns:
MixtureSampler object.
- Return type:
MixtureSampler
- estimator(pseudo_count=None)[source]
Create MixtureEstimator for estimating MixtureDistribution.
- Parameters:
pseudo_count (Optional[float]) – Used to inflate sufficient statistics in estimation.
- Returns:
MixtureEstimator object.
- Return type:
MixtureEstimator
- decomposition()[source]
Mixture components split along the component axis. Responsibilities (logsumexp) are computed INSIDE a shard; across shards the per-component sufficient stats SUM-reduce plus one scalar total-count all-reduce – the homogeneous stacked-kernel + DTensor path (engine_axis=0).
- dist_to_encoder()[source]
Returns a MixtureDataEncoder object for encoding sequences of iid observations from MixtureDistribution.
- Return type:
MixtureDataEncoder
- enumerator()[source]
Returns a MixtureEnumerator iterating the union of component supports in descending mixture probability order.
- Return type:
MixtureEnumerator
- quantized_index(max_bits, bin_width_bits=1.0)[source]
Build a bounded bit-quantized index from a global mixture frontier.
The primary path pulls candidates from weighted component enumerator heads. The log-sum of those heads bounds every unseen value, so construction stops when the live global frontier falls below
2**(-max_bits). This avoids the looser per-componentlog2(K)candidate expansion. If a component cannot enumerate, the method falls back to the structured cross-index path.
- quantized_count_index(quantizer, max_fine_bucket)[source]
BoundedCount for the MARGINAL mixture law: pool weight-scaled component count indices.
log p(x) = logsumexp_k (log w_k + log p_k(x)) has no exact structural count – overlapping component supports would need value-level deduplication. This builds the count semiring’s
plus-fold overscale(component_index, log w_k)instead, which:reaches a 2**M budget structurally (no enumeration), and
is a conservative UPPER bound – a value shared by several components is counted once per component, and each value is binned by its dominant weighted component (the tropical cost, within log2(K) bits of the exact logsumexp).
Every unranked value still carries its exact mixture
log_density(re-evaluated by the budget builder). For an exact small-budget index (best-first union with dedup), usequantized_index. Components that cannot count structurally raise EnumerationError.- Parameters:
max_fine_bucket (int)
- structural_fine_bucket(value, quantizer)[source]
Dominant weighted-component structural bucket (mirrors the plus-of-scaled-children index).
- Return type:
- is_canonical_copy(value, coarse_bin, quantizer)[source]
Stateless dedup: keep
valueonly at its dominant (best-weighted) component’s bin.The canonical bin is the coarse bin of the minimum, over components, of the component’s structural fine bucket shifted by the weight term. O(K) model evaluations, no state.
- class MixtureEnumerator(dist)[source]
Bases:
DistributionEnumerator- Parameters:
dist (MixtureDistribution)
- class MixtureSampler(dist, seed=None)[source]
Bases:
DistributionSampler- Parameters:
dist (MixtureDistribution)
seed (int | None)
- sample(size=None, *, batched=True)[source]
Draw iid samples from a mixture distribution.
The data type drawn from ‘comp_samplers’ is type T, corresponding to the data type of the mixture components.
If size is None, a single sample (of data type T) is drawn and returned. If size is not None, ‘size’-iid mixture samples are drawn and returned as a List with data type List[T].
With
batched=True(default) each component sampler is invoked once with the number of draws assigned to it and the results are scattered back into draw order. Because every component sampler owns an independentRandomState, this yields the same draws as the legacy per-draw loop (batched=False) but far faster.
- class MixtureAccumulator(accumulators, keys=(None, None), name=None, init='dirichlet')[source]
Bases:
SequenceEncodableStatisticAccumulator- Parameters:
- seq_update(x, weights, estimate)[source]
Vectorized update of sufficient statistics from encoded sequence of observations x.
Args value x is a sequence encoded sequence of mixture observations. The data type for each mixture observation is data type T. T1 is the data type produced by MixtureDataEncoder.seq_encode() function used to encode the sequence of type T observations.
Note: Requires a previous estimate of MixtureDistribution be passed. This may require seq_initialize() to be invoked prior to performing seq_update() calls.
Seq_update is similar to MixtureDistribution.seq_posterior(). Results are aggregated to comp_counts and accumulators.
- Parameters:
x (T1) – See above for details.
weights (np.ndarray) – Numpy array of positive floats.
estimate (MixtureDistribution) – MixtureDistribution object representing previous estimate from EM.
- Returns:
None.
- Return type:
None
- update(x, weight, estimate)[source]
Update sufficient statistics of MixtureAccumulator with weighted observation.
Requires previous estimate of MixtureDistribution.
Weights posterior of ‘estimate’ at x. Adds sum to comp_counts, then passes posterior[i] as weight for x into update() call of accumulator[i].
- Parameters:
x (T) – Observation of mixture distribution.
weight (float) – Weight for observation.
estimate (MixtureDistribution) – Previous iteration of EM estimate for MixtureDistribution.
- Returns:
None.
- Return type:
None
- initialize(x, weight, rng)[source]
Initialize MixtureAccumulator object with weighted observation x.
If _init_rng is False, _acc_rng is set with rng. This is done for consistency in initialize and seq_initialize functions.
Initialize mixture weights with a sample from Dirichlet distribution. Each SequenceEncodableStatisticAccumulator is for the mixture components is initialized with a call to accumulator[i].initialize.
- Parameters:
x (T) – Observation of mixture distribution.
weight (float) – Weight for observation.
rng (RandomState) – Used to set _acc_rng if not previously set.
- Returns:
None.
- Return type:
None
- seq_initialize(x, weights, rng)[source]
Vectorized initialization of MixtureAccumulator object for sequence encoded observations x.
If _init_rng is False, _acc_rng is set with rng. This is done for consistency in initialize and seq_initialize functions.
Vectorized implementation of initialize(), for sequence encoded x.
- Parameters:
x (T1) – Sequence encoded observations of mixture distribution.
weights (ndarray[float]) – Numpy array of positive valued floats.
rng (RandomState) – Used to set _acc_rng if not previously set.
- Returns:
None.
- Return type:
None
- combine(suff_stat)[source]
Merge the sufficient statistics of suff_stat with MixtureAccumulator instance.
- Arg suff_stat is a Tuple of length two containing,
suff_stat[0] (ndarray[float]): Aggregated component counts, suff_stat[1] (Tuple[T2,…]): Tuple of K sufficient statistics for the mixture components.
Note: The components of the mixture are assumed to have sufficient statistics of type T2.
- value()[source]
Returns sufficient statistics of MixtureAccumulator instance.
- The sufficient statistics value returned (suff_stat) is a Tuple of length two containing,
suff_stat[0] (ndarray[float]): Aggregated component counts, suff_stat[1] (Tuple[T2,…]): Tuple of K sufficient statistics for the mixture components.
Note: The components of the mixture are assumed to have sufficient statistics of type T2.
- from_value(x)[source]
Set sufficient statistics of MixtureAccumulator instance to x.
- The sufficient statistics value ‘x’ is a Tuple of length two containing,
x[0] (ndarray[float]): Aggregated component counts, x[1] (Tuple[T2,…]): Tuple of K sufficient statistics for the mixture components.
Note: The components of the mixture are assumed to have sufficient statistics of type T2.
- scale(c)[source]
Scale component counts and delegate child sufficient statistics.
- Parameters:
c (float)
- Return type:
MixtureAccumulator
- key_merge(stats_dict)[source]
- Combine the sufficient statistics of MixtureAccumulator instance with other MixtureAccumulator that have
matching weight or component keys.
Arg passed stats_dict is Dict[str, Union[np.ndarray, Tuple[T2,…]]. If the key is weight key, stats_dict value is a numpy array of floats containing component counts for a Mixture. If the key is a component key, the value is a list of SequenceEncodableStatisticAccumulator objects corresponding to the Mixture components.
- key_replace(stats_dict)[source]
- Replace the sufficient statistics of MixtureAccumulator instance with sufficient statistics of matching
weight and/or component keys found in stats_dict.
Arg passed stats_dict is Dict[str, Union[np.ndarray, Tuple[T2,…]]. If the key is weight key, stats_dict value is a numpy array of floats containing component counts for a Mixture. If the key is a component key, the value is a list of SequenceEncodableStatisticAccumulator objects corresponding to the Mixture components.
- acc_to_encoder()[source]
Returns a MixtureDataEncoder object for encoding sequences of iid observations from MixtureDistribution.
- Return type:
MixtureDataEncoder
- class MixtureAccumulatorFactory(factories, keys=(None, None), name=None, init='dirichlet')[source]
Bases:
StatisticAccumulatorFactory- Parameters:
- make()[source]
Return MixtureAccumulator object with SequenceEncodableStatisticAccumulator objects for the components and keys passed.
- Return type:
MixtureAccumulator
- class MixtureEstimator(estimators, fixed_weights=None, suff_stat=None, pseudo_count=None, name=None, keys=(None, None), prior=None, w_min=0.0, robust=False, init=None)[source]
Bases:
ParameterEstimator- Parameters:
- accumulator_factory()[source]
Returns MixtureAccumulatorFactory object passing component StatisticAccumulatorFactory objects and keys.
- Return type:
MixtureAccumulatorFactory
- get_prior()[source]
Return the joint mixture prior, or
Nonefor a plain MLE estimator.When a weight prior is attached the joint prior is the
(weight_prior, tuple(component priors))pair produced bymixture_prior().- Return type:
SequenceEncodableProbabilityDistribution | None
- set_prior(prior)[source]
Attach a weight prior (and optional per-component priors).
With a (symmetric) Dirichlet weight prior the estimator switches to the conjugate MAP weight update; component priors, when supplied, are delegated to each component estimator via
estimator.set_prior(those carry out their own conjugate updates).prior=Noneleaves the estimator a plain MLE estimator (byte-identical behaviour).- Parameters:
prior (SequenceEncodableProbabilityDistribution | None)
- Return type:
None
- model_log_density(model)[source]
Log density of the model parameters under this estimator’s prior (ELBO global term).
Returns the Dirichlet weight-prior log-density evaluated at
model.wplus the sum of each component estimator’smodel_log_densityat the corresponding component model. Returns0.0for a plain MLE estimator with no priors anywhere.- Parameters:
model (MixtureDistribution)
- Return type:
- estimate(nobs, suff_stat)[source]
Estimate MixtureDistribution from aggregated sufficient statistics.
- Args suff_stat is a Tuple length two containing:
suff_stat[0] (np.ndarray): Sufficient statistic for the weights of the mixture components. suff_stat[1] (Tuple[T2, …]): A tuple of length K (number of mixture components), containing the
sufficient statistics of each mixture component of data type T2.
- If fixed_weights is not None, suff_stat[0] is not used and the weights of the MixtureDistribution are set to
fixed_weights.
If pseudo_count is passed, arg suff_stat[0] is aggregated with re-weighted member variable suff_stat. If member variable suff_stat is None, then the arg suff_stat[0] is re-weighted with pseudo_count to estimate the weights.
If pseudo_count is None, ar suff_stat[0] is used to estimate the wieghts.
- class MixtureDataEncoder(encoder)[source]
Bases:
DataSequenceEncoder- Parameters:
encoder (DataSequenceEncoder | Sequence[DataSequenceEncoder])
- seq_encode(x)[source]
Sequence encode a sequence of iid observations drawn from the mixture distribution.
For a homogeneous mixture this delegates to the single shared component encoder. For a heterogeneous mixture each component encoder encodes the data separately and the encodings are bundled in a
_HeteroMixtureEncodedwrapper.- Parameters:
x (Sequence[T]) – A Sequence of iid observations drawn from a mixture distribution with component distributions consistent with the per-component encoders.
- Returns:
Encoded sequence (single shared encoding, or a per-component wrapper).
- Return type:
- class MixtureFisherView(dist)[source]
Bases:
FixedFisherViewComplete-data Fisher view for finite mixture distributions.
Coordinates are component assignment indicators followed by each component’s sufficient statistics gated by that assignment. Observed data map to posterior-expected complete-data statistics.
- Parameters:
dist (Any)
- structured_statistics(x, estimate=None, weight=1.0)[source]
Structured sufficient stats for one observation.
For latent-variable distributions, this is the posterior-expected complete-data sufficient statistic under estimate (or this view’s distribution when estimate is omitted).