mixle.inference.estimation module

Functions for estimating and validating mixle models from observed data.

Useful functions for estimating mixle ‘SequenceEncodableProbabilityDistributions’ from ‘ParameterEstimator’ objects.

class EMStep(iter, model, log_density, delta)[source]

Bases: NamedTuple

One accepted EM iteration, handed to an optimize(on_step=...) callback.

iter is 1-based; model is the current accepted model – snapshot it to checkpoint, and resume with optimize(prev_estimate=...); log_density is the training objective at this step; delta is its gain over the previous step (inf on the first iteration).

Parameters:
iter: int

Alias for field number 0

model: Any

Alias for field number 1

log_density: float

Alias for field number 2

delta: float

Alias for field number 3

optimize(data, estimator=None, max_its=10, delta=1.0e-9, init_estimator=None, init_p=0.1, rng=None, prev_estimate=None, vdata=None, enc_data=None, enc_vdata=None, out=sys.stdout, print_iter=1, num_chunks=1, engine=None, precision=None, fields=None, resources=None, placement=None, sub_chunks=1, chunk_size=None, backend='local', num_workers=None, client=None, comm=None, root=0, root_only=False, strategy=None, reuse_estep_ll=True, objective='auto', on_step=None, structure='auto')[source]
Estimation of ‘estimator’ via EM algorithm for max_its iterations or until

new_loglikelihood - old_loglikelihood < delta.

Parameters:

  • data (Optional[List[T]]) – List of data type T containing observed data. Must be compatible with data type of estimator.

  • estimator (ParameterEstimator | ProbabilityDistribution | None) – What to fit. A ParameterEstimator is used as-is; a distribution prototype (any ProbabilityDistribution) is coerced to its matching estimator via proto.estimator() so you build the model shape only once; None infers an estimator from raw data (mixle.utils.automatic.get_estimator).

  • max_its (int) – Maximum number of EM iterations to be performed. Default value is 10 iterations.

  • delta (Optional[float]) – Stopping criteria for EM algorithm used if max_its is not set: Iterate until |old_loglikelihood - new_loglikelihood| < delta or iterations == max_its.

  • init_estimator (Optional[ParameterEstimator]) – ParameterEstimator to used to initialize EM algorithm parameters. If None, estimator is used. Must be consistent with estimator.

  • init_p (float) – Value in (0.0,1.0] for randomizing the proportion of data points used in initialization.

  • rng (RandomState) – RandomState used to set seed for initializing EM algorithm.

  • vdata (Optional[Sequence[T]]) – Optional validation set.

  • prev_estimate (Optional[SeqeuenceEncodableProbabilityDistribution]) – Optional model estimate used from prior fitting. Must be consistent with estimator.

  • enc_data (Optional[List[Tuple[int, E]]]) – Optional encoded data of form List[Tuple[int, E]]. Formed from data if None.

  • enc_vdata (Optional[List[Tuple[int, E0]]]) – Optional sequence encoded validation set.

  • out (IO) – IO stream to write out iterations of EM algorithm. Pass out=None to silence all output.

  • print_iter (int) – Print the log-likelihood difference every print_iter iterations; the final converged iteration is always reported. Pass print_iter=0 to suppress the periodic lines (keeping only the converged line), or out=None to silence entirely.

  • num_chunks (int) – Number of chunks for encoded data.

  • engine (Optional[Any]) – Optional ComputeEngine for local kernel scoring/accumulation. Distributed engine placement is intentionally deferred to the orchestrator/planner layer.

  • precision (Optional[Any]) – Optional floating-point precision such as 'float32' or np.float64. Pass 'auto' to let mixle.engines.auto_precision choose from the data and engine: float32 only on a GPU torch engine with well-conditioned numeric data, else float64. Pass 'minimal' for the data-aware CPU allocator (mixle.inference.precision_plan): it inspects the data magnitude and the model’s leaf families/conditioning and runs the reduced float32 fused kernel where verified safe (accumulation stays float64), else float64 – the “preserve accuracy with minimal compute” default for local fits.

  • fields (Optional[Any]) – DataFrame column/field selection. A single field yields scalar observations; several fields yield tuple observations unless the estimator/model is record-shaped, in which case dict records are produced by source column name.

  • resources (Optional[Any]) – Optional planner resources. When supplied with raw data, optimize encodes through the shared encoded-data factory so placement, sub-chunks, and per-shard engines use the orchestrator contract.

  • placement (Optional[Any]) – Optional explicit placement produced by mixle.utils.parallel.planner.plan.

  • sub_chunks (int) – Number of sub-chunks per placement shard when resources or placement is supplied.

  • chunk_size (Optional[int]) – Approximate chunk size for ordinary local sequence encoding.

  • backend (str) – Encoded-data backend for raw data. 'local' keeps the historical local encoding unless resources/placement are supplied; 'mp' and 'mpi' use the shared encoded-data factory.

  • num_workers (Optional[int]) – Worker count for backend='mp' and optional partition count hint for backend='dask'.

  • client (Optional[Any]) – Existing dask.distributed client for backend='dask'. If omitted, the dask backend uses an active default client or starts a local threaded client.

  • comm (Optional[Any]) – MPI communicator for backend='mpi'.

  • root (int) – MPI root rank for backend='mpi'.

  • root_only (bool) – MPI root-only data mode for backend='mpi'.

  • strategy (Optional[Any]) – Optional EM strategy from mixle.inference.em (e.g. AnnealedEM, HardEM, MonteCarloEM) or any callable (enc, estimator, model) -> model to use in place of the standard exact E/M step. None uses the standard step.

  • reuse_estep_ll (bool) – Default True. Reuse the data log-likelihood computed during the E-step (the posterior normalizer / forward pass / variational ELBO) for convergence instead of running a separate scoring pass each iteration – typically ~1.5-2x faster per iteration for latent models (mixtures, HMMs and variants, topic models, associations, IBP, …) on the default local engine. Convergence then lags by one iteration (same fixed point) and the best-likelihood model is returned; fixed-iteration fits (delta=None) are identical to the standard loop. Automatically falls back to the standard loop for engines/strategies/ distributed backends or models that can’t report the LL (no slowdown there). Set False to force the exact historical per-iteration scoring behavior.

  • objective (str) – Convergence/selection objective. 'auto' (default) makes the prior the single switch – a model exposing a variational ELBO (seq_local_elbo) is fit by variational Bayes ('vb'), an estimator carrying a parameter prior by penalized log-likelihood ('map'), and everything else by plain maximum likelihood ('mle'). Pass 'mle' / 'map' / 'vb' to force a specific objective. fit accepts the same argument; both share this resolution so a Bayesian estimator is fit on the correct objective regardless of the verb used. (Only 'mle' is compatible with the fused E-step shortcut; reuse_estep_ll is ignored for 'map'/'vb'.)

  • on_step (Optional[Callable[[EMStep], None]]) – Optional per-iteration callback receiving an EMStep (iter, model, log_density, delta) for the accepted model. Use it to checkpoint a long run – e.g. on_step=registry.checkpointer('run', every=5) – and resume with prev_estimate=. Called on every iteration regardless of print_iter.

  • structure (str) – 'auto' (default) makes the tagline literal for flat tuple records fit with no estimator: the cross-field dependency graph is discovered (mixle.inference.learn_bayesian_network()) and returned WHEN it beats the independent composite by BIC — otherwise (no edges, non-record data, or any failure) the historical automatic-composite path proceeds untouched. 'off' restores the unconditional historical behavior. Only consulted when estimator is None and no prev_estimate/init_estimator/strategy/enc_data is supplied.

Returns:

SequenceEncodableProbabilityDistribution corresponding to estimator when stopping criteria of EM algorithm

is met.

Return type:

SequenceEncodableProbabilityDistribution

fit(data, estimator=None, max_its=10, delta=1.0e-6, init_estimator=None, **kwargs)[source]

Fit a model in the Bayesian (variational / MAP) sense, returning the posterior-bearing model.

This is the posterior-returning counterpart of optimize(). fit iterates the EM/VB update that maximizes the objective selected by objective (default 'auto'):

  • 'auto' – the prior is the single switch: 'vb' when the model exposes seq_local_elbo, 'map' when the estimator carries a parameter prior, else 'mle';

  • 'mle' – plain data log-likelihood (ignores any prior in the objective);

  • 'map' / 'vb' – penalized log-likelihood / ELBO obj = data term + prior term, where the data term is the observed-data LL (MAP) or local-ELBO contributions (variational), and the prior term is estimator.model_log_density(model).

Convergence is checked on the chosen objective, so under 'map'/'vb' the prior is part of the stopping rule and conjugate updates never decrease it. The returned model carries its conjugate posterior forward as model.get_prior(). With no prior anywhere, every objective reduces to plain EM, so fit and optimize agree for frequentist estimators. optimize accepts the same objective argument; the two share this resolution so a Bayesian estimator is fit correctly regardless of which verb the caller reaches for.

Args otherwise mirror optimize() (local encoded path). Returns the model with the best validation log-likelihood seen during the run.

fit is a thin wrapper over optimize() – they share the one EM/objective loop. fit adds only the opt-in data-structure check, a Bayesian-leaning default delta (1e-6), and the exact per-iteration-scored loop (reuse_estep_ll=False). Every other optimize() keyword – engines, precision, distributed backend, on_step, the fused E-step – is accepted here too and forwarded verbatim, so reaching for a heavier knob never means switching verbs. estimator accepts the same three spellings as optimize() (estimator, distribution prototype, or None to infer from data).

Parameters:

  • data (Sequence[T] | None)

  • estimator (ParameterEstimator | ProbabilityDistribution | None)

  • max_its (int)

  • delta (float | None)

  • init_estimator (ParameterEstimator | ProbabilityDistribution | None)

  • kwargs (Any)

Return type:

SequenceEncodableProbabilityDistribution

best_of(data, vdata, est, trials, max_its, init_p, delta, rng, init_estimator=None, enc_data=None, enc_vdata=None, out=sys.stdout, print_iter=1, reuse_estep_ll=True, objective='auto')[source]
Performs EM algorithm for trials-number of randomized initial conditions. Returns the best model fit in terms of

maximum log-likelihood value from validation data.

Parameters:

  • data (Optional[List[T]]) – List of data of type T. If None is given, enc_data must be provided as List[Tuple[int, enc_data_type]].

  • vdata (Optional[Sequence[T]]) – Optional validation set.

  • est (ParameterEstimator) – ParameterEstimator for model to be estimated.

  • trials (int) – Integer number >= 1, of randomized initial conditions to perform EM algorithm for.

  • max_its (int) – Integer value >=1, sets the maximum number of iterations of EM to be performed as stopping criteria.

  • init_p (float) – Value in (0.0,1.0] for randomizing the proportion of data points used in initialization.

  • delta (float) – Stopping criteria for EM when |old-log-likelihood - new-log-likelihood| < delta.

  • rng (RandomState) – RandomState for setting seed.

  • init_estimator (Optional[ParameterEstimator]) – Optional ParameterEstimator used for fitting.

  • enc_data (Optional[List[Tuple[int, E]]]) – Optional encoded data, if provided data need not be provided. If None, enc_data is set from data.

  • enc_vdata (Optional[List[Tuple[int, E0]]]) – Optional sequence encoded validation set.

  • out (I0) – Text output stream.

  • print_iter (int) – Print iterations (i.e. log-likelihood difference) every print_iter-iterations.

  • reuse_estep_ll (bool) – Default True. Forwarded to each trial’s optimize call – reuse the E-step likelihood for convergence instead of a separate scoring pass (see optimize). Set False to force the exact historical per-iteration scoring behavior.

  • objective (str) – Convergence/selection objective forwarded to each trial’s optimize call; 'auto' (default) selects MLE / MAP / variational Bayes from the prior (see optimize).

Returns:

Tuple of log-likelihood of best fitting model and the best fitting model from number of trials.

Return type:

tuple[float, SequenceEncodableProbabilityDistribution]

constant(rho)[source]

Return a constant streaming step-size schedule.

Parameters:

rho (float)

harmonic(alpha, offset=1.0)[source]

Return rho_t = (offset + t - 1)^(-alpha) for streaming EM.

Parameters:
posterior_carry()[source]

Return the recursive-conjugate streaming mode name.

In posterior_carry mode each fitted posterior becomes the next batch’s prior, i.e. the stream performs exact recursive Bayesian updating: the conjugate posterior after batch t is fed in as the conjugate prior for batch t + 1.

Return type:

str

forgetting(rho)[source]

Return a constant forgetting / power-prior schedule for streaming Bayes.

rho in (0, 1] down-weights each incoming batch’s sufficient statistics by a constant factor before the conjugate estimate call, so older evidence decays geometrically. rho=1 recovers ordinary (un-forgotten) accumulation.

Parameters:

rho (float)

class BayesianStreamingEstimator(estimator, mode='posterior_carry', schedule=None, model=None, init_estimator=None, init_p=0.1, rng=None, num_chunks=1)[source]

Bases: object

Streaming / recursive-Bayes driver over the mixle.stats estimator protocol.

mode='posterior_carry' (the default) performs exact recursive conjugate updating: each fitted posterior is carried forward as the next batch’s prior by rebuilding the estimator from the fitted model (model.estimator() returns an estimator whose prior is the model’s posterior).

mode='forgetting' applies a power-prior step: the current batch’s accumulated sufficient statistics are scaled by rho = schedule(step) (via the accumulator’s scale, which preserves structural support metadata such as a categorical’s min_val) before the ordinary conjugate estimate call, and the batch’s contribution to nobs is scaled to match.

The public surface is BayesianStreamingEstimator(estimator, mode=..., schedule=...) plus .update(data=None, enc_data=None) and .reset().

Parameters:

  • estimator (ParameterEstimator)

  • mode (str | None)

  • schedule (Any | None)

  • model (SequenceEncodableProbabilityDistribution | None)

  • init_estimator (ParameterEstimator | None)

  • init_p (float)

  • rng (RandomState | None)

  • num_chunks (int)

update(data=None, enc_data=None)[source]

Consume one batch and return the updated posterior-bearing model.

Parameters:
Return type:

SequenceEncodableProbabilityDistribution

reset()[source]

Drop the current model and stream counters.

Return type:

None