mixle.lifecycle module

mixle.Model – the model lifecycle as one object with consistent verbs.

Everything here exists elsewhere in the library; this facade makes the lifecycle discoverable without knowing which subpackage owns which verb:

m = mixle.propose(data)          # a model shape recommended from the data (with confidence + caveats)
m.fit(data)                      # inference chosen from the structure (EM / MLE / closed form)
m.evaluate(holdout)              # held-out scores
m.sample(5)                      # draw new records
m.enumerate().top_k(3)           # most-probable support (discrete/structured families)
m.posterior(x)                   # latent posteriors (mixtures, HMMs, ...)
m.distill(teacher, inputs)       # compact deployable student in front of the teacher (task spine)
m.deploy("artifacts/m")          # durable artifact directory; Model.load() restores it
m.explain()                      # what it is, what it supports, and how it was proposed
m.explain_prediction(x)          # exact per-part attribution of one score
m.forecast(history, h)           # horizon predictions, model-predictive MC intervals (HMMs)
m.do({field: value})             # graph-surgery intervention (learned Bayesian networks)
m(x)                             # use it: log-density of an observation

Model wraps a prototype distribution, an estimator, or nothing (the estimator is inferred from the data); verbs delegate to mixle.inference.optimize(), dist.enumerator(), mixle.task.solve, and mixle.describe(). It adds no new inference – only one place to stand.

json_read_back_failure(text)[source]

Why the JSON in text would fail to decode, or None when it reads back cleanly.

Encoding is not evidence of a round trip. to_serializable accepts any registered class’s __dict__, while the decoder additionally requires that state to reconstruct through the class’s own constructor – so a family whose estimator pins a fit annotation onto the fitted object, or whose state names a parameter differently from the constructor’s, encodes cleanly and then refuses to load. The only honest test of “can this be read back” is to read it back, against the exact text about to be written.

The decode runs inside trusted_deserialization on purpose. This payload was encoded from a live object in this very process moments ago; the trust gate exists to stop an artifact that arrived from somewhere else from executing code, and it has nothing to say about our own in-memory model. Probing untrusted would report every model carrying an embedded torch module as unreadable and demote a perfectly good JSON artifact to a pickle.

Parameters:

text (str)

Return type:

str | None

class DeployedArtifact(path, *, format, format_fallback)[source]

Bases: str

The artifact path Model.deploy() returns, annotated with what was written there.

A plain str (every os.path / Path use keeps working), plus the two facts a caller needs at the call site rather than by re-opening the manifest: format ("json" or "pickle") and format_fallback (None for JSON; otherwise the sentence saying why this artifact is a pickle and therefore costs its readers trust_code=True). A deploy pipeline can assert result.format == "json" to enforce a no-pickle policy in one line.

Parameters:

  • path (str)

  • format (str)

  • format_fallback (str | None)

Return type:

DeployedArtifact

saddle_suspect(fitted, data, *, sample=200, tol=0.02)[source]

Family-agnostic symmetric-saddle check for latent-variable fits.

At the symmetric saddle every component is identical, so every observation’s component posterior is (numerically) uniform. Suspect when, over a data sample, NO observation’s posterior deviates from uniform by more than tol. Non-latent models (no posterior/components) return False.

Parameters:
Return type:

bool

class Model(spec=None, *, notes=None)[source]

Bases: object

One object over the model lifecycle: build / fit / evaluate / enumerate / distill / deploy / use.

Parameters:

  • spec (Any)

  • notes (list[str] | None)

fit(data, *, restarts='auto', calibrate=False, **optimize_kw)[source]

Fit via mixle.inference.optimize(); the algorithm follows from the model’s structure.

The fit iterates EM to its tolerance: unless the caller passes max_its=, the iteration cap defaults to 500 here (optimize’s own default of 10 regularly stops a mixture fit on the initialization plateau – 58 nats short of the optimum on Old Faithful – with no signal). The delta stopping rule ends a converged fit long before the cap; a fit that does hit the cap without converging is disclosed in notes and in self._fit_info (n_iter, converged, read from the model’s FitProvenance() receipt). Passing delta=None through **optimize_kw requests a fixed iteration count instead – see mixle.inference.optimize()’s own delta docs for the caveat that a rejected (non-improving) EM step still ends the loop early even then, which optimize discloses via a UserWarning (not notes) when it happens.

A spec that is a prototype distribution also supplies the EM starting point: its parameter values are honored as the initialization (optimize’s bare prototype coercion keeps only the structure and draws a random subsample start). Pass prev_estimate= yourself to override the start explicitly.

restarts="auto" (default) makes latent-variable fitting genuinely automatic: after the plain fit, a family-agnostic saddle check runs (a mixture stuck at the symmetric saddle gives every observation a ~uniform component posterior) – and a latent fit that exhausted its iteration cap without converging is treated as equally suspect. On suspicion the fit reruns from diversified initializations (hard-partition component starts, falling back to mixle.inference.best_of()), keeping the better log-likelihood and recording what happened in notes. Pass an int to force that many restarts up front, or restarts=None for the raw single fit.

calibrate (opt-in, default off): reserve a holdout slice (a fraction, or True for 25%), fit on the rest, and attach a CalibrationReport on self.calibration measuring calibration quality on held-out data (PIT test + held-out log-density). Off by default because it costs training data.

Neither evidence step can break a fit, but neither silently disappears either: self.evidence carries a typed succeeded/failed/not_applicable record per step, naming the failure class and message when one failed, and a failure also lands in notes (so explain() shows it). certificate=None alone cannot tell you whether certification was never applicable or raised.

Parameters:
Return type:

Model

evaluate(data)[source]

Held-out fit quality: total and mean log-density over data.

A -inf entry is not raised on the way NaN/+inf are (see below): it is not necessarily a scoring bug. The common, DOCUMENTED cause is a field auto-inference froze as an identifier-like/high-cardinality categorical (mixle.utils.automatic.get_estimator, also reached through optimize(df)’s estimator=None path) meeting, here, a label its training split never saw – CategoricalDistribution’s documented default_value=0.0 scores that at -inf by design. Silently averaging it into mean_log_density would still hide the cause, so this warns (naming the affected rows) rather than staying quiet – but does not raise, unlike NaN/+inf, both of which are unconditionally scorer bugs.

Parameters:

data (Any)

Return type:

dict[str, Any]

sample(size=None, *, seed=None)[source]

Draw samples from the fitted distribution.

Parameters:

  • size (int | None)

  • seed (int | None)

Return type:

Any

enumerate()[source]

The fitted distribution’s enumerator (top-k / top-p / rank / seek), where supported.

Return type:

Any

posterior(x)[source]

Latent posterior probabilities for one observation (mixtures, HMMs, …), where supported.

A component-latent model (a mixture) answers with its own posterior: the (k,) component probabilities for x. A chain-latent model (an HMM), whose x is one observation sequence, has no posterior method – the module docstring’s promise used to die there as a bare AttributeError – so it answers through latent_posterior: the (T, k) forward-backward smoothing marginals, one state distribution per timestep. The richer chain-posterior object (Viterbi mode(), FFBS sample(), entropy()) stays available as model.fitted.latent_posterior(x).

A model with no latent state (a plain Gaussian) raises AttributeError naming the two supported shapes instead of the bare delegation failure.

Parameters:

x (Any)

Return type:

Any

distill(teacher=None, inputs=None, **solve_kw)[source]

Distill a compact deployable student via mixle.task.solve().

With teacher=None the fitted model itself teaches: inputs are labeled by their most-probable latent component (posterior argmax), so a fitted mixture becomes a fast, calibrated classifier of its own clusters. Returns a mixle.task.Solution (call it, report(), improve()).

Parameters:
deploy(path)[source]

Persist a durable artifact directory (model + manifest); Model.load() restores it.

A registry-serializable model is written as safe, type-tagged JSON (model.json); a model the serialization registry cannot represent falls back to a pickle (model.pkl). “Pure statistical” does not guarantee “JSON”: besides the expected torch-backed leaves, any base-install family the registry has no JSON form for takes the pickle path too (the Bernoulli-set, Thurstone, and Spearman ranking families – models propose() itself selects for ordinary set-valued and ranking data – deployed that way until their 0.8.0 codec repair; they now write JSON). Two whole modules still fall back this way by design, not regression: every family in mixle.stats.directional (VonMises, WrappedCauchy, WrappedNormal, VonMisesFisher, Kent, Watson, Bingham, ProjectedNormal) and mixle.stats.matrix (Wishart, InverseWishart, MatrixNormal, LKJ) lacks the __pysp_setstate__ hook a family needs to round-trip through JSON, so deploy/load always take the pickle path for them today, pending a future implementation. Never assume the format: read it from the returned path’s format attribute (the return value is a DeployedArtifact, a plain str annotated with format and format_fallback), from the warning a fallback raises here, or from the manifest.

“Cannot represent” is decided by trying it, not by predicting it: the JSON is decoded again before the artifact is accepted, so a model that encodes cleanly and then refuses to load – the state whole families were in, deployed successfully and readable by nothing – takes the pickle path instead of becoming a write-only artifact. Any such fallback is disclosed three ways rather than taken silently: a warning here, format_fallback in the manifest naming the model and the exact read-back error, and a note on the model load() returns. It costs the reader a trust_code=True, so it should be visible; if that is unacceptable for a given family, the fix is a serialization hook on the family, not a quieter deploy.

This is a model export, not an evidence export. The artifact carries the fitted distribution, notes, and the fit record (n/when/artifact_hash) – and nothing else. The EstimationCertificate on certificate, the CalibrationReport on calibration and the candidate frontier are not serialized and do not survive a round trip; a deployed model comes back able to predict but unable to show how it was chosen or how well it was calibrated. Whatever evidence was present at deploy time is listed in the manifest’s evidence_not_exported and reported as a note on the loaded model, so the loss is visible rather than silent. Keep those receipts alongside the artifact yourself if a consumer needs to judge the model, not just run it.

The model file is written first, then hashed, and the manifest naming it (model_file) and binding its model_sha256 is promoted atomically last. A redeployment removes the previous generation’s other-format model file, so an artifact directory never holds a stale model.pkl next to a fresh model.json, and load() re-checks the digest before deserializing anything.

Parameters:

path (str)

Return type:

DeployedArtifact

classmethod load(path, *, trust_code=False)[source]

Restore a Model from an artifact directory created by deploy().

Warning

A pickle-format artifact (a torch-backed model, or one written by an older mixle) executes arbitrary code from the file when loaded, exactly like pickle.load on any untrusted input. JSON-format artifacts are deserialized through a type-tagged registry that never imports or executes code from the payload – unless the model contains a NeuralLeaf-family component, whose weights are embedded as a pickle blob inside the otherwise-safe JSON (see mixle.models._neural_serial); loading one of those also executes code. There is no way to tell which case applies without attempting the load.

Because of this, load refuses to run any code-executing path unless the caller passes trust_code=True – an explicit statement that the artifact’s source is trusted. Without it, a pickle-format artifact raises immediately, and a JSON artifact raises only if it actually contains an embedded module (a pure-statistical model still loads normally). trust_code=False, None and 0 are all read as “no trust” (an artifact that needs no trust loads under any of them); any other value – a truthy string, 1 – is rejected loudly, because only the exact True may authorize code execution.

When the manifest records a model_sha256 (every artifact written by this version does), the named model file is hashed and checked against it before anything is deserialized, so a model file edited, truncated or swapped after deployment – while the manifest stays intact – is refused rather than loaded. An older manifest without that field is loaded unverified, exactly as before – there is nothing to check it against.

What these checks are, and are not. All of them read the manifest itself as ground truth: the digest binds the manifest to the model file’s bytes, the family check binds its self-description to what those bytes deserialize to, and the model_content_hash cross-check (when recorded) compares the deserialized model’s content hash against the one stamped at deploy time – a mismatch there means the manifest’s fit record and the model being served may not come from the same deployment, and it is disclosed as a warning and a note rather than a refusal (a content-hash recomputed by a different mixle version can legitimately drift; refusing would brick valid artifacts). None of this authenticates the manifest: a writer with write access to the artifact directory can rewrite the manifest and model together. Integrity against that threat needs a signature kept outside the directory (sign the artifact, or compare model_sha256 against a digest recorded elsewhere).

The restored Model carries the fitted distribution, notes and the fit record. The certificate, calibration report and candidate frontier are not in the artifact at all (see deploy()); when the deploying model had any of them, a note saying so is appended.

Parameters:
Return type:

Model

explain_prediction(x)[source]

Exact per-part attribution of log p(x) — mixle.inference.explain().

Parameters:

x (Any)

forecast(history, horizon, **kw)[source]

Horizon predictions with MODEL-PREDICTIVE Monte Carlo intervals — mixle.inference.forecast().

The intervals are the fitted model’s own predictive quantiles, not held-out-calibrated bands (STAT-P20-04); forecast_price is the recalibrated route.

Parameters:
do(interventions, **kw)[source]

Graph-surgery intervention — mixle.inference.do() (M0’s generic engine: dependency trees, Bayesian networks, composites, mixtures; reduces to mixle.inference.bn_do()’s exact behavior for a fitted HeterogeneousBayesianNetwork).

Parameters:
explain()[source]

What this model is, what it supports, and how it was proposed.

Return type:

str

propose(data, *, fit=False, llm=None, holdout=0.25, seed=0, max_its=25, max_candidates=None, timeout=None, **recommend_kw)[source]

Propose a model for data from a verified frontier of candidates and return the winner.

The winner comes back UNFITTED unless ``fit=True`` is passed. Candidate ranking fits every candidate internally (on the training split, scored held-out), but what returns by default is the winning specification: Model.fitted is None and the scoring verbs raise until fit(data) runs. propose(data, fit=True) is the one-call propose-and-fit form.

data is anything row-shaped: a list of records, a numpy array, a pandas DataFrame (one record per row across its columns – never a model of the column names), a mapping of equal-length columns, a DataSource, or a one-shot iterator (materialized once).

Two paths return an unverified winner, and both are honestly disclosed (never silently). max_candidates=0 or timeout=0.0 skips every candidate: the winner falls back to the raw heuristic recommendation, with a "search budget: skipped ..." line in Model.notes and a "skipped": "search budget reached (...)" entry per candidate in Model.frontier, both readable through explain(). And when every candidate fails to verify (scoring errors, or every fit rejected as degenerate), the same fallback applies with a "no candidate could be verified: ..." note naming each failure. On BOTH routes, with fit=True the evidence["certificate"] record is downgraded from succeeded to attempted – a certificate produced for a winner nothing verified out-of-sample is not verification, whether the candidates failed or the budget skipped them. Any other max_candidates/timeout still verifies every candidate it has budget for before choosing a winner.

The verified frontier is guarded against degenerate wins: a candidate whose held-out score is a likelihood spike – a continuous fit whose scale collapsed onto a repeated value, detected by a positive log-density spike (max over held-out rows above 0, whether localized or median-wide, per the median) combined with pathological PIT calibration (total-variation error >= 1.0 of the 1.8 maximum) – is rejected from the frontier with a note, and the win falls to the best non-degenerate candidate. The criterion is deterministic and never fires on a model without a scalar predictive CDF (see _degenerate_likelihood_spike).

A joint (multi-field) candidate’s held-out score is also guarded against one field wrongly voiding every other, well-behaved field: when a field legitimately meets a held-out label its training split never saw, CategoricalDistribution’s documented default_value=0.0 scores that row at -inf (an identifier-like/high-cardinality field with a long tail is the common real-world trigger). That field’s contribution is excluded from the candidate’s held-out score – not the whole candidate – with a note naming the field and how many held-out rows it affected (see _unseen_label_rescue); an unexplained non-finite score still fails the candidate outright.

For scalar data the candidate universe follows the ELEMENT TYPE, and the choice is recorded in Model.notes: integer-typed values are modeled by discrete families (categorical / Poisson / binomial …), float-typed values by continuous densities – the two universes score in different units (log-mass vs log-density) and cannot be ranked against each other, so passing .astype(float) on a count column is an affirmative statement that it is continuous.

Candidates come from every proposer the library has — the heuristic recommendation (mixle.task.recommend.recommend_model(), dependency-aware in the narrow sense of a joint multivariate-Gaussian candidate for fully-observed numeric vector rows), a structured-search candidate (mixle.inference.optimize() called with no pre-built estimator — its own no-estimator auto-structure-search, the fuller copula/learned-Bayesian-network dependence upgrade described in The Automatic-Modeling Contract’s “Dependence between fields”), the plain independence baseline (mixle.utils.automatic.get_estimator()), and, when an llm handle is given, an LLM-designed structure (mixle.task.design.design_model(), allowlisted-spec, fit-validated). Each candidate is generated from the training split only, fitted on it, and scored on held-out data – the outer holdout is invisible to every proposer until final ranking (STAT-RR18-01), so the ranking is out-of-sample, not a guess. The winner becomes the returned Model; the full ranking lands in Model.frontier and the per-field confidence / dependency / candidate notes in Model.notes (shown by explain()). Pass fit=True to also fit the winner to all of data before returning.

The frontier search is bounded by max_candidates (evaluate at most this many candidates, in proposer order — the heuristic recommendation is always first) and timeout (stop starting new candidate fits once this many wall-clock seconds have elapsed). Both default to None (unbounded). A candidate skipped for budget is recorded in Model.frontier and Model.notes, never silently dropped, so a bounded search reports exactly what it did not evaluate.

Parameters:
Return type:

Model