mixle.task.design module

LLM-designed models: let an LLM propose the mixle structure from data – and make mixle validate it.

The hardcoded auto-estimator (mixle.task.recommend.recommend_model()) is a fine fallback, but the differentiator is flexibility: an LLM can read a data profile and propose a structure no fixed heuristic encodes – a mixture here, a heavy-tailed leaf there, a composite of mixed families. The risk with “LLM picks the model” is hallucination; the answer is grounding. The LLM emits a small, allowlisted JSON spec (no code, no eval); spec_to_estimator() validates and builds a real mixle estimator; design_model() checks its shape against the observations, fits it without the holdout, and compares independent held-out density with a separately fitted heuristic baseline before trusting it.

So the LLM proposes and mixle disposes. design_model(data, llm) returns the chosen estimator, the accepted spec/source, an acceptance receipt, and structured evidence about a rejected proposal.

spec_to_estimator(spec)[source]

Build a mixle estimator from an allowlisted spec dict (recursively); raise on anything off the allowlist.

Specs:

  • {"family": "<name>"} – a scalar leaf (see ALLOWED_FAMILIES);

  • {"type": "composite", "fields": [spec, ...]} – a tuple record of sub-models;

  • {"type": "mixture", "k": K, "component": spec} – a K-component mixture of the component model.

Parameters:

spec (dict[str, Any])

Return type:

Any

data_profile(data, *, max_rows=500)[source]

A compact, family-agnostic description of data for the LLM: per-field kind and a few sample values.

Parameters:
Return type:

dict[str, Any]

class DesignAcceptanceReceipt(accepted, schema_compatible, n_train, n_holdout, train_mean_log_density, holdout_mean_log_density, baseline_holdout_mean_log_density, holdout_regret, max_holdout_regret, reason)[source]

Bases: object

Independent train/holdout evidence for one proposed or fallback estimator.

Parameters:

  • accepted (bool)

  • schema_compatible (bool)

  • n_train (int)

  • n_holdout (int)

  • train_mean_log_density (float)

  • holdout_mean_log_density (float)

  • baseline_holdout_mean_log_density (float)

  • holdout_regret (float)

  • max_holdout_regret (float)

  • reason (str)

class DesignFailureReceipt(stage, error_type, message, proposed_spec, reply_excerpt, acceptance=None)[source]

Bases: object

Structured evidence retained when an LLM proposal is rejected.

Parameters:

  • stage (str)

  • error_type (str)

  • message (str)

  • proposed_spec (dict[str, Any] | None)

  • reply_excerpt (str)

  • acceptance (DesignAcceptanceReceipt | None)

class DesignedModel(estimator, spec, source, note='', acceptance=None, failure=None)[source]

Bases: object

Chosen estimator, provenance, and independent acceptance/rejection evidence.

Parameters:

  • estimator (Any)

  • spec (dict[str, Any] | None)

  • source (str)

  • note (str)

  • acceptance (DesignAcceptanceReceipt | None)

  • failure (DesignFailureReceipt | None)

fit(data, **kwargs)[source]

Fit the designed estimator with mixle.inference.optimize.

Parameters:
Return type:

Any

design_model(data, llm, *, fallback=True, validate_rows=200, holdout_frac=0.25, validation_seed=17, validation_max_its=10, max_holdout_regret=1.0)[source]

Accept an LLM design only on schema-compatible independent predictive evidence.

Roles are fixed once, in this order, and never re-drawn (MXR-080-1891): the deterministic subset is split into train and holdout FIRST; everything that influences the choice of model – the data profile the LLM designs from and the heuristic recommendation used both as the acceptance baseline and as the fallback structure – is derived from train only; the holdout is then read exactly once per candidate, to score it. So the LLM estimator sees only the training part and must produce finite holdout density no more than max_holdout_regret nats/observation below an independently fitted heuristic estimator. A fallback is returned only after it passes the same fit/holdout validity checks. Rejected proposals remain available in DesignedModel.failure.

Parameters:
Return type:

DesignedModel