mixle.evolve package

mixle.evolve – the self-improvement core: measure -> propose -> verify -> promote.

A pure, serving-agnostic orchestration algebra over capabilities that already exist in the library. It adds no new modeling capability: the proper scores and calibration diagnostics (mixle.inference.scoring / mixle.inference.calibration), the comparison tests (mixle.inference.model_comparison), the fitters (mixle.inference.estimation.optimize(), the streaming estimators, mixle.utils.automatic.get_estimator()), and the decision layer (mixle.inference.decision.bayes_action()) are all pre-existing. evolve wires them into a single anti-regression loop.

Phase 1 surface:

• measure – Objective + nll / log_score / crps / interval / calibration / decision_regret builders.

• propose – ImprovementOperator + Refit, OnlineUpdate, AutoSelect, Recalibrate, a scoped operator registry.

• verify – challenger_beats_champion() + Verdict (the anti-regression gate).

• drive – improve() + ImprovementResult, auto_select(), and the EvolutionLedger telemetry.

Phase 2-3 surface:

• search – search() over a typed Space (Real / Integer / Categorical), method='evolutionary' / 'bandit' / 'bo', returning a SearchResult.

• meta-search – Population + OperatorBandit (the policy that learns which operators help).

• structure search – Recompose / Mutate (genetic-programming structural moves over the model’s compositional tree) + structural_distance() (a tree-edit genotype distance) driving the population’s diversity; registered but off by default (structural + expensive).

improve(model, data, *, objective, operators=None, holdout=0.25, alpha=0.05, min_effect=0.0, budget=None, seed=0, ledger=None, parent_hash=None, require_calibration=True)

Propose challengers, gate them, and return the best verified improvement (or the champion).

Parameters:

• model (Any) – the champion (a fitted mixle distribution).

• data (Sequence[Any]) – the raw held-out dataset (split once into train/verify here).

• objective (Objective) – the Objective to improve on.

• operators (Sequence[ImprovementOperator] | None) – the proposal moves; defaults to the applicable subset of [Refit, OnlineUpdate, AutoSelect, Recalibrate].

• holdout (float | tuple) – train/verify split fraction, or an explicit (train, verify) tuple.

• alpha (float) – significance level for the verify gate.

• min_effect (float) – practical effect-size floor passed to the gate.

• budget (float | None) – optional cost ceiling – operators whose cost_hint exceeds the remaining budget are skipped (cheapest-first).

• seed (int) – RNG seed for the split and the sampled objectives.

• ledger (EvolutionLedger | None) – optional EvolutionLedger to record every attempt into.

• parent_hash (str | None) – optional lineage hash for the champion (carried onto candidates and ledger rows).

• require_calibration (bool) – forward to the gate’s calibration no-regression check.

Returns:

An ImprovementResult. result.verified is True guarantees result.model beat the champion significantly and non-regressively on the verify split.

Return type:

ImprovementResult

class ImprovementResult(model, verified, operator, delta, verdict, evidence=<factory>, parent_hash=None)

Bases: object

The outcome of an improve() run.

Parameters:

• model (Any)

• verified (bool)

• operator (str | None)

• delta (float)

• verdict (Verdict | None)

• evidence (dict)

• parent_hash (str | None)

model: Any

verified: bool

operator: str | None

delta: float

verdict: Verdict | None

evidence: dict

parent_hash: str | None = None

auto_select(data, *, space=None, criterion='bic', verify=True, holdout=0.25, seed=0, max_its=20)

Infer and fit a model from raw data, optionally gated by a held-out proper score.

Parameters:

• data (Sequence[Any]) – the raw dataset.

• space (Any | None) – reserved for the Phase-2 typed search space; must be None in Phase 1.

• criterion (str | Objective) – 'bic' (delegate to the automatic in-sample pick) or a proper-score Objective (add the held-out verify gate on top of BIC).

• verify (bool) – when criterion is an Objective, whether to run the held-out gate (the BIC pick fitted on the train split is the champion; the BIC pick refitted on all data is the challenger, promoted only if it wins out of sample).

• holdout (float) – held-out fraction for the proper-score gate.

• seed (int) – RNG seed for the split and sampled objectives.

• max_its (int) – EM iterations for the fits.

Returns:

An ImprovementResult. For criterion='bic' it carries the fitted automatic model with verified=False (no out-of-sample test was requested). For an Objective criterion with verify=True it carries the gate verdict and verified reflects whether the full-data model beats the train-only model out of sample.

Return type:

ImprovementResult

class Objective(*args, **kwargs)

Bases: Protocol

A lower-is-better-or-higher-with-a-flag scalar fitness with an optional paired vector.

name: str

lower_is_better: bool

pointwise(model, data)

The (n,) per-observation score, or None if the objective is scalar-only.

Parameters:

• model (Any)

• data (Any)

Return type:

ndarray | None

scalar(model, data)

The single comparable fitness (mean of pointwise by default).

Parameters:

• model (Any)

• data (Any)

Return type:

float

nll_objective()

Negative log-likelihood: per-obs -log p(y_i) (strictly proper, lower is better).

Return type:

Objective

log_score_objective()

Logarithmic score (log loss) of the predictive density at the realised outcomes.

Return type:

Objective

crps_objective(*, ensemble=256, seed=0)

Continuous Ranked Probability Score from a sampled predictive ensemble (lower is better).

Parameters:

• ensemble (int) – number of predictive draws per observation.

• seed (int) – RNG seed for the ensemble (reproducible).

Return type:

Objective

interval_objective(level=0.9, *, ensemble=256, seed=0)

Winkler interval score for the central level predictive interval (lower is better).

Parameters:

• level (float) – central coverage of the interval (e.g. 0.9 for a 90% interval).

• ensemble (int) – number of predictive draws used to read off the interval endpoints.

• seed (int) – RNG seed for the ensemble.

Return type:

Objective

calibration_objective(*, ensemble=256, seed=0, bins=10)

PIT calibration error of the predictive distribution (scalar-only, lower is better).

Uses the rank-based Probability Integral Transform of a sampled ensemble: under a calibrated continuous forecast the PIT values are Uniform(0, 1), and pit_calibration_error measures the histogram’s mean absolute deviation from uniform. There is no honest per-observation paired vector for a histogram statistic, so pointwise returns None and the verify gate scores this on the bootstrapped scalar.

(For classification models the natural calibration scalar is mixle.inference.calibration.expected_calibration_error(); this builder targets the continuous-predictive case, which is the common one for the streaming/auto-select loop.)

Parameters:

• ensemble (int)

• seed (int)

• bins (int)

Return type:

Objective

decision_regret_objective(loss, actions, *, n=2000, seed=0)

Realized decision regret under a fitted predictive posterior (scalar-only, lower is better).

For the chosen Bayes action a* = bayes_action(posterior(model), loss, actions) this reports the expected loss E_draw[ loss(a*, draw) ] – the realized cost of acting optimally under the model’s own belief. A better-calibrated model yields a lower expected loss for the same loss and action set, so it is a first-class promotion metric. Scalar-only (the regret is an expectation over posterior draws, not a per-observation quantity), so pointwise returns None.

Parameters:

• loss (Callable[[Any, Any], float]) – loss(action, draw) -> float (or a numpy-vectorized loss(action, draws) -> array).

• actions (Sequence[Any]) – the finite candidate-action set.

• n (int) – posterior draws for the Monte-Carlo expectation.

• seed (int) – RNG seed.

Return type:

Objective

challenger_beats_champion(champion, challenger, data, *, objective, alpha=0.05, min_effect=0.0, require_calibration=True, nonnested=False, multiplicity=None, calib_tol=1.0e-3, seed=0, elpd_pointwise=None)

Decide whether challenger significantly and non-regressively beats champion on data.

Parameters:

• champion (Any) – two fitted models scored on the same held-out data.

• challenger (Any) – two fitted models scored on the same held-out data.

• data (Any) – the held-out responses (one batch -> both models scored in the same order).

• objective (Objective) – the Objective to compare on.

• alpha (float) – significance level for the paired test (and the CI is at 1 - alpha).

• min_effect (float) – practical effect-size floor on |mean score difference|.

• require_calibration (bool) – if True, run the calibration no-regression check.

• nonnested (bool) – if True (a family swap), additionally require Vuong + Clarke to favor the challenger.

• multiplicity (str | None) – None or a mixle.inference.multiple_testing.adjust_pvalues() method ('bonferroni' / 'bh' / …) when this is one of many simultaneous challengers – adjusts the p-value before the gate.

• calib_tol (float) – tolerance on the calibration-error increase the challenger may carry.

• seed (int) – RNG seed for the (sampled) calibration scalars.

• elpd_pointwise (tuple[ndarray, ndarray] | None) – optional (pointwise_champion, pointwise_challenger) LOO/WAIC arrays; when given, the compare_elpd() 2-SE band is required to also favor the challenger.

Returns:

A Verdict. verdict.promote is the single promotion predicate.

Return type:

Verdict

class Verdict(favored, delta, p_value, ci, calibrated, evidence=<factory>)

Bases: object

The outcome of a single champion/challenger comparison.

Parameters:

• favored (str)

• delta (float)

• p_value (float)

• ci (tuple[float, float])

• calibrated (bool)

• evidence (dict)

favored: str

delta: float

p_value: float

ci: tuple[float, float]

calibrated: bool

evidence: dict

property promote: bool

True iff the challenger is favored and passed calibration – the promotion predicate.

as_dict()

Return type:

dict[str, Any]

class ImprovementOperator(*args, **kwargs)

Bases: Protocol

A uniform proposal move: a cheap applicability pre-flight plus a fitted-challenger propose.

name: str

cost_hint: float

applicable(model, data, *, ctx)

Cheap structural gate – can this operator even run on this model/data?

Parameters:

• model (Any)

• data (Any)

• ctx (dict)

Return type:

bool

propose(model, data, *, ctx)

Return a fitted challenger (or raise if the proposal cannot be built).

Parameters:

• model (Any)

• data (Any)

• ctx (dict)

Return type:

Candidate

class Candidate(model, operator, parent_hash=None, meta=<factory>)

Bases: object

A proposed, already-fitted challenger plus the provenance of how it was made.

Parameters:

• model (Any)

• operator (str)

• parent_hash (str | None)

• meta (dict)

model: Any

operator: str

parent_hash: str | None = None

meta: dict

class Refit(name='refit', cost_hint=1.0, max_its=20)

Bases: object

Re-fit the champion’s family on fresh data, warm-started from the champion’s parameters.

Parameters:

• name (str)

• cost_hint (float)

• max_its (int)

name: str = 'refit'

cost_hint: float = 1.0

max_its: int = 20

applicable(model, data, *, ctx)

Parameters:

• model (Any)

• data (Any)

• ctx (dict)

Return type:

bool

propose(model, data, *, ctx)

Parameters:

• model (Any)

• data (Any)

• ctx (dict)

Return type:

Candidate

class OnlineUpdate(mode='streaming', cost_hint=0.2)

Bases: object

Fold a fresh batch into the champion via a streaming estimator.

mode:

• 'streaming' – decay-mode StreamingEstimator (running-accumulator forgetting).

• 'incremental' – Neal-Hinton IncrementalEstimator (replace one chunk).

• 'posterior_carry' – exact recursive-Bayes BayesianStreamingEstimator (needs a

  conjugate family; applicable honestly checks ConjugateUpdatable).

• 'forgetting' – power-prior BayesianStreamingEstimator.

Parameters:

• mode (str)

• cost_hint (float)

mode: str = 'streaming'

cost_hint: float = 0.2

property name: str

applicable(model, data, *, ctx)

Parameters:

• model (Any)

• data (Any)

• ctx (dict)

Return type:

bool

propose(model, data, *, ctx)

Parameters:

• model (Any)

• data (Any)

• ctx (dict)

Return type:

Candidate

class AutoSelect(name='auto_select', cost_hint=3.0, max_its=20)

Bases: object

Infer an estimator from the raw data (get_estimator) and fit it – a possible family swap.

Parameters:

• name (str)

• cost_hint (float)

• max_its (int)

name: str = 'auto_select'

cost_hint: float = 3.0

max_its: int = 20

applicable(model, data, *, ctx)

Parameters:

• model (Any)

• data (Any)

• ctx (dict)

Return type:

bool

propose(model, data, *, ctx)

Parameters:

• model (Any)

• data (Any)

• ctx (dict)

Return type:

Candidate

class Recalibrate(name='recalibrate', cost_hint=0.5, ensemble=256, seed=0, grid=(0.6, 0.75, 0.9, 1.0, 1.1, 1.25, 1.5, 2.0))

Bases: object

Learn a predictive spread temperature T that flattens the PIT, no parameter refit.

applicable requires a sampler (used both to estimate the predictive center and to evaluate PIT calibration). The temperature is chosen on the train split by minimising the PIT calibration error over a small grid; T == 1 (the identity) is always in the grid, so the recalibrated model can never be worse-calibrated than the base on the fitting data.

Parameters:

• name (str)

• cost_hint (float)

• ensemble (int)

• seed (int)

• grid (tuple[float, ...])

name: str = 'recalibrate'

cost_hint: float = 0.5

ensemble: int = 256

seed: int = 0

grid: tuple[float, ...] = (0.6, 0.75, 0.9, 1.0, 1.1, 1.25, 1.5, 2.0)

applicable(model, data, *, ctx)

Parameters:

• model (Any)

• data (Any)

• ctx (dict)

Return type:

bool

propose(model, data, *, ctx)

Parameters:

• model (Any)

• data (Any)

• ctx (dict)

Return type:

Candidate

class Recompose(name='recompose', cost_hint=4.0, max_its=30)

Bases: object

Propose a richer STRUCTURE for the champion: a 2-component mixture of its family, each component warm-fit on a different half of the data so EM starts non-degenerate (avoiding the identical-component collapse). It wins the verify gate only when the data has structure a single component misses — anti-regression keeps it honest. Expensive, so it is registered but kept OUT of the default operator set.

Parameters:

• name (str)

• cost_hint (float)

• max_its (int)

name: str = 'recompose'

cost_hint: float = 4.0

max_its: int = 30

applicable(model, data, *, ctx)

Parameters:

• model (Any)

• data (Any)

• ctx (dict)

Return type:

bool

propose(model, data, *, ctx)

Parameters:

• model (Any)

• data (Any)

• ctx (dict)

Return type:

Candidate

class Mutate(name='mutate', cost_hint=4.0, max_its=30)

Bases: object

Genetic-programming structure search (Koza 1992): apply a random structural mutation to the champion and refit. Moves: grow (add a bootstrap-fit component), shrink (drop the lowest-weight component), and perturb (bootstrap re-fit). Repeated application inside a Population + the verify gate + a tree-edit genotype distance IS structure induction by selection (cf. Bayesian model merging, Stolcke & Omohundro 1994) — not open research. Registered but OUT of the default set (expensive).

Parameters:

• name (str)

• cost_hint (float)

• max_its (int)

name: str = 'mutate'

cost_hint: float = 4.0

max_its: int = 30

applicable(model, data, *, ctx)

Parameters:

• model (Any)

• data (Any)

• ctx (dict)

Return type:

bool

propose(model, data, *, ctx)

Parameters:

• model (Any)

• data (Any)

• ctx (dict)

Return type:

Candidate

structural_distance(a, b)

A [0, 1] genotype distance between two models’ structures (tree-edit distance, size-normalized).

Parameters:

• a (Any)

• b (Any)

Return type:

float

model_signature(model)

The compositional structure of model as a (type_label, [child signatures]) tree.

Parameters:

model (Any)

Return type:

tuple[str, list]

tree_edit_distance(a, b)

Unordered tree-edit distance: relabel cost (0/1) + greedy min-cost matching of children, with unmatched subtrees fully inserted/deleted. Exact for identical trees; a standard greedy approximation otherwise.

Parameters:

• a (tuple[str, list])

• b (tuple[str, list])

Return type:

int

register_operator(operator)

Register operator in the scoped evolve operator registry (returns it for decorator use).

Parameters:

operator (ImprovementOperator)

Return type:

ImprovementOperator

unregister_operator(name)

Remove a previously-registered operator by name (no-op if absent).

Parameters:

name (str)

Return type:

None

registered_operators()

A copy of the current scoped operator registry.

Return type:

dict[str, ImprovementOperator]

default_operators()

The Phase-1 default operator set: refit, online update, auto-select, recalibrate. (Recompose + Mutate are structural + expensive, so they are available via the registry but not enabled by default.)

Return type:

list[ImprovementOperator]

class EvolutionLedger(rows=<factory>)

Bases: object

An ordered, JSON-serializable log of improvement attempts.

Parameters:

rows (list[dict[str, Any]])

rows: list[dict[str, Any]]

record(*, operator, delta, verdict, cost, parent_hash, meta=None)

Append one attempt row and return it.

Parameters:

• operator (str)

• delta (float)

• verdict (dict | None)

• cost (float)

• parent_hash (str | None)

• meta (dict | None)

Return type:

dict[str, Any]

to_json(**dumps_kwargs)

Serialize the full ledger to a JSON string (rows are already plain dicts).

Parameters:

dumps_kwargs (Any)

Return type:

str

search(space, data, *, objective, build_fn, method='bo', n_iter=25, holdout=0.25, seed=0, **method_kwargs)

Search space for the config whose build_fn model scores best on a held-out split.

Parameters:

• space (Space) – the typed Space to search.

• data (Sequence[Any]) – the raw dataset (split once into train/val here for the inner objective).

• objective (Objective) – the held-out Objective (lower-is-better aware).

• build_fn (Callable[[dict[str, Any]], Any]) – caller-supplied config -> fitted model (the search is family-agnostic).

• method (str) – 'bo' (Bayesian optimization over the numeric box), 'evolutionary' (a (mu + lambda) loop over sample / neighbors), or 'bandit' (delegate the operator policy to an OperatorBandit).

• n_iter (int) – search budget (BO acquisition steps / evolutionary generations / bandit generations).

• holdout (float) – held-out fraction for the inner objective.

• seed (int) – RNG seed.

• method_kwargs (Any) – backend-specific knobs (e.g. mu / lam for the evolutionary loop, operators / size for the bandit population).

Returns:

A SearchResult with best_config / best_model / best_score (native orientation) / history.

Return type:

SearchResult

class SearchResult(best_config, best_model, best_score, history=<factory>)

Bases: object

The outcome of a search() (or Population.run()) run.

Parameters:

• best_config (dict[str, Any])

• best_model (Any)

• best_score (float)

• history (list[dict[str, Any]])

best_config: dict[str, Any]

best_model: Any

best_score: float

history: list[dict[str, Any]]

class Space(dims)

Bases: object

A typed, named search space over Real / Integer / Categorical dims.

Construct from a dict mapping each parameter name to its dimension:

space = Space({"mu": Real(-5, 5), "k": Integer(1, 4), "family": Categorical(["a", "b"])})

Dimension order is the insertion order of the dict; encode() / decode() and to_bounds() all use that same fixed order so the numeric vector and the box align.

Parameters:

dims (dict[str, Dimension])

property ndim: int

to_bounds()

The continuous (low, high) box for the BO backend (categoricals as integer indices).

Return type:

list[tuple[float, float]]

sample(rng)

Draw a random config dict, each dimension sampled natively.

Parameters:

rng (RandomState)

Return type:

dict[str, Any]

encode(config)

Encode a config dict into the numeric vector (in the fixed dimension order).

Parameters:

config (dict[str, Any])

Return type:

ndarray

decode(vector)

Decode a numeric vector (BO proposal) back into a config dict (rounding / clipping).

Parameters:

vector (Sequence[float])

Return type:

dict[str, Any]

neighbors(point)

All configs one local move away in exactly one dimension (the evolutionary mutation set).

Parameters:

point (dict[str, Any])

Return type:

list[dict[str, Any]]

class Real(lo, hi)

Bases: object

A continuous dimension over [lo, hi].

Parameters:

• lo (float)

• hi (float)

lo: float

hi: float

bounds()

Return type:

tuple[float, float]

sample(rng)

Parameters:

rng (RandomState)

Return type:

float

encode(value)

Parameters:

value (Any)

Return type:

float

decode(x)

Parameters:

x (float)

Return type:

float

neighbors(value)

A coarse local move: +/- 10% of the range, clipped to bounds.

Parameters:

value (Any)

Return type:

list[float]

class Integer(lo, hi)

Bases: object

An integer dimension over the inclusive range [lo, hi].

Parameters:

• lo (int)

• hi (int)

lo: int

hi: int

bounds()

Return type:

tuple[float, float]

sample(rng)

Parameters:

rng (RandomState)

Return type:

int

encode(value)

Parameters:

value (Any)

Return type:

float

decode(x)

Parameters:

x (float)

Return type:

int

neighbors(value)

Parameters:

value (Any)

Return type:

list[int]

class Categorical(choices)

Bases: object

An unordered categorical dimension over a finite list of choices.

Parameters:

choices (tuple[Any, ...])

choices: tuple[Any, ...]

bounds()

Return type:

tuple[float, float]

sample(rng)

Parameters:

rng (RandomState)

Return type:

Any

encode(value)

Parameters:

value (Any)

Return type:

float

decode(x)

Parameters:

x (float)

Return type:

Any

neighbors(value)

Parameters:

value (Any)

Return type:

list[Any]

class Population(seeds, *, objective, operators=None, bandit=None, size=12, diversity_quota=2, seed=0)

Bases: object

A diversity-preserving population of model structures, evolved by the OperatorBandit.

seeds are fitted models (the starting structures). Each step() selects operators via the bandit, applies them to parents chosen by fitness, gates the challengers with the Phase-1 champion/challenger rule (Benjamini-Hochberg multiplicity, since a generation produces many challengers at once), rewards the bandit with the verified deltas, and keeps the verified-best plus a coarse capability-diversity quota.

Parameters:

• seeds (Sequence[Any]) – the initial fitted models (at least one).

• objective (Objective) – the Objective to optimize (lower-is-better aware).

• operators (Sequence[ImprovementOperator] | None) – the proposal-move pool; defaults to the Phase-1 safe set.

• bandit (OperatorBandit | None) – an OperatorBandit over operators (built with the default policy if omitted).

• size (int) – the carrying capacity of the population.

• diversity_quota (int) – how many of size slots are reserved for capability-diverse members (the rest go to the fittest); the quota keeps the search from collapsing onto one structure too early.

• seed (int) – RNG seed for parent sampling and the bandit.

step(data)

Run one generation: select -> propose -> gate -> reward -> survivor selection.

Parameters:

data (Any)

Return type:

GenerationReport

run(data, generations=5)

Evolve for generations steps; return a SearchResult.

The returned best_model is the run incumbent, guaranteed no worse than the best seed on the objective (anti-regression). history is one row per generation (proposals / verified / score).

Parameters:

• data (Any)

• generations (int)

Return type:

Any

property champion: Any

The current run incumbent (the best model seen, anti-regression guaranteed).

class OperatorBandit(operators, *, policy='thompson', decay=0.97, prior_cost_aware=True, seed=0)

Bases: object

A non-stationary bandit over a fixed pool of ImprovementOperator.

select(k) returns the k highest-value operators under the chosen policy; reward folds a verified delta + cost back into the chosen arm; report is the “which operators help” artifact.

Policies:

• 'thompson' – Beta-Bernoulli Thompson sampling on the win indicator (reward > 0), scaled by the mean positive reward, so an operator that wins rarely but big and one that wins often but small are compared on expected verified delta.

• 'ucb' – UCB1 on the mean reward with a sqrt(2 ln N / n) exploration bonus.

Non-stationarity: each reward first multiplies every arm’s statistics by decay (a forgetting factor in (0, 1]), so old evidence fades and the policy can track a shifting best operator.

Parameters:

• operators (Sequence[ImprovementOperator])

• policy (str)

• decay (float)

• prior_cost_aware (bool)

• seed (int)

value(name)

The current policy value of operator name (a Thompson draw or the UCB index).

Parameters:

name (str)

Return type:

float

select(k=1)

Return the k operators with the highest policy value (a fresh Thompson draw each call).

Parameters:

k (int)

Return type:

list[ImprovementOperator]

reward(op_name, delta, cost)

Fold a verified delta (0 if the challenger was rejected) and cost into the arm.

Decays every arm first (non-stationarity), then updates the chosen arm. A delta is clipped at 0 below – a rejected challenger is a zero reward, never negative, matching the anti-regression guarantee (we never punish an operator for a rejected proposal beyond not rewarding it).

Parameters:

• op_name (str)

• delta (float)

• cost (float)

Return type:

None

report()

Per-operator win-rate, mean verified delta, mean cost, and (decayed) pull count.

Return type:

dict[str, Any]

Submodules

• mixle.evolve.improve module
• mixle.evolve.ledger module
• mixle.evolve.objective module
• mixle.evolve.operators module
• mixle.evolve.population module
• mixle.evolve.search module
• mixle.evolve.space module
• mixle.evolve.structure module
• mixle.evolve.verify module