mixle.evolve.closed_loop module

L1: closed-loop self-evolution with operator credit – the loop at five altitudes, wired end to end.

Router-harvested failures -> acquisition (A5’s mixle.task.acquire.acquire()) -> challenger production (a distill/refine/evolve operator, all reused ImprovementOperator instances, not reimplemented) -> the held-out challenger_beats_champion() gate -> deploy, as ONE budgeted background loop (ClosedLoopSelfEvolution). A per-context meta-bandit (OperatorCreditBandit, wrapping mixle.task.bandit.UCB1 – the same UCB1 machinery reused across this codebase for meta-bandits) learns which challenger-production operator wins for which kind of failure, and every ADOPTED champion gets a genealogy receipt (GenealogyLedger, built directly on EvolutionLedger) recording its parent, the operator that produced it, and the measured gap – a real, walk-backable lineage.

Data roles (MXR-080-0042). ClosedLoopSelfEvolution.step() splits each arriving batch into a candidate pool and a held-out verify_data split BEFORE harvest/acquire touch anything, fits a challenger only on an acquired subset of the candidate pool, and gates it against verify_data by default (see step()’s “Note on verify”) – never on data the challenger was just fit on, and never on the biased-by-construction complement of acquire’s own hardest-example selection. See mixle.evolve.population’s module docstring for the fuller three-role framing (fit / reusable validation / one-shot holdout) this loop shares: unlike Population, each step here ordinarily gates against a FRESH batch from the caller’s stream, so a single step’s promotion decision is itself evidence-bearing; only the per-context OperatorCreditBandit’s reward, accumulated across many steps, is a heuristic search signal in the same sense.

Which subsystems this wires, not rebuilds:

  • mixle.task.router – harvested_from_router() reads a real Router’s .harvested() (the frontier-answered cases the cheap tiers could not handle) as the harvested-failure source when a caller already runs one. For domains without a live Router (e.g. this module’s own tests, which evolve a marginal label model rather than a routed per-input classifier), harvest_failures() is the same idea generalized to any Objective: the observations where the current champion did NOT score at its own best-attainable pointwise value – literally “the cases it got wrong.”

  • mixle.task.acquire.acquire() (A5) ranks the harvested pool before it is spent on challenger production. acquire needs a “scoreable” model (predict_proba-shaped); a champion that is a bare marginal distribution (does not condition on the item) is wrapped by _ConstantProbaAdapter so the real acquire() code path runs (entropy strategy) instead of being skipped – honestly, this degenerates to “prioritize while the champion overall is unconfident” since the per-item probability is constant, but it is the real primitive, not a mock.

  • mixle.evolve.operators – three EXISTING operators stand in for L1’s “distill / refine / evolve ops” triad (see default_challenger_operators()): AutoSelect (cold-start refit – distillation’s “train a fresh model from the teacher-labeled pool” shape) as "distill", Refit (warm-started refit of the champion’s own parameters) as "refine", and Mutate (the genetic-programming structure-edit operator) as "evolve".

  • mixle.evolve.verify.challenger_beats_champion() is the held-out gate, used exactly as-is.

  • mixle.evolve.ledger.EvolutionLedger is the genealogy substrate, wrapped (not replaced) by GenealogyLedger.

  • mixle.task.bandit.UCB1 is the meta-bandit’s arm-selection machinery, one instance per context.

Principled crossover. If a challenger-production operator ever combines two existing champions (a “crossover” in evolutionary-algorithm terms), it MUST do so via mixture composition (mixle.ops.mixture(), exactly what Mutate’s grow move and Recompose already do) – never by cutting and pasting incompatible weight sub-blocks from two unrelated models. principled_crossover() is the explicit primitive for that case.

accuracy_objective()[source]

Higher-is-better accuracy for a model exposing a single “most likely label” prediction (a fitted CategoricalDistribution’s argmax(pmap)): per-observation 1{y == predicted_label}.

Return type:

Objective

harvest_failures(champion, batch, objective)[source]

The objective-generic analog of mixle.task.router.Router.harvested(): the observations in batch where champion did NOT score at its own best-attainable pointwise value under objective – for accuracy_objective() this is exactly “the cases it got wrong.” A scalar-only objective (no honest per-observation vector) harvests the whole batch – there is no finer signal to rank on.

Parameters:
Return type:

list[Any]

harvested_from_router(router)[source]

Pull the REAL harvested failure pool from a live mixle.task.router.Router: every input that escalated all the way to the frontier, paired with its frontier label – exactly router.harvested(), just zipped into one pool for mixle.task.acquire.acquire().

Parameters:

router (Any)

Return type:

list[tuple[Any, Any]]

default_challenger_operators()[source]

The three L1 challenger-production operators, each a reused ImprovementOperator (no bespoke fitting logic):

  • "distill" -> AutoSelect – cold-start: fit a fresh model from the harvested pool, the “train a new model from teacher-labeled data” shape of distillation.

  • "refine" -> Refit – warm-started refit of the champion’s own parameters on the harvested pool.

  • "evolve" -> Mutate – genetic-programming structure edit (grow/shrink/perturb), mixle.evolve’s own structure-search operator.

Return type:

dict[str, ImprovementOperator]

principled_crossover(model_a, model_b, *, weight_a=0.5)[source]

Combine two champions the ONLY principled way this framework allows: mixture composition (mixle.ops.mixture()), never gene-splicing (cutting/pasting incompatible weight sub-blocks). Returns an unfitted mixture prototype; refit it against data (e.g. via its own .estimator()) before treating it as a challenger.

Parameters:
Return type:

Any

class OperatorCreditBandit(operator_names, *, c=1.0, seed=0)[source]

Bases: object

A per-context meta-bandit over challenger-production operators, wrapping one mixle.task.bandit.UCB1 per context (e.g. a failure type/domain) – so the loop learns, independently for each context, which operator actually produces winning challengers there.

Parameters:
select(context)[source]

The bandit-chosen operator name for context.

Parameters:

context (str)

Return type:

str

reward(context, operator, reward)[source]

Fold an observed (non-negative, anti-regression) reward back into operator’s arm.

reward must be finite: max(nan, 0.0) is nan, which would poison the arm’s running mean and every selection that reads it, permanently.

Parameters:
Return type:

None

report()[source]

Per-context, per-operator mean reward and pull count.

Return type:

dict[str, dict[str, float]]

class GenealogyLedger(ledger=<factory>)[source]

Bases: object

Genealogy receipts for adopted (gate-passing) champions, built directly on EvolutionLedger (never storing model objects in the ledger rows themselves – only their operator, measured gap, and a stable id – exactly the ledger’s own JSON-serializability discipline).

A model’s id is DURABLE and content-addressed (mixle.data.hashing.model_hash(), a hex SHA-256 over its serialized parameters) – never the model object’s id() (a CPython memory address). That makes ids stable across processes (a model reloaded from disk hashes to the exact same id it had when first recorded, so lineage() finds it) and immune to id()-reuse aliasing (once a model is garbage-collected, an unrelated new object reallocated at the same address can never be mistaken for it, because the id no longer depends on the address at all). Combined with the ledger rows’ existing JSON-serializability, the lineage this class records is genuinely replayable: every id in a persisted ledger can be recomputed independently from any model, in any later process.

Parameters:

ledger (EvolutionLedger)

record_adoption(*, parent, child, operator, gap, context, meta=None)[source]

Record ONE adoption: child replaced parent via operator, measured gap (the verified challenger-beats-champion delta). parent=None marks the root of a lineage.

Parameters:
Return type:

dict[str, Any]

lineage(model)[source]

Reconstruct model’s full lineage: the ordered (root-first) chain of adoption receipts {operator, delta (the measured gap), parent_hash, meta: {child_hash, context, ...}} back to the first recorded ancestor. Returns [] if model was never recorded as an adoption – including if model can’t be hashed at all (e.g. it isn’t a fitted mixle model), which is just another way of never having been recorded. Because the id is content-addressed (see class docstring), this works for ANY object whose serialized content matches a previously-recorded one – not only the exact live Python instance record_adoption() was originally called with – e.g. the same model reloaded from disk in a fresh process.

Parameters:

model (Any)

Return type:

list[dict[str, Any]]

class LoopStepResult(context, operator, promoted, delta, champion_score, champion, cost=0.0)[source]

Bases: object

One ClosedLoopSelfEvolution.step() outcome.

cost is what this step actually charges against a ClosedLoopSelfEvolution.run() budget: the selected operator’s cost_hint iff the attempt was genuinely applicable and completed without raising, 0.0 otherwise – skipped-by-budget, inapplicable, and failed attempts are all free.

Parameters:
class ClosedLoopSelfEvolution(champion, *, objective, operators=None, context_fn=None, acquire_k=32, acquire_strategy='entropy', bandit_c=1.0, holdout=0.25, seed=0)[source]

Bases: object

The whole L1 loop, one budgeted step at a time: harvest -> acquire -> propose -> gate -> deploy, with per-context operator credit and genealogy receipts on every adoption.

Parameters:

  • champion (Any) – the initial fitted model (e.g. a CategoricalDistribution).

  • objective (Objective) – the Objective the loop optimizes (e.g. accuracy_objective()).

  • operators (dict[str, ImprovementOperator] | None) – challenger-production operators by name; defaults to default_challenger_operators().

  • context_fn (Callable[[Sequence[Any]], str] | None) – batch -> context key for the per-context bandit; defaults to a single global context ("default") when the caller has no domain/failure-type signal to condition on.

  • acquire_k (int) – how many harvested failures A5’s acquire prioritizes per step.

  • acquire_strategy (str) – the acquire() ranking strategy (default "entropy", the one that works for a marginal/non-conditional champion via _ConstantProbaAdapter).

  • holdout (float) – fraction of each arriving batch set aside, BEFORE harvest/acquire ever see it, as the default verify_data when a caller omits verify in step() – see step()’s “Note on verify” (MXR-080-0042). Irrelevant if the caller always passes an explicit verify.

  • seed (int) – RNG seed threaded through the bandit, the split, and the gate.

  • bandit_c (float)

step(batch, *, verify=None, context=None, budget_remaining=None)[source]

One cycle of the loop on one arriving batch of held-out-shaped observations.

  1. harvest failures under the CURRENT champion,

  2. rank them with A5’s acquire,

  3. pick a challenger-production operator via the per-context bandit,

  4. propose + gate the challenger (skipped if budget_remaining can’t afford it),

  5. score the model the step would deploy, while it is still only staged,

  6. commit as one transaction iff the gate promotes: genealogy receipt, deployment, bandit reward, and history entry, in that order.

Steps 5 and 6 are deliberately ordered that way. A step is all-or-nothing: either it returns a LoopStepResult that is also in history and whose champion/receipt/bandit state all agree, or it raises having changed nothing at all and can simply be retried. Scoring used to happen after the champion had already been swapped and its receipt written, so a scoring failure stranded the loop with a deployed challenger, an adoption row, and a bandit reward for a step that – as far as its caller and its own history were concerned – never happened.

Note on verify (MXR-080-0042 – data roles, see mixle.evolve.population’s module docstring for the fuller three-role framing this loop shares): train_data (what the challenger is fit on) is acquire’s top-priority SUBSET of batch. When verify is omitted, verify_data no longer defaults to the whole batch (which CONTAINS train_data, so the gate used to score a challenger on rows it had just been fit on – weaker evidence than a disjoint split). Instead, batch is split (_split(), fraction self.holdout) into a candidate pool and verify_data FIRST, before harvest/acquire ever see it, and ONLY the candidate pool is harvested + ranked for train_data. This is deliberately a RANDOM split up front, not batch minus whatever acquire happened to pick: acquire selects the pool’s most informative/hardest-to-classify items on purpose (that is the whole point of active acquisition), so its complement is systematically skewed toward whatever the CURRENT champion already gets right – an unrepresentative, biased verify set that would under-detect exactly the improvement the loop exists to find, which is arguably worse than the overlap it would replace. Splitting first keeps verify_data a fair, representative sample of batch’s actual distribution, genuinely disjoint from anything harvest/acquire/the challenger’s fit ever touches. Callers who want to control the split themselves (or whose batch is already disjoint from wherever train_data will come from) should pass an explicit verify batch instead. Note too that ONE step’s gate decision (verdict.promote above) is a fresh, one-shot look at that step’s own (now-disjoint, representative) data – the per-step promotion decision itself is evidence-bearing. What is only a heuristic is the per-CONTEXT OperatorCreditBandit’s ACCUMULATED reward across many steps: exactly like Population’s bandit, it is fine when each step draws fresh data (the normal run()-over-a-stream usage), but a caller who calls step() repeatedly against the SAME (or overlapping) verify/batch data should treat the bandit’s learned preferences as search guidance, not as a promotion-grade statistical claim about any one operator.

Note on budget_remaining: when given, the bandit-selected operator’s own cost_hint is checked against it BEFORE the operator runs – an operator whose cost exceeds what remains is skipped exactly like an inapplicable one (no propose/gate attempted, nothing charged). result.cost reports what this step actually charges: the operator’s cost_hint iff the attempt was genuinely applicable and completed without raising, 0.0 otherwise – see run(), which sums result.cost to track total spend. It must be a finite, nonnegative amount: every budget comparison here is an ordered one, and every ordered comparison against NaN is false, so a NaN ceiling is not a ceiling (_budget_amount()).

Parameters:
Return type:

LoopStepResult

run(stream, *, verify_batches=None, contexts=None, budget=None)[source]

Run the loop over a SEQUENCE of batches (a stream) – the budgeted background loop’s outer iteration. budget (if given) is a ceiling on total spend: each step’s operator cost is checked against the REMAINING budget and must fit BEFORE that operator runs (see step()’s budget_remaining), and only result.cost – what the step actually charged – accumulates spent, so a skipped-by-budget, inapplicable, or failed attempt is genuinely free, exactly as documented; only a genuinely-applicable, genuinely-completed attempt is charged its cost_hint. The loop stops early once the budget is fully spent, leaving the champion at whatever it last became.

verify_batches and contexts, when given, are per-step and must line up ONE-TO-ONE with stream. Both are validated here, before the first step runs, rather than being indexed as the loop goes (MXR-080-1902): a short contexts used to raise IndexError part-way through, after the earlier steps had already deployed champions, written adoption receipts, rewarded per-context bandit arms and appended history – an argument error discovered only once it had produced durable, half-finished state that no return value described. A long one was worse, because it raised nothing at all: the surplus entries were silently dropped, so a caller who had mis-zipped their stream against their contexts got a completed-looking run in which every step had been attributed to the wrong context. budget is validated the same way and for the same reason (_budget_amount()).

Parameters:
Return type:

list[LoopStepResult]