mixle.evolve.search module

auto_select and search: model selection and config search from data.

  • auto_select() elevates the existing automatic engine (mixle.utils.automatic.get_estimator()) into the evolve contract and, when the criterion is a proper-score Objective, adds the held-out champion/challenger gate on top of the in-sample BIC pick so the returned model wins out of sample, not merely on BIC.

  • search() searches a typed Space for the config that builds the best model under a held-out Objective, with three interchangeable backends:

    • method='bo' – encode the space as a numeric box and drive mixle.doe.minimize().

    • method='evolutionary'– a (mu + lambda) loop over Space.sample() / Space.neighbors().

    • method='bandit' – delegate the which-operator decision to the OperatorBandit via a Population.

    build_fn(config, train_data) -> fitted model is caller-supplied and given ONLY the training split (never val), so search is family-agnostic.

auto_select(data, *, space=None, criterion='bic', verify=True, holdout=0.25, seed=0, max_its=20)[source]

Infer and fit a model from raw data, optionally gated by a held-out proper score.

Parameters:

  • data (Sequence[Any]) – the raw dataset.

  • space (Any | None) – reserved for typed search-space selection; must currently be None for auto_select.

  • criterion (str | Objective) – 'bic' (delegate to the automatic in-sample pick) or a proper-score Objective (add the held-out verify gate on top of BIC).

  • verify (bool) – when criterion is an Objective, whether to run the held-out gate (the BIC pick fitted on the train split is the champion; the challenger is the same automatic family refit on that SAME train split, so the comparison on the held-out data is genuinely held out. Promotion deploys a fresh refit of the verified winner on all the data – never the object that was itself scored – so a promoted model still benefits from the held-out split’s extra evidence).

  • holdout (float) – held-out fraction for the proper-score gate.

  • seed (int) – RNG seed for the split and sampled objectives.

  • max_its (int) – EM iterations for the fits.

Returns:

An ImprovementResult. For criterion='bic' it carries the fitted automatic model with verified=False (no out-of-sample test was requested). For an Objective criterion with verify=True it carries the gate verdict and verified reflects whether the train-only challenger beat the train-only champion out of sample.

Return type:

ImprovementResult

search(space, data, *, objective, build_fn, method='bo', n_iter=25, holdout=0.25, seed=0, **method_kwargs)[source]

Search space for the config whose build_fn model scores best on a held-out split.

Parameters:

  • space (Space) – the typed Space to search.

  • data (Sequence[Any]) – the raw dataset (split once into train/val here for the inner objective).

  • objective (Objective) – the held-out Objective (lower-is-better aware).

  • build_fn (Callable[[dict[str, Any], list[Any]], Any]) – caller-supplied (config, train_data) -> fitted model (the search is family-agnostic). train_data is ALWAYS the train split search computed – never val – so a config can actually be fit to real data without risking a held-out leak.

  • method (str) – 'bo' (Bayesian optimization over the numeric box), 'evolutionary' (a (mu + lambda) loop over sample / neighbors), or 'bandit' (delegate the operator policy to an OperatorBandit).

  • n_iter (int) – the TOTAL evaluation budget (build/score attempts), consistently for bo and evolutionary – each reserves every evaluation against it before spending, so neither can exceed it (the evolutionary backend used to treat it as a generation count and spend mu + lam * n_iter). Both need at least one evaluation to produce any result, so n_iter<=0 is rejected with a clear error instead of silently spending an evaluation anyway. (bandit interprets n_iter as a generation count, unchanged – it delegates to Population, whose per-generation work depends on the operator policy rather than on a fixed offspring count.)

  • holdout (float) – held-out fraction for the inner objective.

  • seed (int) – RNG seed.

  • method_kwargs (Any) – backend-specific knobs (e.g. mu / lam for the evolutionary loop, operators / size for the bandit population).

Returns:

A SearchResult with best_config / best_model / best_score (native orientation) / history, plus search_failed / n_evaluations / n_successes for bo / evolutionary (see SearchResult).

Return type:

SearchResult

class SearchResult(best_config, best_model, best_score, history=<factory>, search_failed=False, n_evaluations=0, n_successes=0)[source]

Bases: object

The outcome of a search() (or Population.run()) run.

search_failed / n_evaluations / n_successes distinguish “we searched and found a real (if mediocre) model” from “every configuration failed and best_model is a hollow placeholder” – without them, a totally failed search (best_model=None, best_score at the internal penalty sentinel) is shaped identically to a real result. Every backend populates them, including the bandit one via mixle.evolve.population.Population.run() – it used to leave them at their defaults, so a run that evaluated seeds and generations still reported n_evaluations=0 and search_failed=False, which reads as a successful search that did no work.

Parameters: