mixle.task.acquire module

Generic active-acquisition glue: rank an unlabeled pool for any scoreable model.

mixle.task.active.active_distill already runs the acquire-label-refit loop end to end, but its ranking step (acquisition_scores()) is hardwired to a single concrete shape: a TaskModel whose adapter exposes proba_batch over batches of text. That is the demo, not the library primitive – other candidate pools (records, images, raw feature vectors, …) and other scoreable models (a fitted mixle.stats distribution, an ensemble of them, a plain predict_proba classifier) need the same ranking logic without cloning active_distill’s internals. acquire() is that primitive: acquire(pool, model, k, strategy) scores every pool item under strategy and returns the top k.

Dispatch, not a hardcoded type. A model is “scoreable” if _proba_batch() can get a row-stochastic (n, k) prediction matrix out of it, tried in order: a bare predict_proba method (the generic/sklearn-shaped case); the TaskModel adapter shape (model.adapter.proba_batch(model.model, items) – the exact call active_distill already makes, so every existing distilled student keeps working); or, recursively, a weighted ensemble of scoreable sub-models (see below), whose mixture prediction is the weight-averaged member prediction. No strategy here ever branches on isinstance(model, TaskModel) – it is just one more shape that happens to satisfy the same duck-typed contract.

Which EIG machinery. The acceptance criteria names two candidate sources: mixle.epistemic’s nested-MC portfolio estimator (mixle.epistemic.loop._portfolio_eig_nmc()) and mixle.doe.active.expected_information_gain_nmc. Neither is called directly here, and the choice between them is really a choice about which one’s math, not its exact function, fits a discrete already-materialized pool of candidates with a categorical outcome:

  • mixle.doe.active.expected_information_gain_nmc() is written against a continuous numpy parameter space (prior_sampler(rng, n) -> (n, k) array plus a simulate callable) – exactly the mismatch mixle.epistemic.loop’s own docstring calls out for its portfolio use case. Forcing a pool of discrete “which hypothesis/model in my ensemble is right” questions through that interface would mean flattening every ensemble member into a numeric vector, which defeats the point of an arbitrary scoreable-model ensemble.

  • mixle.epistemic.portfolio.HypothesisPortfolio is exactly the right shape instead: a weighted, typed set of hypotheses – which is exactly what an ensemble of scoreable models is. The eig strategy below (_eig_strategy()) is the discrete-pool, categorical-outcome specialization of the same nested-MC EIG identity EIG = E_{h,y}[log p(y|h) - log E_{h'}[p(y|h')]] that mixle.epistemic.loop._portfolio_eig_nmc() estimates by simulation: for a categorical y with a known per-hypothesis predictive distribution (no simulation needed, the sum over the finite outcome space is exact) that identity reduces in closed form to the mutual-information / BALD decomposition EIG(x) = H[E_h[p(y|x,h)]] - E_h[H[p(y|x,h)]] (Houlsby et al. 2011) – entropy of the mixture prediction minus the expected entropy of each member’s own prediction. That closed form is what _eig_strategy() computes: no Monte Carlo, no simulate callable, and it accepts either a real HypothesisPortfolio or the lighter duck-typed members/weights ensemble shape below, so a caller who doesn’t want the portfolio’s reweighting/pruning machinery isn’t forced to build one just to rank a pool.

Ensemble shape. model participates in the eig/disagreement strategies if it is a HypothesisPortfolio (its active hypotheses’ payload``s are the scoreable members, its weights the ensemble weights) or exposes ``model.members (a sequence of scoreable sub-models) with an optional model.weights (defaults to uniform). A single non-ensemble scoreable model has no disagreement/EIG to compute (there is only one opinion) and raises CapabilityError; it works fine with "entropy", which needs only one predictive distribution per pool item.

Strategies are a registry, not a branch. Mirroring mixle.doe.bayesopt.register_acquisition()’s “register, don’t branch” pattern: built-ins ("eig", "disagreement", "entropy") are registered by name below via register_strategy(), and a caller registers a custom one the same way – acquire never special-cases a strategy name.

acquire(pool, model, k, strategy='eig', **strategy_kwargs)[source]

Rank pool by strategy under model and return the top k items to label next.

The model-agnostic generalization of active_distill’s hardwired text-classifier ranking step (mixle.task.active.acquisition_scores()): any scoreable model (see the module docstring’s dispatch rules) and any pool of candidates work, not just a TaskModel over text. strategy is either a registered name ("eig", "disagreement", "entropy", or a custom one registered via register_strategy()) or a bare callable with the same fn(pool, model, **kwargs) -> scores contract. Returns the highest-scoring min(k, len(pool)) pool items, most worth-labeling first; an empty pool or non-positive k returns [].

Parameters:
Return type:

list[Any]

register_strategy(name, fn)[source]

Register an acquisition strategy under name.

fn is called as fn(pool, model, **strategy_kwargs) and must return an array of scores, one per pool item, where higher means more worth labeling. This is the extension point for new strategies – registering is all acquire() needs, no edits to acquire itself.

Parameters:
Return type:

None

available_strategies()[source]

Return the sorted names of every registered strategy.

Return type:

list[str]