mixle.task.active module

Active labeling – spend the teacher’s expensive labels only where they buy the most, not at random.

Labeling is the dominant cost of building a task model: every label is a frontier call or a human minute. Random labeling wastes most of them on examples the student already gets right. This is experimental design applied to the labeling decision (the discrete-pool analogue of mixle.doe active learning): label a small seed, fit a student, then repeatedly query the teacher only for the pool examples the student is most unsure about (and, optionally, most novel), refit, and continue until the budget runs out. The same student quality is reached for far fewer labels – direct money saved.

Acquisitions score the student’s own predictions (uncertainty as a ranking, which needs no calibrated probability) and can blend in the generative density (mixle.task.density.DensityGate) for diversity:

  • margin – smallest gap between the top two class scores (the classic, robust default);

  • entropy – highest predictive entropy;

  • least_confidence – lowest top-class score;

  • random – the baseline this is meant to beat.

active_distill returns the student plus a per-round log of labels-spent vs. agreement, so the labeling efficiency is auditable.

acquisition_scores(student, texts, method='margin')[source]

Informativeness of each unlabeled text under the student (higher = more worth labeling).

Parameters:
Return type:

ndarray

class ActiveResult(model, labels_used, history=<factory>, labeled_texts=<factory>, labeled_labels=<factory>)[source]

Bases: object

The actively-distilled student plus an audit trail of labels spent vs. quality reached each round.

Parameters:
model: TaskModel
labels_used: int
history: list[dict[str, Any]]
labeled_texts: list[str]
labeled_labels: list[Any]
active_distill(teacher, pool, *, budget, seed_size=20, rounds=5, acquisition='margin', labels=None, recipe=None, val_texts=None, seed=0)[source]

Distill from pool under a labeling budget, querying the teacher only for the most informative items.

Labels a seed_size random seed, then over rounds adds the top-scoring unlabeled examples (by acquisition) until budget labels are spent, refitting the student each round. If val_texts is given, the teacher labels it once and each round’s agreement on it is logged.

Parameters:
Return type:

ActiveResult