mixle.task.multilabel module

solve_multilabel – replace rigid code that returns a SET of labels, with per-label honesty.

The multi-label shape of the solve loop: teacher(x) -> list[str] (tags, flags, categories – any subset of a label universe). The student is one shared-feature net with a sigmoid head per label; the calibration is per-label conformal thresholds from the held-out slice:

  • A_l – the 1 - alpha quantile of label l’s scores among calibration inputs where l is ABSENT: a score above it is confidently-present (at most ~alpha of absents score that high);

  • P_l – the alpha quantile among inputs where l is PRESENT: a score below it is confidently-absent.

A label is decided when its score clears one of those bars; anything in between is ambiguous. The input is answered locally ONLY when every label is decided – one ambiguous tag escalates the whole request to the teacher (whose answer is harvested), so a locally-returned set never contains a guess. Labels with too few calibration examples on either side are never decided locally (their bars are inf/-inf): under-calibrated is treated as ambiguous, not as confident.

class MultiLabelSolution(net, featurizer, labels, teacher, upper_absent, lower_present, alpha, holdout_set_agreement, train_inputs=<factory>, train_sets=<factory>, cal_inputs=<factory>, cal_sets=<factory>, hidden=(64, ), epochs=300, lr=0.01, seed=0, n_requests=0, n_escalated=0, harvested_inputs=<factory>, harvested_sets=<factory>)[source]

Bases: object

A per-label-calibrated tagger in front of the routine it replaces.

Parameters:
net: Any
featurizer: Any
labels: list[str]
teacher: Callable[[...], Any]
upper_absent: ndarray
lower_present: ndarray
alpha: float
holdout_set_agreement: float
train_inputs: list
train_sets: list
cal_inputs: list
cal_sets: list
hidden: tuple = (64,)
epochs: int = 300
lr: float = 0.01
seed: int = 0
n_requests: int = 0
n_escalated: int = 0
harvested_inputs: list
harvested_sets: list
try_local(x)[source]

The decided label set, or None when any label is ambiguous (= must escalate).

Parameters:

x (Any)

Return type:

list[str] | None

report()[source]
Return type:

dict[str, Any]

save(path)[source]

Persist net + featurizer + per-label bars; load() restores a serving tagger.

Parameters:

path (str)

Return type:

str

classmethod load(path, teacher, *, device='cpu')[source]

Reconstitute a serving MultiLabelSolution (no training/calibration data; improve() raises).

Parameters:
Return type:

MultiLabelSolution

improve()[source]

Re-fit with harvested sets; promote only if held-out set agreement does not regress.

Return type:

bool

solve_multilabel(teacher, inputs, *, alpha=0.1, holdout=0.25, kind=None, hidden=(64,), epochs=300, lr=1e-2, dim=256, prelabeled=None, seed=0)[source]

Replace a set-of-labels routine with a per-label-calibrated student (see module docstring).

prelabeled — already-teacher-labeled (inputs, label_sets), typically harvested escalations from a serving deployment — folds into the TRAINING split only, never calibration (which stays a fresh split of inputs, so the per-label bars keep their finite-sample rank guarantee). Labels seen only in prelabeled still enter the label space.

Parameters:
Return type:

MultiLabelSolution