mixle.task.multilabel module

solve_multilabel – replace rigid code that returns a set of labels, with joint-set honesty.

The multi-label shape of the solve loop: teacher(x) -> list[str] (tags, flags, categories – any subset of a label universe). The student is one shared-feature net with a sigmoid head per label; the calibration uses one joint nonconformity score: the largest binary-label error across the whole label vector. Its split-conformal quantile covers the complete teacher set with probability at least 1 - alpha under exchangeability. A request is answered locally only when that joint prediction set contains exactly one label vector; multiple or zero admissible vectors escalate to the teacher.

Scope of that statement. The 1 - alpha joint-set coverage is finite-sample and MARGINAL over the calibration draw and the query jointly, under exchangeability of the calibration rows and incoming traffic. It is NOT an accuracy guarantee conditional on answering locally: serving conditions on the joint set being a singleton, and coverage conditional on that event is not controlled (the classification side measured 9% marginal versus 47% answered-slice error from the same selection effect). Answered-slice quality is a MEASUREMENT, never a guarantee, and report() carries two distinct numbers that must not be conflated: holdout_set_agreement is the RAW route-independent 0.5-threshold exact-set agreement on the evaluation rows (the serving gate never runs in it), while answered_slice applies the real joint-qhat singleton rule to the same disjoint evaluation rows and reports exact-set agreement among the rows the gate actually answered, with the answered denominator and an exact 95% Clopper-Pearson interval (None when it answered none). After an improve() promotion those numbers describe the winner of a 2-way comparison made on the same rows, so they are post-selection measurements (the receipt records this). Distribution shift or any other break of exchangeability voids the coverage statement silently; re-measure on drifted traffic. report() carries the scope machine-readably in coverage_contract_scope.

class MultiLabelSolution(net, featurizer, labels, teacher, upper_absent, lower_present, joint_qhat, alpha, holdout_set_agreement, eval_rows=0, answered_eval_n=0, answered_eval_correct=0, train_inputs=<factory>, train_sets=<factory>, cal_inputs=<factory>, cal_sets=<factory>, eval_inputs=<factory>, eval_sets=<factory>, hidden=(64, ), epochs=300, lr=0.01, seed=0, n_requests=0, n_escalated=0, harvested_inputs=<factory>, harvested_sets=<factory>, calibration_receipt=<factory>)[source]

Bases: AnsweredSliceGuard

A per-label-calibrated tagger in front of the routine it replaces.

Parameters:
try_local(x)[source]

The decided label set, or None when any label is ambiguous (= must escalate).

Parameters:

x (Any)

Return type:

list[str] | None

decide(x)[source]

Return the local multilabel decision, or None when the example should escalate.

Parameters:

x (Any)

Return type:

list[str] | None

report()[source]

Return multi-label agreement, escalation, and harvest metrics.

Return type:

dict[str, Any]

save(path)[source]

Persist net + featurizer + per-label bars; load() restores a serving tagger.

Parameters:

path (str)

Return type:

str

classmethod load(path, teacher, *, device='cpu')[source]

Reconstitute a serving MultiLabelSolution (no training/calibration data; improve() raises).

Parameters:
Return type:

MultiLabelSolution

improve(evidence_inputs=None)[source]

Re-fit harvested sets and use a fresh, single-use evidence batch for promotion and calibration.

Parameters:

evidence_inputs (Sequence[Any] | None)

Return type:

bool

solve_multilabel(teacher, inputs, *, alpha=0.1, holdout=0.25, kind=None, hidden=(64,), epochs=300, lr=1e-2, dim=256, prelabeled=None, seed=0)[source]

Replace a set-of-labels routine with a per-label-calibrated student (see module docstring).

prelabeled — already-teacher-labeled (inputs, label_sets), typically harvested escalations from a serving deployment — folds into the TRAINING split only, never calibration (which stays a fresh split of inputs, so the per-label bars keep their finite-sample rank guarantee). Labels seen only in prelabeled still enter the label space.

Parameters:
Return type:

MultiLabelSolution