mixle.task.disagreement module¶

Disagreement gate: escalate where the student has historically diverged from the teacher.

This is distinct from escalating only where inputs look statistically atypical (DensityGate) or where the conformal set itself is ambiguous (CalibratedTaskModel).

fit_disagreement_gate() turns a set of (text, student_label, teacher_label) triples into a compact binary agree/disagree classifier over the student’s own feature space (reusing distill_from_labels() – the disagreement gate is itself a distilled student, just of a different target). The resulting DisagreementGate exposes ood_mask with the exact same duck-typed shape as DensityGate, so it plugs into CalibratedTaskModel(..., density_gate=...) directly – or unions with a real density gate via union_gate() – with no changes needed to mixle.task.calibrate’s extension point.

measure_disagreement_mass() is the plain fraction-of-examples-where-student-differs-from-teacher metric the active-labeling loop (active_distill()) is measured against: label the gate-flagged region with the teacher, re-distill including those labels, and confirm the region’s mass shrinks.

measure_disagreement_mass(student, texts, teacher_labels)[source]

Fraction of texts where the student’s label differs from the teacher’s.

Parameters:

student (TaskModel)
texts (Sequence[str])
teacher_labels (Sequence[Any])

Return type:

float

class DisagreementGate(classifier, threshold=0.5, calibration_receipt=<factory>)[source]

Bases: object

A fitted agree/disagree classifier over the student’s feature space, plus an escalation threshold.

Parameters:

classifier (TaskModel)
threshold (float)
calibration_receipt (dict[str, Any])

disagreement_proba(texts)[source]

P(disagree | x) under the fitted classifier.

Parameters:: texts (Sequence[str])
Return type:: ndarray

is_ood(text)[source]

Return whether one input is predicted to disagree with the teacher.

Parameters:: text (str)
Return type:: bool

ood_mask(texts)[source]

Same duck-typed shape as mixle.task.density.DensityGate.ood_mask() – drops straight into CalibratedTaskModel(..., density_gate=this).

Parameters:: texts (Sequence[str])
Return type:: ndarray

fit_disagreement_gate(student, texts, teacher_labels, *, dim=256, hidden=(32,), epochs=150, lr=1e-2, seed=0, alpha=0.05, calibration_frac=0.2, calibration_texts=None, calibration_teacher_labels=None)[source]

Fit a DisagreementGate from a labeled sample: run student on texts, label each example "disagree" where it differs from teacher_labels and "agree" otherwise, and distill a compact binary classifier of that target over the same hashed n-gram feature family the student itself uses (a different, wider/deeper recipe is fine – what matters is the classifier learns a decision surface over the input text, not that it matches the student’s exact recipe).

Parameters:

student (TaskModel)
texts (Sequence[str])
teacher_labels (Sequence[Any])
dim (int)
hidden (Sequence[int])
epochs (int)
lr (float)
seed (int)
alpha (float)
calibration_frac (float)
calibration_texts (Sequence[str] | None)
calibration_teacher_labels (Sequence[Any] | None)

Return type:

DisagreementGate

class UnionGate(*gates)[source]

Bases: object

Escalate if ANY constituent gate flags an input – composes a DisagreementGate with a real DensityGate (or any other ood_mask-exposing gate) with no changes to either gate’s own code.

Parameters:: gates (Any)

ood_mask(texts)[source]

Return the elementwise OR of all constituent gate masks.

Every constituent must return exactly one boolean per input. A gate returning a different length used to fail (or, when the lengths happened to broadcast, silently succeed on a recycled mask) inside the NumPy reduction rather than at the gate boundary, so an escalation policy could be reported from a prefix of the claimed evidence.

Raises:: ValueError – if any constituent gate returns a mask that is not 1-D of length len(texts).
Parameters:: texts (Sequence[str])
Return type:: ndarray