mixle.task.disagreement module¶
Disagreement gate: escalate where the student has historically diverged from the teacher.
This is distinct from escalating only where inputs look statistically atypical
(DensityGate) or where the conformal set itself is ambiguous
(CalibratedTaskModel).
fit_disagreement_gate() turns a set of (text, student_label, teacher_label) triples into a compact
binary agree/disagree classifier over the student’s own feature space (reusing
distill_from_labels() – the disagreement gate is itself a distilled student,
just of a different target). The resulting DisagreementGate exposes ood_mask with the exact
same duck-typed shape as DensityGate, so it plugs into
CalibratedTaskModel(..., density_gate=...) directly – or unions with a real density gate via
union_gate() – with no changes needed to mixle.task.calibrate’s extension point.
measure_disagreement_mass() is the plain fraction-of-examples-where-student-differs-from-teacher
metric the active-labeling loop (active_distill()) is measured against: label the
gate-flagged region with the teacher, re-distill including those labels, and confirm the region’s mass
shrinks.
- measure_disagreement_mass(student, texts, teacher_labels)[source]
Fraction of
textswhere the student’s label differs from the teacher’s.
- class DisagreementGate(classifier, threshold=0.5)[source]
Bases:
objectA fitted agree/disagree classifier over the student’s feature space, plus an escalation threshold.
- Parameters:
classifier (TaskModel)
threshold (float)
- disagreement_proba(texts)[source]
P(disagree | x)under the fitted classifier.
- is_ood(text)[source]
Return whether one input is predicted to disagree with the teacher.
- fit_disagreement_gate(student, texts, teacher_labels, *, dim=256, hidden=(32,), epochs=150, lr=1e-2, seed=0, threshold=0.5)[source]
Fit a
DisagreementGatefrom a labeled sample: runstudentontexts, label each example"disagree"where it differs fromteacher_labelsand"agree"otherwise, and distill a compact binary classifier of that target over the same hashed n-gram feature family the student itself uses (a different, wider/deeper recipe is fine – what matters is the classifier learns a decision surface over the input text, not that it matches the student’s exact recipe).