mixle.task.collapse module¶
collapse_monitor – the shared collapse-detection utility for self-improvement loops.
Every self-improvement round claims to be getting better. This shared check evaluates that claim without each loop reimplementing collapse detection: across rounds, the held-out verified score must be non-decreasing, and the proposal diversity must not be shrinking. A loop that improves its score by collapsing onto a few candidates is overfitting to the verifier, not genuinely improving.
verdict = collapse_monitor(history) verdict.ok # True iff both checks hold across every round verdict.reason # None, or “score_decreased” / “diversity_shrunk” (which check failed)
history is one entry per round: a dict with the round’s held-out verified score and its pool of
candidates, or a precomputed diversity number.
- class CollapseVerdict(ok, reason, scores=<factory>, diversities=<factory>, failed_round=None)[source]
Bases:
objectThe result of
collapse_monitor():okplus which check failed, and the raw series.
- collapse_monitor(history, *, score_key='score', candidates_key='candidates', diversity_fn=distinct_count_diversity, score_tol=0.0, diversity_tol=0.0)[source]
Check a self-improvement round history for collapse: score non-decreasing and diversity not shrinking.
Each entry of
historysupplies the round’s held-out verified score underscore_keyand either its candidate pool undercandidates_key(diversity computed viadiversity_fn) or, whencandidates_keyis absent, a precomputed diversity number directly under"diversity".score_tol/diversity_tolallow a small, explicitly-named amount of round-to-round noise before a decrease/shrink counts as a real regression (0.0 = strict non-decreasing). The first round to violate either check ends the scan –reasonnames which check failed,failed_roundwhere.
- distinct_count_diversity(candidates)[source]
Diversity proxy: the number of distinct candidates (by
stridentity) in the round’s pool.