mixle.task.imagine module

Verified structural proposal at a capacity ceiling.

When a capacity ladder reports that no rung in the current model class meets a target, a proposal step can generate a new structural candidate (a richer family, e.g. a mixture where the current class is a single component) outside what was tried. Every proposal is verified on held-out data before adoption (a proposal that improves train but not held-out is rejected as overfitting), and every proposal must name a genuine new INFORMATION SOURCE – a structural capability the starting class provably lacks – never adopted on train-improvement alone: a richer family with more free parameters can always fit train data better, so train improvement is not evidence of a real capability gain. A proposal naming no new information source is rejected regardless of any measured improvement.

ceiling = ceiling_report(current_class_held_out, target) # “no rung meets target” verdict = propose_structure(candidates, train, held_out, target) # verified-or-rejected, each

On a task with a known paradigm-shift fix (a capability the starting class provably cannot represent but a specific richer structure can), the proposer should find a verified structure that breaks the ceiling; a candidate with NO new information source is correctly rejected even where it would improve held-out. Treat a negative result (no verified candidate breaks the ceiling) as an expected outcome, not a failure to hide.

class CeilingReport(held_out_score, target, met)[source]

Bases: object

Whether the CURRENT structural class meets target on held-out data – the capacity ladder’s verdict, computed once before any new structure is proposed.

Parameters:
class ImagineResult(ceiling, verdicts=<factory>, breaks_ceiling=None)[source]

Bases: object

Capacity ceiling result plus candidate verdicts from structural imagination.

Parameters:

  • ceiling (CeilingReport)

  • verdicts (list[ProposalVerdict])

  • breaks_ceiling (str | None)

class ProposalVerdict(name, accepted, train_score, held_out_score, reason='')[source]

Bases: object

Evaluation verdict for one proposed structural candidate.

Parameters:
class StructuralCandidate(name, fit, new_information='')[source]

Bases: object

One proposed richer structure. new_information MUST name the specific capability the starting class provably lacks (e.g. “2-component mixture: represents a bimodal posterior a single Gaussian cannot”) – empty/None means “no new information source” and the candidate is rejected regardless of any measured improvement.

Parameters:
ceiling_report(held_out_score, target)[source]

Return whether held-out score reaches the requested target.

Parameters:
Return type:

CeilingReport

propose_structure(candidates, train, held_out, ceiling)[source]

Fit and verify each candidate in order. A candidate is accepted only if it names a genuine new information source and improves held-out score over the ceiling’s own held-out score (never train alone, since a richer family can always fit train better without a real capability gain). The first accepted candidate that also reaches ceiling.target breaks the ceiling.

Parameters:
Return type:

ImagineResult