mixle.task.checkpoint_family_ladder module

J2: the checkpoint -> family ladder – iterate G3/J1 down a size ladder, receipted (roadmap J2).

Scope, read this first. J2 is explicitly named “thin orchestration” over machinery that is already built: J1’s unified compress() front door (itself wrapping G3’s coarsen() for the non-sampling/hybrid paths and the existing sampling-KD stack for the full-data path) and F10’s evaluate_checkpoint() / track_regression(). This module builds neither compression nor evaluation machinery; it drives J1 repeatedly down a sequence of decreasing target sizes (a small, laptop-sized stand-in for the roadmap’s real “70B -> 8B -> 1B -> edge” progression), collecting BOTH J1/G3’s own divergence receipts and a fresh F10 eval report at every rung, and gates each rung’s eval scores against the previous rung’s (or the headline’s) via F10’s own track_regression() – the same “score, don’t just threshold” GO/NO-GO shape mixle.task.pilot_ladder (F7) and mixle.task.capacity use for their own rung ladders.

Which of J2’s named dependencies this module can actually reach, from this worktree’s base (origin/pilot-ladder) and which it cannot:

  • J1 (unified compress() front door, PR #170) and F10 (eval harness, PR #184) are NOT ancestors of origin/pilot-ladder – both live on their own divergent branches (origin/compress-front-door, origin/eval-harness) that were never merged into this worktree’s base. Both are REQUIRED foundations for J2 (not optional opt-ins like F2/F5 were for F7), so unlike mixle.task.pilot_ladder’s treatment of its own unreachable pieces (document + raise NotImplementedError on opt-in), this module cannot merely skip them. Instead the exact files this module imports were pulled across via git show <branch>:<path> and committed alongside this module, byte-for-byte, from:

    • mixle/models/compress.py <- origin/compress-front-door (PR #170)

    • mixle/models/coarsening.py <- origin/compress-front-door (G3, PR #170’s own dependency)

    • mixle/models/sigma_weighted_projection.py <- origin/compress-front-door (G2, transitive dependency)

    • mixle/models/eval_harness.py <- origin/eval-harness (PR #184)

    plus their own test files (compress_test.py, coarsening_test.py, sigma_weighted_projection_test.py, eval_harness_test.py), run unmodified in this worktree to confirm the vendored copies are the exact, already-reviewed versions. Before vendoring, every OTHER transitive dependency compress.py needs (mixle/models/moment_propagation.py, mixle/task/acquire.py, mixle/task/bandit.py, mixle/task/distill_methods.py, mixle/models/transformer.py) was diffed against origin/compress-front-door’s copies and found byte-identical to what already lives on origin/pilot-ladder – so nothing else needed vendoring, and there is no version-skew risk between the vendored files and this branch’s existing ones. mixle/models/eval_harness.py has no repo-internal imports beyond numpy and is entirely self-contained.

A real constraint this discovered, not papered over: coarsen()’s output (CoarsenedLM) may contain MergedBlock instances in place of plain Block``s. ``MergedBlock does not expose the .attn/.ln1/.ln2/.mlp attributes coarsen()’s own internals (_block_branch) require – calling coarsen() a second time on an already-coarsened model raises AttributeError: 'MergedBlock' object has no attribute 'ln1' (confirmed directly: see this module’s own test file). So a literal CHAIN of compress() calls (rung i’s output model becomes rung i+1’s input) only works when rung i used method="sampling_kd" (a fresh architecture with plain Block``s) -- it is NOT generally safe for the default ``"non_sampling"/ "hybrid" methods this module defaults to. Rather than silently restricting the ladder to sampling_kd (which would abandon J2’s explicit “data-free except where receipts demand micro-calibration” requirement) or crash on the second rung, build_checkpoint_family() has every rung compress() the ORIGINAL headline model directly, with an increasingly generous divergence budget/trust_region per rung. This is a real, honest limitation of a single (non-chained) coarsen() pass: from n_layer blocks it can merge at most every adjacent pair once, i.e. reach ceil(n_layer / 2) at best – a real depth floor for one pass, not an unbounded cascade. The ladder still walks a genuinely DECREASING (non-increasing) sequence of measured model sizes; it just cannot go below that floor without a different method for that rung (a documented extension point, not built here).

class FamilyLadderResult(headline_eval, headline_n_params, rungs, halted_at, total_calibration_samples, calibration_pool_size)[source]

Bases: object

The whole ladder’s receipted outcome: the headline’s own eval report, every attempted rung, where (if anywhere) it halted, and the TOTAL real calibration-sample spend across the whole ladder – J2’s “total calibration data measured and reported” acceptance criterion.

Parameters:

  • headline_eval (EvalReport)

  • headline_n_params (int)

  • rungs (list[FamilyRung])

  • halted_at (str | None)

  • total_calibration_samples (int)

  • calibration_pool_size (int)

full_kd_equivalent_samples()[source]

What full sampling-KD would have cost had EVERY rung used it: pool_size * n_rungs – the denominator J1’s own <=1%-of-full-KD acceptance criterion is measured against, extended to a whole ladder rather than one call.

Return type:

int

total_calibration_fraction()[source]

total_calibration_samples / full_kd_equivalent_samples() – the real fraction of a full-sampling-KD-every-rung ladder this run actually spent, 0.0 if there were no rungs.

Return type:

float

class FamilyRung(name, real_target, model, n_params, compression_ratio, compression_receipt, non_sampling_receipts, eval_report, calibration_samples_spent, within_eval_budget, regression_flags=<factory>, reason='')[source]

Bases: object

One size-ladder step’s full receipted OUTPUT: the compressed model, J1/G3’s own divergence receipts, F10’s eval report, how many REAL calibration samples this rung spent, and whether it stayed within its stated eval budget.

Parameters:

  • name (str)

  • real_target (str)

  • model (Any)

  • n_params (int)

  • compression_ratio (float)

  • compression_receipt (CompressionReceipt)

  • non_sampling_receipts (dict[str, ScaleReceipt])

  • eval_report (EvalReport)

  • calibration_samples_spent (int)

  • within_eval_budget (bool)

  • regression_flags (list[RegressionFlag])

  • reason (str)

class RungSpec(name, real_target, method='hybrid', budget=1.0, trust_region=1.0, n_mc=32, sample_budget=None, hybrid_sample_fraction=0.01, hybrid_max_stages=3, hybrid_epochs=20, hybrid_lr=0.0005, kd_epochs=200, kd_lr=0.01, target_n_layer=None, seed=0, max_relative_eval_regression=0.15, regression_reference='prior')[source]

Bases: object

One size-ladder step’s INPUT: a documentary label for the real rung this stands in for, plus the compress() and eval-budget knobs for that step.

method defaults to "hybrid" – J1’s receipt-directed micro-calibration path – per J2’s “data-free except where receipts demand micro-calibration” requirement: every rung starts from G1/G3’s free, data-free coarsen() result and spends a REAL but tiny calibration budget only on the stages G1’s own closure-error receipts flag as poorly approximated (see mixle.models.compress’s _hybrid). budget/trust_region are the lever that makes successive rungs SMALLER (a more generous divergence budget lets coarsen() accept more depth merges); callers walk them in an increasing sequence across a RungSpec list to build a decreasing size ladder – this module does not choose that sequence for you (there is no honest way to invert “target size” to “divergence budget” without running coarsen() itself), matching G3’s own data-free, budget-not-size-parameterized interface.

Parameters:

  • name (str)

  • real_target (str)

  • method (str)

  • budget (float)

  • trust_region (float)

  • n_mc (int)

  • sample_budget (int | None)

  • hybrid_sample_fraction (float)

  • hybrid_max_stages (int)

  • hybrid_epochs (int)

  • hybrid_lr (float)

  • kd_epochs (int)

  • kd_lr (float)

  • target_n_layer (int | None)

  • seed (int)

  • max_relative_eval_regression (float)

  • regression_reference (str)

build_checkpoint_family(headline_model, rung_specs, *, calibration_data, eval_data=None, eval_seed=0, eval_n_examples=256)[source]

Walk rung_specs in order, compress()-ing headline_model at each rung’s own divergence budget, collecting J1/G3’s receipts plus a fresh F10 eval report, and gating progression on whether the rung’s eval scores stayed within its stated budget of the reference report (F10’s own track_regression(), mirroring F7’s GO/NO-GO gate one level up).

Every rung compress()-es the ORIGINAL headline_model (not the previous rung’s output) – see this module’s docstring for why chaining is unsafe for the default non_sampling/hybrid methods. The ladder still halts the first time a rung fails its own eval budget, exactly like mixle.task.pilot_ladder.run_pilot_ladder().

Parameters:

  • headline_model (Any)

  • rung_specs (Sequence[RungSpec])

  • calibration_data (Any)

  • eval_data (Any)

  • eval_seed (int)

  • eval_n_examples (int)

Return type:

FamilyLadderResult

count_params(model)[source]

Real parameter count of a torch module – the scalar every rung’s “target size” claim is measured against (never assumed from a nominal label).

Parameters:

model (Any)

Return type:

int