mixle.task.capacity module

The capacity ladder: fit a student at each rung of increasing representation family, and report where it stops matching the teacher.

Distillation (mixle.task.distill) and recipe search (mixle.task.tune) both assume the student’s representation family is fixed (hashed n-grams) and search knobs within it. Some teachers need a richer family – a rule that generalizes across synonyms a hashed n-gram featurizer cannot see, for instance. capacity_ladder() climbs a small ordered set of representation families (“rungs”), measures each rung’s held-out agreement with the teacher, and returns the smallest rung that meets a target – or a measured “not capturable at these rungs” outcome with every rung’s ceiling attached, never an exception.

DEFAULT_RUNGS: tuple[str, ...] = ('hashed_ngram', 'embedding_head')

the two rungs this module can fit; later rungs are recognized but may be unavailable in this environment.

KNOWN_RUNGS: tuple[str, ...] = ('hashed_ngram', 'embedding_head', 'strong_encoder', 'small_lm')

every rung name this module understands, in increasing-capacity order (used by climb_to()).

class WordEmbeddingFeaturizer(vectors, dim, seed=0)[source]

Bases: object

Average per-word embedding vectors from a fixed lookup table – a dependency-free “embedding head” featurizer.

Unlike HashedNGram (which treats distinct surface tokens as unrelated hash buckets), two words given nearby vectors in vectors produce nearby features regardless of their spelling – the property a synonym-generalizing rule needs. A word missing from vectors falls back to a deterministic hashed sub-vector, so out-of-vocabulary text still produces a valid feature; it just earns no semantic generalization it was never given a vector for.

Parameters:
transform(texts)[source]

Map texts to normalized embedding features with hashed fallback rows.

Parameters:

texts (list[str])

Return type:

ndarray

to_spec()[source]

Serialize embedding vectors and fallback hashing settings.

Return type:

dict[str, Any]

classmethod from_spec(spec)[source]

Reconstruct the embedding featurizer from an artifact spec.

Parameters:

spec (dict[str, Any])

Return type:

WordEmbeddingFeaturizer

class EmbeddingHeadIO(featurizer, labels)[source]

Bases: _ClassifierIO

str -> label classifier over WordEmbeddingFeaturizer features – the “embedding_head” rung.

Parameters:

  • featurizer (WordEmbeddingFeaturizer)

  • labels (list[str])

class RungResult(rung, score, model, note='')[source]

Bases: object

One rung’s measured outcome: its held-out agreement score, the fitted student (if built), and a note.

Parameters:

  • rung (str)

  • score (float | None)

  • model (TaskModel | None)

  • note (str)

class LadderResult(target, rungs, winner)[source]

Bases: object

The ladder’s outcome: every rung’s measured score, and the smallest rung meeting target (or None).

Parameters:

  • target (float)

  • rungs (list[RungResult])

  • winner (str | None)

ceiling(rung)[source]

The measured score of rung, or None if that rung was unavailable in this environment.

Parameters:

rung (str)

Return type:

float | None

capacity_ladder(teacher_or_labels, texts, *, target, rungs=DEFAULT_RUNGS, val_texts=None, val_labels=None, labels=None, word_vectors=None, calibration_frac=0.3, n=3, dim=256, hidden=(64,), epochs=200, lr=1e-2, seed=0, device='cpu')[source]

Fit a student at each rung of rungs (increasing representation family) and measure held-out agreement.

teacher_or_labels is either a callable teacher (labels texts and, if given separately, val_texts) or a sequence of labels already aligned with texts – mirroring the distill/distill_from_labels duality. When val_texts/val_labels are not given, a calibration_frac held-out slice of (texts, teacher labels) is used (same split machinery as routing calibration), so a paraphrase/synonym generalization gap between train and held-out is measurable even with a single corpus.

word_vectors (word -> dense vector) is the only thing that makes the "embedding_head" rung semantically richer than "hashed_ngram" – without it, that rung still builds (never skipped, it is one of the two minimum rungs) but falls back to hashed features per out-of-vocabulary word, so it will not beat "hashed_ngram". "strong_encoder"/"small_lm" are recognized rung names with no estimator wired in this environment: they are skipped with a note, never raised as an error.

Returns a LadderResult with every rung’s measured score and either the smallest rung meeting target or winner=None with every built rung’s ceiling attached – “target unmet” is a valid result, never an exception.

Parameters:
Return type:

LadderResult

climb_to(fault, *, rungs=KNOWN_RUNGS)[source]

Given a refinement-loop fault localized to a saturated leaf’s current rung, return the next rung up.

fault is either a bare rung name or an object naming its current rung via a rung or dominant attribute (the shape diagnose()’s FaultReport will eventually carry) – this lets a caller climb straight to the next rung for the one saturated leaf, without re-running the whole ladder. Raises ValueError if the current rung is already the top of rungs.

Parameters:
Return type:

str