mixle.task.capacity module

The capacity ladder: fit a student at each rung of increasing representation family, and report where it stops matching the teacher.

Distillation (mixle.task.distill) and recipe search (mixle.task.tune) both assume the student’s representation family is fixed (hashed n-grams) and search knobs within it. Some teachers need a richer family – a rule that generalizes across synonyms a hashed n-gram featurizer cannot see, for instance. capacity_ladder() climbs a small ordered set of representation families (“rungs”), measures each rung’s held-out agreement with the teacher, and returns the smallest rung that meets a target – or a measured “not capturable at these rungs” outcome with every rung’s ceiling attached, never an exception.

DEFAULT_RUNGS: tuple[str, ...] = ('hashed_ngram', 'embedding_head')

the two rungs this module can fit; later rungs are recognized but may be unavailable in this environment.

KNOWN_RUNGS: tuple[str, ...] = ('hashed_ngram', 'embedding_head', 'strong_encoder', 'small_lm')

every rung name this module understands, in increasing-capacity order (used by climb_to()).

class WordEmbeddingFeaturizer(vectors, dim, seed=0)[source]

Bases: object

Average per-word embedding vectors from a fixed lookup table – a dependency-free “embedding head” featurizer.

Unlike HashedNGram (which treats distinct surface tokens as unrelated hash buckets), two words given nearby vectors in vectors produce nearby features regardless of their spelling – the property a synonym-generalizing rule needs. A word missing from vectors falls back to a deterministic hashed sub-vector, so out-of-vocabulary text still produces a valid feature; it just earns no semantic generalization it was never given a vector for.

Parameters:
transform(texts)[source]

Map texts to normalized embedding features with hashed fallback rows.

Parameters:

texts (list[str])

Return type:

ndarray

to_spec()[source]

Serialize embedding vectors and fallback hashing settings.

Return type:

dict[str, Any]

classmethod from_spec(spec)[source]

Reconstruct the embedding featurizer from an artifact spec.

Parameters:

spec (dict[str, Any])

Return type:

WordEmbeddingFeaturizer

class EmbeddingHeadIO(featurizer, labels)[source]

Bases: _ClassifierIO

str -> label classifier over WordEmbeddingFeaturizer features – the “embedding_head” rung.

Parameters:

  • featurizer (WordEmbeddingFeaturizer)

  • labels (list[str])

class RungResult(rung, score, model, note='', status='measured')[source]

Bases: object

One rung’s measured outcome or explicit record that it was not evaluated.

Parameters:

  • rung (str)

  • score (float | None)

  • model (TaskModel | None)

  • note (str)

  • status (str)

class LadderResult(target, rungs, winner, outcome)[source]

Bases: object

Every requested rung plus an outcome that distinguishes a measured ceiling from missing evaluation.

Parameters:

  • target (float)

  • rungs (list[RungResult])

  • winner (str | None)

  • outcome (str)

ceiling(rung)[source]

The measured score of rung, or None if that rung was unavailable in this environment.

Parameters:

rung (str)

Return type:

float | None

fully_evaluated()[source]

Whether every requested rung produced a held-out measurement.

Return type:

bool

capacity_ladder(teacher_or_labels, texts, *, target, rungs=DEFAULT_RUNGS, val_texts=None, val_labels=None, labels=None, word_vectors=None, calibration_frac=0.3, n=3, dim=256, hidden=(64,), epochs=200, lr=1e-2, seed=0, device='cpu', teacher_mode='auto')[source]

Fit a student at each rung of rungs (increasing representation family) and measure held-out agreement.

teacher_or_labels is either a callable teacher (labels texts and, if given separately, val_texts) or a sequence of labels already aligned with texts – mirroring the distill/distill_from_labels duality. When val_texts/val_labels are not given, a calibration_frac held-out slice of (texts, teacher labels) is used (same split machinery as routing calibration), so a paraphrase/synonym generalization gap between train and held-out is measurable even with a single corpus.

word_vectors (word -> dense vector) is the only thing that makes the "embedding_head" rung semantically richer than "hashed_ngram" – without it, that rung still builds (never skipped, it is one of the two minimum rungs) but falls back to hashed features per out-of-vocabulary word, so it will not beat "hashed_ngram". "strong_encoder"/"small_lm" are recognized rung names with no estimator wired in this environment: they are recorded as status="not_evaluated".

LadderResult.outcome is "target_met", "capacity_ceiling_measured" (all requested rungs were evaluated), or "not_evaluated" (the target was unmet but at least one requested family was unavailable). The latter must not be reported as a measured capacity ceiling.

teacher_mode declares a callable teacher’s calling convention, using the same vocabulary as mixle.task.distill._as_batched(): "batch" calls teacher(texts) once, "item" calls teacher(text) once per text. The default "auto" preserves the historical behaviour of discovering the convention by calling – which costs one extra, discarded invocation when the teacher turns out to be per-item (MXR-080-1895). Declare the mode for any teacher with side effects: a metered API, an audit log, or a request counter should not be invoked on an argument shape it was never written to accept, and "auto" cannot avoid that without calling first.

Parameters:
Return type:

LadderResult

climb_to(fault, *, rungs=KNOWN_RUNGS)[source]

Given a refinement-loop fault localized to a saturated leaf’s current rung, return the next rung up.

fault is either a bare rung name or an object naming its current rung via a rung or dominant attribute (the shape diagnose()’s FaultReport will eventually carry) – this lets a caller climb straight to the next rung for the one saturated leaf, without re-running the whole ladder. Raises ValueError if the current rung is already the top of rungs.

Parameters:
Return type:

str