mixle.task.cascade module

Cascade – the serving object where the savings are actually realized (and the loop that compounds them).

This ties the spine together into one callable that makes money. Each request is answered locally when the CalibratedTaskModel is confident and in-distribution, and only escalated to the expensive teacher when it is not. The cascade tracks actual spend against a CostModel, so report() is realized dollars saved versus frontier-only – not a projection.

The compounding part: every escalated request is a place the cheap model was unsure, and the teacher just answered it – a free, perfectly-targeted training label. harvested() returns those (text, label) pairs; feeding them back into distillation (see mixle.task.distill.distill()) shrinks the next model’s escalation rate, which lowers per-request cost, which widens the margin. The cascade gets cheaper the more it is used.

class CascadeStats(n_requests=0, n_escalated=0, escalated_texts=<factory>, escalated_labels=<factory>)[source]

Bases: object

Running tally of how a cascade served traffic – the basis for realized cost and the harvest.

Parameters:
n_requests: int = 0
n_escalated: int = 0
escalated_texts: list[Any]
escalated_labels: list[Any]
property realized_escalation_rate: float
class Cascade(model, teacher, *, cost=None)[source]

Bases: object

Serve text -> label cheaply: local model when confident, teacher otherwise; track spend, harvest labels.

Parameters:

  • model (CalibratedTaskModel)

  • teacher (Callable[..., Any])

  • cost (CostModel | None)

serve(texts)[source]
Parameters:

texts (Sequence[Any])

Return type:

list[Any]

harvested()[source]

The escalated (texts, teacher_labels) – targeted training data to re-distill a cheaper model.

Return type:

tuple[list[Any], list[Any]]

realized_cost()[source]

Actual spend so far: c_local per request plus c_frontier per escalation (requires a CostModel).

Return type:

float

report()[source]

Realized economics: requests, escalation rate, spend, and savings vs serving everything on the frontier.

Return type:

dict[str, Any]

plan(*, volume, n_label, max_escalation=None)[source]

Project the cheapest route at volume using the realized escalation rate (needs a CostModel).

Parameters:

  • volume (int)

  • n_label (int)

  • max_escalation (float | None)

Return type:

RoutePlan