mixle.task.cascade module

Cascade serving with realized cost tracking and targeted retraining data.

Each request is answered locally when the CalibratedTaskModel is confident and in-distribution, and escalated to the teacher otherwise. The cascade tracks actual spend against a CostModel, so report() returns observed cost and savings relative to a teacher-only route.

Every escalated request marks a case where the local model deferred and the teacher supplied a targeted label. harvested() returns those (text, label) pairs for the next distillation run.

class CascadeStats(n_requests=0, n_escalated=0, escalated_texts=<factory>, escalated_labels=<factory>)

Bases: object

Running tally of how a cascade served traffic – the basis for realized cost and the harvest.

Parameters:

• n_requests (int)

• n_escalated (int)

• escalated_texts (list[Any])

• escalated_labels (list[Any])

property realized_escalation_rate: float

Return the observed fraction of requests escalated to the teacher.

class Cascade(model, teacher, *, cost=None)

Bases: object

Serve text -> label through a confident local model, escalating to the teacher when needed.

Parameters:

• model (CalibratedTaskModel)

• teacher (Callable[..., Any])

• cost (CostModel | None)

serve(texts)

Serve a batch of requests through the cascade.

Parameters:

texts (Sequence[Any])

Return type:

list[Any]

harvested()

Return escalated (texts, teacher_labels) as targeted retraining data.

Return type:

tuple[list[Any], list[Any]]

realized_cost()

Actual spend so far: c_local per request plus c_frontier per escalation (requires a CostModel).

Return type:

float

report()

Realized economics: requests, escalation rate, spend, and savings vs serving everything on the frontier.

Return type:

dict[str, Any]

plan(*, volume, n_label, max_escalation=None)

Project the lowest-cost route at volume using the realized escalation rate.

Parameters:

• volume (int)

• n_label (int)

• max_escalation (float | None)

Return type:

RoutePlan