mixle.task.cascade module

Cascade serving with realized cost tracking and targeted retraining data.

Each request is answered locally when the CalibratedTaskModel is confident and in-distribution, and escalated to the teacher otherwise. The cascade tracks actual spend against a CostModel, so report() returns observed cost and savings relative to a teacher-only route.

Every escalated request marks a case where the local model deferred and the teacher supplied a targeted label. harvested() returns those (text, label) pairs for the next distillation run.

class CascadeStats(n_requests=0, n_escalated=0, escalated_texts=<factory>, escalated_labels=<factory>)

Bases: object

Running tally of how a cascade served traffic – the basis for realized cost and the harvest.

Parameters:

• n_requests (int)

• n_escalated (int)

• escalated_texts (list[Any])

• escalated_labels (list[Any])

property realized_escalation_rate: float

Return the observed fraction of requests escalated to the teacher.

class Cascade(model, teacher, *, cost=None, teacher_mode='batch')

Bases: object

Serve text -> label through a confident local model, escalating to the teacher when needed.

Parameters:

• model (CalibratedTaskModel)

• teacher (Callable[..., Any])

• cost (CostModel | None)

• teacher_mode (str)

serve(texts)

Serve a batch of requests through the cascade.

Parameters:

texts (Sequence[Any])

Return type:

list[Any]

serve_with_teacher_labels(texts, teacher_labels)

Serve a batch against one immutable snapshot of teacher outcomes.

This is the evaluation/replay counterpart to serve(): the caller obtains exactly one teacher label per request up front, and this method uses those labels only for local deferrals while updating the same traffic and harvest accounting as live serving.

Parameters:

• texts (Sequence[Any])

• teacher_labels (Sequence[Any])

Return type:

list[Any]

harvested()

Return escalated (texts, teacher_labels) as targeted retraining data.

Return type:

tuple[list[Any], list[Any]]

realized_cost()

Actual spend so far: c_local per request plus c_frontier per escalation (requires a CostModel).

Return type:

float

report()

Realized economics: requests, escalation rate, spend, and savings vs serving everything on the frontier.

Return type:

dict[str, Any]

plan(*, volume, n_label, max_escalation=None)

Project the lowest-cost route at volume using the realized escalation rate.

This is a POINT projection, not a forecast with uncertainty: it prices every route at the escalation rate realized on the traffic served so far and assumes that rate persists at volume. Distribution shift re-prices it silently – re-measure on drifted traffic before acting on a large-volume recommendation.

Parameters:

• volume (int)

• n_label (int)

• max_escalation (float | None)

Return type:

RoutePlan