mixle.task.economics module

Cost comparisons for local distillation, cascades, and teacher-only serving.

Every routing choice has a cost model. This module combines a conformal escalation rate (mixle.task.calibrate.CalibratedTaskModel.escalation_rate(), the empirical p_escalate) with unit costs:

• frontier-only – pay c_frontier for every request, forever.

• local-only – distill once (n_label teacher calls + training), then pay c_local per request.

• cascade – run the low-cost local model first, escalate only the ambiguous fraction: per request c_local + p_escalate * c_frontier, with the singletons covered at 1 - alpha.

break_even_volume() is the request count at which a distilled route recovers its one-time setup cost. recommend_route() picks the lowest-cost route at a given volume, optionally constrained by a maximum tolerated escalation rate, and reports the savings.

class CostModel(c_frontier, c_local=0.0, c_label=0.0, train_cost=0.0)

Bases: object

Unit costs in any consistent currency.

Parameters:

• c_frontier (float)

• c_local (float)

• c_label (float)

• train_cost (float)

setup_cost(n_label)

Return the one-time label and training cost for a local model.

Parameters:

n_label (int)

Return type:

float

cascade_cost_per_request(cost, p_escalate)

Expected per-request cost of the cascade: always run local, escalate the p_escalate fraction.

Parameters:

• cost (CostModel)

• p_escalate (float)

Return type:

float

break_even_volume(cost, n_label, *, p_escalate=0.0)

Requests after which a distilled route undercuts frontier-only (inf if it never does).

Setup is amortized against the per-request saving c_frontier - per_request(route). With p_escalate=0 this is the local-only break-even; pass the model’s escalation rate for the cascade break-even.

Parameters:

• cost (CostModel)

• n_label (int)

• p_escalate (float)

Return type:

float

class RoutePlan(route, volume, per_request, total, savings_vs_frontier, p_escalate, break_even, options)

Bases: object

Costed route comparison for a fixed request volume.

Parameters:

• route (str)

• volume (int)

• per_request (float)

• total (float)

• savings_vs_frontier (float)

• p_escalate (float)

• break_even (float)

• options (dict[str, float])

recommend_route(cost, *, volume, n_label, p_escalate, max_escalation=None)

Pick the lowest-cost route over volume requests.

local_only is offered only when the caller explicitly disallows escalation by setting max_escalation == 0. Otherwise the cascade route keeps local answers for calibrated inputs and escalates the remaining traffic to the teacher.

Parameters:

• cost (CostModel)

• volume (int)

• n_label (int)

• p_escalate (float)

• max_escalation (float | None)

Return type:

RoutePlan

select_alpha_for_cost(model, cal_texts, cal_labels, probe_texts, cost, *, volume, n_label, alphas=(0.01, 0.05, 0.1, 0.15, 0.2, 0.3))

Select alpha from a CostModel target.

The sweep connects recommend_route() to the calibration step so threshold selection reflects both model behavior and the caller’s cost assumptions.

model is anything with the CalibratedTaskModel shape: a mutable alpha attribute, calibrate(texts, labels), and escalation_rate(texts). For each candidate in alphas, this recalibrates model and measures its realized escalation rate on probe_texts (a held-out slice disjoint from cal_texts), then scores that escalation rate with recommend_route() over volume requests. The winner is the alpha whose recommended route is lowest-cost overall; model is left calibrated at that winning alpha. Returns (best_alpha, best_plan, plan_by_alpha) so the full sweep remains auditable.

Parameters:

• model (Any)

• cal_texts (Sequence[Any])

• cal_labels (Sequence[Any])

• probe_texts (Sequence[Any])

• cost (CostModel)

• volume (int)

• n_label (int)

• alphas (Sequence[float])

Return type:

tuple[float, RoutePlan, dict[float, RoutePlan]]