mixle.task.router module

Router – calibrated N-tier model routing: tiny models first, the frontier only when necessary.

The multi-tier generalization of Cascade, and the honest version of “LLM routing”: each tier is a calibrated task model that answers only when its conformal set is a confident singleton and (if gated) the input is in-distribution — otherwise the request falls through to the next tier, ending at the frontier/teacher, which always answers. Routing decisions carry coverage guarantees, not learned vibes; the report carries realized cost, not projections:

router = Router.from_solutions([tiny, small], teacher=frontier, costs=[0.0001, 0.001, 0.03])
router(x)                     # answered by the cheapest tier that is SURE
router.report()               # per-tier traffic, realized $/req, savings vs frontier-only
router.harvested()            # the frontier's answers on hard inputs = training data for the tiers

Every request the frontier answers is a teacher-labeled example exactly where the local tiers were unsure — feed harvested() back through solve(prelabeled=...) and the routing gets cheaper.

class TierStats(name: 'str', cost_per_request: 'float', answered: 'int' = 0)[source]

Bases: object

Parameters:
name: str
cost_per_request: float
answered: int = 0
class RouterStats(tiers: 'list[TierStats]' = <factory>, harvested_inputs: 'list[Any]' = <factory>, harvested_labels: 'list[Any]' = <factory>)[source]

Bases: object

Parameters:
tiers: list[TierStats]
harvested_inputs: list[Any]
harvested_labels: list[Any]
property n_requests: int
class Router(tiers)[source]

Bases: object

Route each request to the cheapest tier whose calibrated model is confident; the last tier always answers.

Parameters:

tiers (list[tuple[str, Any, float]])

classmethod from_solutions(solutions, teacher, *, costs, names=None)[source]

Build from Solution objects (cheapest-first) + the frontier callable.

costs has one entry per solution plus one for the teacher (per-request).

Parameters:
Return type:

Router

serve(xs)[source]
Parameters:

xs (Any)

Return type:

list[Any]

harvested()[source]

The frontier-answered (inputs, labels) — targeted training data for the cheaper tiers.

Return type:

tuple[list[Any], list[Any]]

report()[source]

Per-tier traffic and REALIZED economics vs sending everything to the final tier.

Return type:

dict[str, Any]

summary()[source]
Return type:

str

route_stack(solutions, teacher, *, costs)[source]

Convenience: Router.from_solutions() with tiers sorted cheapest-first by cost.

Parameters:
Return type:

Router