Task Serving, Routing, And Edge Deployment

The Task Distillation guide covers the basic teacher/student workflow: label with a teacher, train a local student, calibrate answer sets, and serve a cascade. mixle.task also contains the production-facing pieces around that loop: one-call replacement for label, numeric, multi-label, and dict-valued tasks, multi-tier routing, edge-device search, post-training quantization, harnesses for common legacy-code shapes, and scorecards.

One-Call Task Solving

solve points Mixle at a function that already performs a task and returns a deployable Solution.

from mixle.task import solve

def route_ticket(ticket):
    if ticket["amount"] > 10_000:
        return "finance-review"
    return "auto-approve"

solution = solve(route_ticket, historical_tickets, seed=0)
label = solution({"amount": 425.0, "country": "US"})
report = solution.report()

The teacher function labels examples. The student is trained and calibrated. At runtime the solution answers locally only when confident; otherwise it calls the teacher. Solution.improve folds harvested escalations into the next round and promotes only if verification does not regress.

Numeric Task Replacement

solve_regression handles routines that return numbers instead of labels: pricing functions, score calculators, risk estimates, sizing rules, or scientific surrogate functions.

from mixle.task import solve_regression

def price(item):
    base = {"basic": 20.0, "pro": 80.0, "max": 150.0}[item["tier"]]
    return base + 0.5 * item["size"]

solution = solve_regression(price, historical_items, tol=5.0, alpha=0.1)

value = solution({"tier": "pro", "size": 42.0})
yhat, lo, hi = solution.interval({"tier": "pro", "size": 42.0})

The teacher labels the examples exactly as in solve. The student is a small local regressor. Calibration is split conformal: on held-out calibration examples, Mixle computes an absolute-residual quantile qhat so [yhat - qhat, yhat + qhat] covers the teacher’s answer with probability at least 1 - alpha under the usual exchangeability assumption.

Runtime behavior is intentionally conservative. The local regressor answers only when qhat <= tol. If the calibrated interval is wider than the precision your application can tolerate, every request escalates to the teacher until enough harvested examples support a tighter promoted model.

Use this when the original code returns a scalar and the operational contract can be expressed as “local answers are acceptable within this tolerance.” Keep tol tied to the business, scientific, or safety requirement rather than to the model’s average error.

Multi-Label Task Replacement

solve_multilabel handles routines that return a set of labels: compliance flags, alert annotations, routing tags, document categories, moderation facets, or any task where several labels may be true at once.

from mixle.task import solve_multilabel

def flags(transaction):
    out = []
    if transaction["amount"] > 400:
        out.append("high-value")
    if transaction["region"] == "eu":
        out.append("eu-rules")
    return out

solution = solve_multilabel(flags, historical_transactions, alpha=0.1)

tags = solution({"amount": 525.0, "region": "eu"})
local_tags = solution.try_local({"amount": 60.0, "region": "us"})

The student uses one shared featurizer with a sigmoid head per label. On the calibration split, Mixle learns two bars for each label: an absent-label score bar above which the label is confidently present, and a present-label score bar below which the label is confidently absent.

Runtime behavior is conservative in the same way as solve and solve_regression. A label can be present, absent, or ambiguous. The whole request escalates when any label is ambiguous, and labels with too little calibration evidence are treated as ambiguous rather than guessed. A locally returned set is therefore made only of labels the calibrated student decided.

Structured Output Replacement

solve_structured handles routines that return a dictionary with a stable schema: enrichers, triagers, quote builders, extracted metadata, or any task where several named outputs must be produced together.

from mixle.task import solve_structured

def enrich(ticket):
    return {
        "route": "finance" if ticket["amount"] > 10_000 else "ops",
        "priority": "high" if ticket["age_hours"] > 24 else "normal",
        "reserve": ticket["amount"] * 0.15,
    }

solution = solve_structured(
    enrich,
    historical_tickets,
    tol={"reserve": 25.0},
    alpha=0.1,
)

output = solution({"amount": 12_500.0, "age_hours": 31})
maybe_local = solution.try_local({"amount": 500.0, "age_hours": 2})
report = solution.report()

Mixle calls the teacher once per training example, infers the output fields, and trains one calibrated sub-solution per field. Categorical fields use the same conformal singleton rule as solve. Numeric fields use the same split-conformal interval rule as solve_regression and require a scalar tol or a per-field tolerance dictionary.

The composition rule is strict. A structured output is returned locally only when every field can answer locally. One uncertain categorical label, one numeric field whose calibrated interval is too wide, or one under-calibrated field escalates the entire request to the teacher. That keeps the returned dictionary coherent instead of mixing trusted local fields with guessed ones.

Multi-Tier Routing

Router generalizes Cascade from one local tier plus teacher to several calibrated tiers.

from mixle.task import Router

router = Router.from_solutions(
    [tiny_solution, small_solution],
    teacher=frontier_teacher,
    costs=[0.0001, 0.001, 0.03],
    names=["tiny", "small", "frontier"],
)

y = router(request)
print(router.report())

Each tier must expose decide(x) and may return ESCALATE. The final tier is the teacher or frontier model and always answers. Requests answered by the teacher are harvested as targeted labels for the next solve round.

Use route_stack when you already have several solutions and per-request costs and want the tiers sorted cheapest-first.

Tool Calling

distill_tool_caller turns a teacher’s function-calling behavior into a tiny local selector plus per-tool argument extractors.

from mixle.task import ToolSpec, distill_tool_caller

tools = [
    ToolSpec("lookup_customer", ["customer_id"]),
    ToolSpec("refund", ["order_id", "amount"]),
]

caller = distill_tool_caller(frontier_tool_teacher, requests, tools, seed=0)
call = caller("refund order A-102 for 19.95")

The teacher returns {"tool": name_or_none, "args": {...}}. Mixle trains:

• a calibrated selector over the request text;

• one argument extractor per tool;

• an escalation path back to the teacher.

The local caller emits a tool call only when the selector is confident and all required arguments are present. If the selector abstains, the tool is unknown, or required arguments are missing, the caller escalates to the teacher and harvests the trace for later improvement.

This is single-step tool calling. Use planning when a request needs multiple verified steps. See Agentic Task Distillation for the full tool-calling and planning workflow.

Planning

distill_planner trains a sequence of calibrated next-step predictors from teacher traces. A plan is an autoregressive chain of tool calls ending in STOP.

from mixle.task import ToolSpec, distill_planner

tools = [
    ToolSpec("search", ["query"]),
    ToolSpec("summarize", ["document_id"]),
]

planner = distill_planner(frontier_planner, requests, tools, seed=0)
result = planner("find the latest policy and summarize it")

Each training trace is flattened into contexts of the form “request plus plan so far” and a next action. At runtime a step is accepted only when:

• the selector confidently chooses the next tool or STOP;

• the required arguments extract successfully;

• optional execution succeeds when an execute map is provided.

Any uncertainty, malformed step, missing argument, execution failure, or maximum-step exhaustion escalates the whole request to the teacher. The planner does not return half-trusted plans as if they were local successes.

Generative Planning And Traces

sft_planner trains a small causal LM to write an entire serialized plan, then gates the generated text with a strict parser, tool-spec validation, copy-fidelity checks, and optional grammar-constrained decoding. Use it when a single generated plan is a better abstraction than a sequence of next-step classifiers.

harvest_agent_traces can turn stored agent conversations into deterministic teachers for distill_tool_caller, distill_planner, or sft_planner. It infers ToolSpec objects from observed tool usage and exposes call_teacher and plan_teacher lookup functions.

See Agentic Task Distillation for examples and operational guidance.

Edge Deployment

Edge deployment is not only hyperparameter tuning. The model family itself may need to change to fit flash, latency, or runtime constraints.

Key objects:

• DeviceSpec declares hard constraints such as max_bytes, max_ops, and torch_free.

• EdgeFootprint records measured bytes, operation count, and torch dependency.

• EdgeSpace describes candidate student families and training recipes.

• DesignModel stores a surrogate over design choices and outcomes.

• distill_for_edge searches for the best feasible student.

• distill_designer compresses accumulated design knowledge into a tiny local model.

from mixle.task import DeviceSpec, distill_for_edge

device = DeviceSpec(max_bytes=128_000, max_ops=40_000, torch_free=True)
result = distill_for_edge(
    teacher,
    examples,
    device=device,
    seed=0,
)

student = result.model
print(result.footprint)

Use measure_inference_seconds and measure_ops_per_second on the target hardware when latency is a hard requirement.

Post-Training Quantization

quantize_mlp converts a trained Torch MLP student into a NumPy-only TaskModel with int8 or int4 weights.

from mixle.task import quantize_mlp

q8 = quantize_mlp(student, bits=8)
q4 = quantize_mlp(student, bits=4)

Quantized students use QuantizedMLP and QuantizedClassifierIO. They store arrays rather than Torch modules and can qualify for torch_free devices.

Version 0.6.2 tightened this artifact path: int4 weights are packed in the arrays payload, extreme outlier weights can be clipped before quantization with clip_percentile, and empty batches return correctly shaped probability arrays instead of failing during reshape.

lns_classifier and LNSStructuredClassifierIO provide integer log-space execution for structured students where the model is a sum of factor log-densities.

Harnesses For Existing Code

Harnesses package common replacement patterns:

• replace_extractor replaces a regex or parser that maps text to fields;

• replace_alerter replaces a threshold rule over sliding windows;

• replace_matcher replaces a deduplication or matching rule over pairs.

These are wrappers around the same safety contract: local answer only when calibrated; otherwise fall back to the original code.

from mixle.task import replace_matcher

matcher = replace_matcher(old_match_rule, example_pairs, seed=0)
result = matcher(record_a, record_b)
print(matcher.report())

Scorecards

scorecard measures a deployed student or router against the teacher it replaces.

from mixle.task import scorecard

card = scorecard(
    solution,
    teacher,
    heldout_inputs,
    student_cost=0.0001,
    teacher_cost=0.03,
    task="ticket routing",
)

print(card.table())

The scorecard reports end-to-end accuracy, local agreement, escalation rate, latency, artifact size, and blended cost when costs are provided.

Artifacts

Task artifacts are durable. Relevant helpers include:

• TaskModel;

• TaskManifest and SCHEMA_VERSION;

• save_json and load_json;

• save_arrays and load_arrays;

• save_module and load_module;

• register_builder and register_arrays_builder;

• read_manifest.

Use artifact helpers when adding a new student payload type. Ordinary users usually call TaskModel.save and TaskModel.load.

Calibrated artifacts preserve non-finite conformal thresholds such as qhat=inf with a JSON-safe sentinel and restore them as float("inf"). That keeps small or difficult calibration splits loadable without pretending the model is locally answerable.

Economics And Route Planning

Economic helpers include:

• CostModel;

• RoutePlan;

• cascade_cost_per_request;

• break_even_volume;

• recommend_route.

These functions make cost assumptions explicit. A cascade or router should report realized cost from actual traffic, while route planning estimates which deployment shape is worth trying.

API Reference

• mixle.task package

• mixle.task.solve module

• mixle.task.regress module

• mixle.task.multilabel module

• mixle.task.structured_out module

• mixle.task.generative_text module

• mixle.task.router module

• mixle.task.edge module

• mixle.task.quantize module

• mixle.task.toolcall module

• mixle.task.plan module

• mixle.task.sft_plan module

• mixle.task.constrained module

• mixle.task.traces module

• mixle.task.scorecard module

• mixle.task.harness module

Operational Standard

For a task-serving deployment, keep:

• the teacher definition or endpoint version;

• the examples used for labeling;

• the student recipe and artifact manifest;

• calibration data and alpha;

• escalation policy and OOD gate settings;

• scorecard results on held-out data;

• harvested escalations and labels;

• edge footprint when deploying outside a server environment.

That record makes it possible to improve a task model without quietly changing what the original task meant.