mixle.task.regress module

solve_regression for calibrated replacement of numeric task functions.

The regression shape of the solve loop. teacher(x) -> float is the scorer/pricer/estimator being replaced; a small student learns it, and calibration is split conformal for regression: on held-out CALIBRATION rows the absolute-residual quantile qhat is set so the interval [yhat - qhat, yhat + qhat] covers the teacher’s answer with probability >= 1 - alpha (finite-sample, distribution-free, exchangeable inputs). The escalate-or-answer rule is a precision rule: answer locally only when the interval is tight enough for the caller’s purpose (qhat <= tol); otherwise run the real code. If the student cannot achieve the required precision, every request escalates.

What the coverage statement is – and what deployment does to it. The 1 - alpha statement is marginal over the calibration draw AND the query jointly. Split conformal’s randomness includes the calibration sample, so conditioning on the deployment decision – the gate event qhat <= tol is a function of the calibration draw – is selection, and coverage conditional on deploying is NOT guaranteed to be 1 - alpha. Exact counterexample (external review pass eleven): iid Uniform(0,1) residuals, n=19, alpha=0.1, tol=0.8 – the gate opens with probability 0.083, and coverage conditional on opening is 0.751, not 0.90. The effect shrinks as the calibration slice grows and as tol moves away from qhat’s sampling distribution, but it is not zero and this module does not certify it. What the deployed artifact carries instead is a MEASUREMENT: report() includes the empirical coverage of the current interval on the SELECTION rows, a slice the calibration quantile never touches. Both the guarantee and the measurement assume exchangeability and fail silently under distribution shift – re-measure on drifted traffic.

Two immutable holdout roles, and fresh calibration after every harvest. The held-out rows are split, deterministically at solve time, into CALIBRATION rows (they set the initial qhat and are never read by a promotion decision) and SELECTION rows (they decide every improve() promotion and produce the reported error/coverage measurements). Without the split, improve() chose the student with the same rows that then set its interval – expected coverage fell from 0.90 to 0.75 after twenty candidate selections in the reviewer’s exact construction (STAT-RR11-1). The split alone is NOT enough: the solve-time calibration rows also decide, through the deployment gate, whether requests are harvested at all, so the candidate’s training data is a function of that draw and the same rows cannot certify the candidate they helped construct – reusing them gave exactly 0.8857 coverage against the claimed 0.90 (STAT-RR12-1). Every promotion therefore recalibrates on FRESH rows that postdate the harvest (reserved from the harvest itself, or a caller-supplied evidence_inputs batch); calibration_evidence in report() names the regime that certified the current qhat. selection_uses counts how many promotion decisions the selection rows have made: after the first, the selection-side numbers are selection scores, not fresh held-out estimates (the same honesty rule as classification solve).

def price(item): … # the rigid pricing routine sol = solve_regression(price, items, tol=5.0) # dataset <- price(i); train; calibrate sol(item) # a float: local (width <= tol) or teacher sol.interval(item) # (yhat, lo, hi); marginal 1 - alpha coverage sol.improve(); sol.report() # promote on selection; calibrate on fresh rows

qhat is one global width, as in standard split conformal regression.

class RecordRegressionFeaturizer(dim=256, seed=0)[source]

Bases: object

Record featurizer for regression tasks.

HashedRecord (built for classification) squashes numerics through tanh, which saturates and erases the magnitude signal a regressor needs. Here numeric keys (learned from the fit sample) map to dedicated standardized columns; everything else uses hashing.

Parameters:
fit(records)[source]

Learn numeric-key normalization statistics from sample records.

Parameters:

records (list[Any])

Return type:

RecordRegressionFeaturizer

transform(records)[source]

Transform records into standardized numeric columns plus hashed categorical features.

Parameters:

records (list[Any])

Return type:

ndarray

to_spec()[source]

Serialize numeric-key statistics and hashing settings.

Return type:

dict[str, Any]

classmethod from_spec(spec)[source]

Reconstruct a regression featurizer from an artifact spec.

Parameters:

spec (dict[str, Any])

Return type:

RecordRegressionFeaturizer

featurizer_spec(f)[source]

Tagged, artifact-ready spec for the featurizers the solve shapes use.

Parameters:

f (Any)

Return type:

dict[str, Any]

featurizer_from_spec(spec)[source]

Reconstruct a task featurizer from a tagged artifact spec.

Parameters:

spec (dict[str, Any])

Return type:

Any

class RegressionSolution(net, featurizer, teacher, qhat, alpha, tol, holdout_mae, y_mean, y_scale, train_inputs=<factory>, train_ys=<factory>, cal_inputs=<factory>, cal_ys=<factory>, sel_inputs=<factory>, sel_ys=<factory>, selection_uses=0, calibration_evidence='solve-split', hidden=(64, ), epochs=300, lr=0.01, seed=0, n_requests=0, n_escalated=0, harvested_inputs=<factory>, harvested_ys=<factory>)[source]

Bases: object

A calibrated numeric student in front of the routine it replaces.

Parameters:
interval(x)[source]

Return (yhat, lo, hi) with calibrated teacher-answer coverage.

Parameters:

x (Any)

Return type:

tuple[float, float, float]

property answers_locally: bool

Whether the calibrated precision meets the tolerance at all (else everything escalates).

decide(x)[source]

Return the calibrated point estimate when local precision is sufficient.

If answers_locally is false, return None to signal escalation. Unlike __call__, this method never falls through to the teacher itself, so a Router tier can decide whether to escalate to the next tier.

Parameters:

x (Any)

Return type:

float | None

selection_coverage()[source]

Measured coverage of the CURRENT interval on the selection rows, with its denominator.

The selection rows never touch the calibration quantile, so for a freshly solved model this is an unbiased per-artifact measurement of the deployed interval’s coverage – the honest substitute for the deployment-conditional guarantee split conformal does not provide (see the module docstring). After improve() promotions the same rows have also chosen the model; selection_uses records that the number is then a selection score. Returns (covered_fraction, n_rows) or None when there are no selection rows.

Return type:

tuple[float, int] | None

report()[source]

Return calibration, precision, measurement, request, and harvest metrics.

Return type:

dict[str, Any]

save(path)[source]

Persist the network, featurizer, and calibration metadata.

Parameters:

path (str)

Return type:

str

classmethod load(path, teacher, *, device='cpu')[source]

Reconstitute a serving RegressionSolution (no training/calibration data; improve() raises).

Parameters:
Return type:

RegressionSolution

improve(evidence_inputs=None)[source]

Re-fit with harvested pairs; promote on SELECTION evidence, calibrate on FRESH evidence.

The promote/reject decision reads only the selection rows (STAT-RR11-1’s repair), and the deployed qhat is computed on rows that DID NOT EXIST before this round’s harvest began – never on the solve-time calibration rows. Reusing those rows leaks even without any direct comparison: the incumbent’s calibration-derived gate decides whether requests escalate at all, so whether a harvest exists – and therefore the candidate’s training data and whether this method can run – is a function of the old calibration draw, and a slice that shaped the candidate cannot also certify it. In the reviewer’s exact exchangeable construction, reusing the gate-contaminated slice gives 0.8857 coverage against the claimed 0.90, and fresh calibration restores 0.90 (STAT-RR12-1).

Fresh evidence comes from one of two places. Passing evidence_inputs teacher-labels a fresh batch and uses it as the calibration slice. Without it, the LAST max(ceil(1/alpha) - 1, len(harvest) // 4) harvested pairs are RESERVED as the calibration slice and excluded from candidate training; this is sound for the regression loop specifically because its gate is all-or-none – when the route escalates, EVERY request is harvested, so conditional on the deployed artifact the harvest is the raw serving stream, not a threshold-selected slice (contrast the classification loop, whose per-query-selected escalations cannot serve as calibration evidence). When the harvest cannot yet fund the reserved slice plus at least one training row, this returns False and keeps harvesting. On promotion the reserved slice BECOMES the artifact’s calibration data (calibration_evidence records the regime), so the next round reserves from the next harvest and no round ever reuses rows that predate its own harvest. A consequence of honest recalibration: the deployed qhat may grow across a promotion, because it is not the selection metric. selection_uses records each decision the selection rows make.

Parameters:

evidence_inputs (Sequence[Any] | None)

Return type:

bool

solve_regression(teacher, inputs, *, tol, alpha=0.1, holdout=0.25, kind=None, hidden=(64,), epochs=300, lr=1e-2, dim=256, prelabeled=None, seed=0, teacher_mode='auto')[source]

Replace a numeric routine with a conformally-calibrated student (see module docstring).

Parameters:

  • teacher (Callable[[...], Any]) – the numeric routine (teacher(x) -> float); labels the dataset, remains the fallback.

  • inputs (Sequence[Any]) – example inputs (text or dict/tuple records).

  • tol (float) – the caller’s precision requirement — answer locally only when the calibrated qhat <= tol. NOTE: the coverage statement is marginal over the calibration draw and the query jointly; conditioning on this deployment gate is selection over calibration draws and is NOT covered by the guarantee — see the module docstring, and read report()’s measured selection_coverage for the deployed artifact.

  • alpha (float) – interval miscoverage level (1 - alpha marginal coverage of the teacher’s answer).

  • holdout (float) – fraction reserved and split into two IMMUTABLE roles at solve time — conformal CALIBRATION rows (the larger half; set qhat, never read by promotions) and SELECTION rows (decide improve() promotions, produce the reported measurements).

  • prelabeled (tuple[Sequence[Any], Sequence[float]] | None) – already-teacher-labeled (inputs, values) — typically harvested escalations from a serving deployment — folded into the TRAINING split only, never calibration. The finite-sample guarantee of the resulting qhat requires the BASE inputs to be a genuinely fresh sample: re-solving with the same inputs and seed reproduces the exact same calibration rows (the split is deterministic, and prelabeled does not perturb it), and those rows gated the very serving run that harvested prelabeled – the STAT-RR12-1 leak in re-solve form. Supply fresh base inputs, or treat the re-solved interval as empirical rather than guaranteed.

  • kind (str | None)

  • hidden (Sequence[int])

  • epochs (int)

  • lr (float)

  • dim (int)

  • seed (int)

  • teacher_mode (str)

Return type:

RegressionSolution