mixle.task.density module

DensityGate – a real p(x) over inputs, so the model can escalate on “I’ve never seen this”.

Conformal sets calibrate which label is plausible, but a softmax over a ReLU net has no p(x) – it cannot tell a typical input from a wildly novel one, and will hand back a confident singleton for both. That residual is exactly what a describable random process fixes: fit a generative density over the input features (a diagonal-Gaussian mixture by EM – mixle’s home turf), and an input whose log p(x) falls below a calibrated floor is out-of-distribution -> escalate, regardless of how confident the classifier looks.

Pair it with mixle.task.calibrate.CalibratedTaskModel (which accepts a density_gate=): the cascade then escalates when the conformal set is ambiguous or the input is atypical – the union of “unsure which label” and “unlike anything I trained on”. The density is a fitted mixle distribution, so it serializes into the artifact and reloads identically.

class DensityGate(featurizer, density=None, log_threshold=None, calibration_receipt=None)

Bases: object

A generative density over featurized inputs with a calibrated out-of-distribution floor on log p(x).

The featurizer is any transform(list) -> matrix: HashedNGram for text, or HashedRecord for dict/tuple records (so record models get the same OOD protection).

Parameters:

• featurizer (Any)

• density (Any)

• log_threshold (float | None)

• calibration_receipt (dict[str, Any] | None)

fit(texts, *, calibration_texts=None, calibration_frac=0.2, n_components=4, alpha=0.02, max_its=60, min_covar=1e-3, seed=0)

Fit density on one slice and calibrate its OOD floor on a disjoint slice.

When calibration_texts is omitted, a reproducible internal split is made before fitting. An explicit calibration sequence lets a caller share an existing held-out split. The receipt records counts and content digests without retaining potentially sensitive examples.

Parameters:

• texts (Sequence[Any])

• calibration_texts (Sequence[Any] | None)

• calibration_frac (float)

• n_components (int)

• alpha (float)

• max_its (int)

• min_covar (float)

• seed (int)

Return type:

DensityGate

log_density(texts)

log p(x) of each input under the fitted density (higher = more typical of training data).

Parameters:

texts (Sequence[str])

Return type:

ndarray

is_ood(text)

True when the input is atypical: log p(x) below the calibrated floor.

Parameters:

text (str)

Return type:

bool

ood_mask(texts)

Return a boolean mask marking inputs below the calibrated density floor.

Parameters:

texts (Sequence[str])

Return type:

ndarray

to_spec()

Serialize the featurizer, fitted density, and threshold for task artifacts.

Return type:

dict[str, Any]

classmethod from_spec(spec)

Rebuild a density gate from to_spec() output.

Parameters:

spec (dict[str, Any])

Return type:

DensityGate