mixle.models.sorted_profile_quantizer module

Sorted-profile (permutation x monotone) quantizer (roadmap G4): head-exact + parametric-tail per tensor.

Per the R1 copula note (roadmap doc, R1 -> G4, F6, I2, H4), any flattened tensor value vector v decomposes as v = P . s: a sorted profile s (the empirical quantile function) composed with a permutation P (the arrangement mapping sorted rank back to original position). H4’s mixle/experimental/tying_discovery.py (tensor_profile / profile_distance, see that module’s docstring) already uses the marginal half of this decomposition – a fixed-length RESAMPLED profile – as a tying-discovery signal, and deliberately throws the permutation away. G4 keeps BOTH halves and turns the decomposition into an actual per-tensor storage format:

• s is not stored as a raw sorted array – it is FIT as a parametric mixle distribution (reusing this codebase’s real mixle.stats/mixle.inference.estimate machinery, not a hand-rolled curve fit), so the non-outlier bulk of the tensor collapses to a handful of distribution parameters instead of one float per element;

• P is stored as literal permutation indices (an integer array) – per the R1 note’s honest acknowledgment that “arbitrary permutations are gather ops”: there is no closed-form compact encoding of an arbitrary permutation short of n*log2(n) bits, so this module does not pretend otherwise. The sort itself is an exact, free (deterministic, non-iterative) operation – unlike G2’s mixle.models.sigma_weighted_projection.sigma_weighted_permutation(), no Sinkhorn/OT solver is needed here, because there is nothing to OPTIMIZE: sorting a tensor’s own values against itself has one unambiguous answer. (G2’s Sinkhorn permutation solver is for the DIFFERENT problem of matching one tensor’s rows to another’s under a Sigma-weighted cost – not reused here.)

• the head (top-k largest-magnitude values) is carved out and stored EXACTLY before any of the above, because outliers are exactly where a smooth parametric quantile fit is worst – this is the “head-exact” half of “head-exact + parametric-tail”;

• a per-tensor goodness-of-fit RECEIPT (a real, computed Kolmogorov-Smirnov statistic, reusing mixle.utils.evaluation.ks_test() rather than a hand-rolled discrepancy measure) is attached to every encoding, and a bad receipt triggers a DENSE FALLBACK rather than silently accepting a bad lossy fit.

Honest scope (do not read this module as a general weight quantizer): the roadmap doc scopes G4 to exactly three use cases –

1. optimizer states (F6) – e.g. Adam’s second-moment buffer, which is positive, heavy-tailed, and mostly smooth (a good match for a Gamma/log-normal-family tail fit); this module builds the mechanism generically enough to apply there without F6 itself existing yet;

2. KV-cache tails (E2/I2) – same story, not built here;

3. anomaly detection (detect_anomaly()) – the goodness-of-fit receipt IS the anomaly signal: a tensor that suddenly stops matching its own historical value-profile family is itself worth flagging.

Hardware reality (R1): arbitrary permutations are memory-bound gather ops with no FLOP savings, so this scheme is honestly a STORAGE/regularization/receipt-structure win (real when the permutation indices fit in fewer bits than the values they replace – e.g. uint16 indices against float32 values for tensors under 65536 elements) rather than a speed win, unless restricted to block forms that map to tensor cores (not attempted here).

class SortedProfileEncoding(shape, top_k_values, top_k_indices, tail_distribution, permutation_indices, goodness_of_fit, used_dense_fallback, dense_values=None, n_tail=0, _index_dtype=<factory>)

Bases: object

Storage format for one tensor’s sorted-profile (permutation x monotone) encoding.

Either the used_dense_fallback=False branch (top_k_* / tail_distribution / permutation_indices populated, dense_values=None) or the used_dense_fallback=True branch (dense_values populated, the rest None/empty) is populated – never both – so reconstruct() can dispatch on the flag alone.

Parameters:

• shape (tuple)

• top_k_values (ndarray | None)

• top_k_indices (ndarray | None)

• tail_distribution (Any | None)

• permutation_indices (ndarray | None)

• goodness_of_fit (float)

• used_dense_fallback (bool)

• dense_values (ndarray | None)

• n_tail (int)

• _index_dtype (dtype)

shape

Original tensor shape (reconstruction reshapes back to this).

Type:

tuple[int, …]

top_k_values

Exact values of the top-k largest-magnitude entries (“head-exact”). None/empty when used_dense_fallback.

Type:

np.ndarray | None

top_k_indices

Flat indices (into the original tensor, C order) the top_k_values came from.

Type:

np.ndarray | None

tail_distribution

A fitted mixle.stats distribution object (exposing .cdf and .quantile) over the non-outlier (“tail”) values – the parametric replacement for storing those values directly.

Type:

Any | None

permutation_indices

Length-n_tail array of flat original indices, ordered so that permutation_indices[r] is where the r-th smallest non-outlier value belongs. This IS the permutation P in v = P . s.

Type:

np.ndarray | None

goodness_of_fit

KS D-statistic between the fitted tail_distribution and the actual non-outlier values (0 = perfect fit; see DEFAULT_GOF_THRESHOLD). Set even when used_dense_fallback (it is the receipt that CAUSED the fallback), so the receipt itself is never silently thrown away.

Type:

float

used_dense_fallback

True if the fit was rejected and the tensor is stored densely instead.

Type:

bool

dense_values

The full flattened tensor, only populated when used_dense_fallback.

Type:

np.ndarray | None

n_tail

Number of non-outlier elements (= permutation_indices.size in the non-fallback case; kept explicitly so nbytes/receipts are meaningful in the fallback case too).

Type:

int

property size: int

Total element count of the original tensor.

nbytes()

Measured storage footprint of the encoding, in bytes.

Dense fallback: exactly the byte count of dense_values (float32). Otherwise: top-k exact values (float32) + top-k indices (minimal dtype) + permutation indices (minimal dtype) + _DISTRIBUTION_PARAM_BYTES for the fitted tail distribution + one float32 for the goodness-of-fit receipt itself (a real, non-decorative receipt is part of what is shipped).

Return type:

int

class AnomalyReport(ks_statistic, reference_goodness_of_fit, is_anomaly)

Bases: object

Result of scoring a new tensor against a reference encoding’s tail-distribution family.

Parameters:

• ks_statistic (float)

• reference_goodness_of_fit (float)

• is_anomaly (bool)

ks_statistic

KS D-statistic of the new tensor’s non-outlier values against the REFERENCE encoding’s fitted tail_distribution (the family is held fixed; only the data changes – this is a re-SCORING, not a re-fit).

Type:

float

reference_goodness_of_fit

The reference encoding’s own receipt, for context.

Type:

float

is_anomaly

Whether ks_statistic has degraded significantly relative to reference_goodness_of_fit (see detect_anomaly() for the exact rule).

Type:

bool

fit_sorted_profile(tensor, top_k=0, tail_family=None, gof_threshold=DEFAULT_GOF_THRESHOLD)

Encode tensor as head-exact outliers + a fitted parametric tail distribution + permutation.

Parameters:

• tensor (Any) – A torch tensor or numpy array of any shape.

• top_k (int) – Number of largest-magnitude entries to carve out and store EXACTLY (“head-exact”), before any fitting happens – outliers are exactly where a smooth parametric quantile fit is worst, so they are never asked to survive the parametric tail model. 0 disables head-exact storage entirely (the whole tensor goes through the tail fit).

• tail_family (Any) – A mixle.stats ParameterEstimator instance (e.g. GaussianEstimator(), GammaEstimator()) used to fit the non-outlier values via mixle.inference.estimate. Defaults to GaussianEstimator(). Pick a family whose support matches the tensor’s actual values – e.g. GammaEstimator() for a strictly-positive optimizer second-moment buffer, per F6’s honest scope note (see module docstring); a mismatched family is not silently accepted – it is caught by the goodness-of-fit receipt below and triggers the dense fallback.

• gof_threshold (float) – Maximum acceptable KS D-statistic (see DEFAULT_GOF_THRESHOLD) before falling back to dense storage.

Returns:

either a populated head/tail/permutation encoding (used_dense_fallback=False) or a dense fallback (used_dense_fallback=True), always carrying the real, computed goodness_of_fit receipt either way.

Return type:

SortedProfileEncoding

reconstruct(encoding)

Invert a SortedProfileEncoding back to an (approximate, or dense-exact) tensor.

The head (top-k outliers) is EXACT in both branches (either stored verbatim, or – in the dense fallback case – simply part of the densely-stored tensor). The tail is exact under dense fallback and approximate (reconstructed from the fitted parametric quantile function) otherwise.

Returns:

float32 array reshaped to encoding.shape.

Return type:

np.ndarray

Parameters:

encoding (SortedProfileEncoding)

detect_anomaly(tensor, reference_encoding, ratio_threshold=DEFAULT_ANOMALY_RATIO, abs_margin=DEFAULT_ANOMALY_ABS_MARGIN)

Anomaly-detection use of the goodness-of-fit receipt (roadmap G4, use case 3).

A tensor that historically fit reference_encoding.tail_distribution’s family well and suddenly stops fitting it – a burst of extreme values, a distribution shift – is itself an anomaly signal, independent of whatever downstream task the tensor feeds. This function re-SCORES tensor against the reference’s ALREADY-FITTED family (it does not fit a new distribution to tensor), then compares the resulting KS D-statistic to the reference’s own receipt.

The new tensor’s outliers are excluded using the reference encoding’s own top-k COUNT (not its specific indices, which belong to a different tensor) so the comparison is apples-to-apples with how the reference receipt itself was computed.

Flagging rule: is_anomaly fires when the new D-statistic exceeds max(ratio_threshold * reference_goodness_of_fit, reference_goodness_of_fit + abs_margin) – a ratio threshold alone breaks down when the reference D is already tiny (sampling noise alone can double it), so it is combined with an absolute floor. Both directions are meaningful test cases: a similarly-distributed new draw should score close to (or even below) the reference’s own receipt; a genuinely shifted or outlier-contaminated tensor should score well past the combined threshold.

Returns:

AnomalyReport

Parameters:

• tensor (Any)

• reference_encoding (SortedProfileEncoding)

• ratio_threshold (float)

• abs_margin (float)

Return type:

AnomalyReport