mixle.engines.formats module

Numeric format codecs across mixle’s precision spectrum: low-bit float, fixed-point, and codebook/VQ – the compression + “fixed precision” end, each with a provable error bound so a precision-allocation pass can pick the smallest format that preserves accuracy.

A NumericFormat is a lossy codec: quantize maps a float64 array to a compact representation, dequantize maps it back, and max_abs_error / max_rel_error bound the round trip. The bound is the “logic” a caller uses to spend minimal bits while keeping error under a target (see min_float_mantissa_bits()). Fixed-point and codebook codecs store an actually smaller array (real compression, vectorized in numpy); the float codec rounds to an n-bit float’s representable set to measure that band’s accuracy (true sub-byte bit-packing is the Cython/C tail).

class NumericFormat[source]

Bases: object

Base codec: quantize / dequantize plus round-trip error bounds and a storage bit count.

name = 'identity'
bits_per_value = 64.0
quantize(x)[source]
Parameters:

x (Any)

Return type:

Any

dequantize(q)[source]
Parameters:

q (Any)

Return type:

ndarray

round_trip(x)[source]
Parameters:

x (Any)

Return type:

ndarray

measured_max_abs_error(x)[source]

Empirical max absolute round-trip error over x (codecs also expose analytic bounds).

Parameters:

x (Any)

Return type:

float

compression_ratio()[source]

Stored bits per value relative to float64.

Return type:

float

class FloatFormat(mantissa_bits, exp_bits=11)[source]

Bases: NumericFormat

A float with mantissa_bits of mantissa – rounds x to that band’s representable set.

Below ~52 mantissa bits this is lossy (fp4/fp8/fp16 storage); the round trip measures the band’s accuracy. max_rel_error == 2**-(mantissa_bits+1) from round-to-nearest.

Parameters:

  • mantissa_bits (int)

  • exp_bits (int)

classmethod fp(total_bits)[source]

Build an n-bit float format with an IEEE-like exponent/mantissa split (n = 1..1024+).

Parameters:

total_bits (int)

Return type:

FloatFormat

property max_rel_error: float
quantize(x)[source]
Parameters:

x (Any)

Return type:

ndarray

dequantize(q)[source]
Parameters:

q (Any)

Return type:

ndarray

class FixedPointFormat(frac_bits, int_bits=31)[source]

Bases: NumericFormat

Fixed-point: store round(x * 2**frac_bits) as an integer; real compression + a hard error bound.

int_bits sets the representable magnitude [-2**int_bits, 2**int_bits); out-of-range clamps. max_abs_error == 2**-(frac_bits+1) (round-to-nearest), independent of x.

Parameters:

  • frac_bits (int)

  • int_bits (int)

property max_abs_error: float
quantize(x)[source]
Parameters:

x (Any)

Return type:

ndarray

dequantize(q)[source]
Parameters:

q (Any)

Return type:

ndarray

class CodebookFormat(codebook)[source]

Bases: NumericFormat

Scalar vector-quantization: store an index into a learned codebook; log2(K) bits per value.

The genuine pure-numpy compression codec – quantize gathers the nearest code (an unsigned index array), dequantize gathers the code values back. Fit the codebook to data with fit().

Parameters:

codebook (Any)

classmethod fit(data, n_codes, iters=25, seed=0)[source]

Learn n_codes codes by 1-D k-means (Lloyd) on data; codes are the cluster means.

Parameters:
Return type:

CodebookFormat

quantize(x)[source]
Parameters:

x (Any)

Return type:

ndarray

dequantize(q)[source]
Parameters:

q (Any)

Return type:

ndarray

compress(x)[source]

Quantize x and bit-pack the indices to bytes: returns (packed_uint8, count).

For K <= 16 codes the indices are sub-byte and pack 8//bits per byte, realizing the advertised compression (e.g. 16 codes -> 4-bit indices -> 2 values/byte -> 16x vs float64).

Parameters:

x (Any)

Return type:

tuple[ndarray, int]

decompress(packed, count)[source]

Inverse of compress(): unpack indices and gather the codebook back to float64.

Parameters:
Return type:

ndarray

min_float_mantissa_bits(target_rel_error)[source]

Smallest mantissa-bit count whose round-to-nearest relative error meets target_rel_error.

The error-tracing primitive: given a tolerated relative error, return the minimal float precision that preserves it – i.e. spend the fewest bits the accuracy budget allows.

Parameters:

target_rel_error (float)

Return type:

int