mixle.engines.packing module

Sub-byte bit-packing – genuine fp1/fp2/fp4 (and below-byte index) STORAGE compression in pure numpy.

The low-bit end of mixle’s spectrum is a storage win, not a CPU compute speedup (sub-byte arithmetic has no native CPU support and dequant-to-fp32 is slower than fp32; that fast-dequant kernel is the Cython/C tail). But the packing – cramming bits-wide codes into bytes – vectorizes cleanly with numpy shifts and is what actually shrinks the bytes on disk / on the wire. Power-of-two widths {1,2,4,8} pack exactly 8/bits codes per byte; this is the codec CodebookFormat and the low-bit float formats use to realize their advertised compression ratio.

pack_bits(codes, bits)

Pack unsigned codes (each < 2**bits) into a uint8 array, 8//bits per byte.

bits must be a power of two in {1, 2, 4, 8} (the widths that tile a byte exactly). Little-endian within each byte: code j of a group occupies bit positions [j*bits, (j+1)*bits).

Parameters:

• codes (Any)

• bits (int)

Return type:

ndarray

unpack_bits(packed, bits, count)

Inverse of pack_bits(): recover the first count codes as a uint64 array.

Parameters:

• packed (Any)

• bits (int)

• count (int)

Return type:

ndarray

packed_nbytes(count, bits)

Number of bytes pack_bits() produces for count codes of width bits.

Parameters:

• count (int)

• bits (int)

Return type:

int