mixle.engines.qlut module

Quantized function lookup tables – every scalar nonlinearity becomes table[code].

The corollary of quantization: once an operand is the integer code round(x/step), any scalar function f is a precomputed table indexed by that code – no transcendental ever runs. This turns the nonlinear ops of a model (activations: sigmoid/tanh/GELU/SiLU/softplus; the exp/log that convert between the log (LNS) and linear domains; the Gaussian-log of mixle.engines.lns) into integer gathers. Combined with integer-add products and the integer logsumexp, a fully-quantized forward pass has no floating-point exp/log at all.

Nearest-code lookup has error <= (step/2) * sup|f'| on the tabulated range (e.g. sigmoid: 0.125*step). Unbounded functions (GELU, softplus, ReLU) extrapolate linearly beyond the range using the boundary slope, so the table need only cover the curved region. Measured 1.5-8x faster than the real transcendental.

class QuantizedFunction(func, step, lo, hi)[source]

Bases: object

A scalar function tabulated over quantized inputs; f(x) ≈ table[round(x/step)] with linear tails.

Parameters:

  • func (Callable[[np.ndarray], np.ndarray])

  • step (float)

  • lo (float)

  • hi (float)

lookup(code)[source]

Gather directly from integer codes already in the quantized domain (the pure-integer path).

Parameters:

code (Any)

Return type:

ndarray

max_abs_error(x)[source]

Empirical max absolute error vs the true function over x.

Parameters:

x (Any)

Return type:

float

quantized_activation(name, step=0.01, span=20.0)[source]

A common NN activation as a quantized LUT (sigmoid/tanh/gelu/silu/softplus/relu).

Parameters:
Return type:

QuantizedFunction

quantized_exp(log_step=0.01, lo_log=-30.0)[source]

exp from an LNS log-code (units of log_step) back to the linear domain – a pure table gather.

Call with integer log-codes via QuantizedFunction.lookup(); this is the softmax/attention “back to linear” step, with no real exp.

Parameters:
Return type:

QuantizedFunction

error_bound(sup_abs_derivative, step)[source]

The nearest-code lookup error bound (step/2) * sup|f'| (e.g. sigmoid sup|f’|=0.25).

Parameters:
Return type:

float

table_bytes(step, lo, hi, itemsize=8)[source]

Bytes a table spanning [lo, hi] at step occupies (cache-residency check).

Parameters:
Return type:

int

step_for_tolerance(tol, sup_abs_derivative)[source]

Largest step with error_bound(sup|f'|, step) <= tol – spend the fewest table entries.

Parameters:
Return type:

float