mixle.reason.embedding module¶
A rate-adaptive embedding whose active dimension scales with information content.
A fixed-width embedding wastes capacity on low-information inputs and truncates high-information
ones. This encoder learns a latent code with a variational per-coordinate posterior
q(z_k | x) = N(m_k(x), s_k(x)^2) and an ARD (automatic relevance determination) gate: a
coordinate whose posterior stays at its prior (KL(q || p) ~ 0) carries no information and is
inactive. The active dimension of an input is therefore #{k : KL(q(z_k|x) || p) > tau} –
it grows with the mutual information between the input and the latent.
Training is a rate–distortion (beta-VAE) objective: reconstruct the input subject to a rate budget
on the total KL. The beta knob sets bits-per-embedding; the data decides how those bits are
spent across coordinates, so a dense high-entropy input lights up more coordinates than a sparse one.
CONTRACT (MXR-080-0282): one instance is a plain autoencoder over ONE fixed input feature space. All
inputs it is fit and later queried on share that one instance’s ordered coordinate system, so codes
produced by THE SAME FITTED INSTANCE are comparable to each other and can index a
mixle.reason.CrossModalStore as one corpus’s retrieval keys. This class has no modality
identity, no paired views, no contrastive/alignment objective, and no modality-specific front ends –
it does NOT align codes across different instances. Two independently-fit ScaledEmbedding``s
(e.g. one per modality) converge to arbitrary, unrelated latent rotations, so comparing a code from
one instance against a code from another (cosine similarity, nearest neighbor, shared coordinates,
...) is meaningless, even when both share the same ``max_dim. For genuine cross-modal comparison,
use a mechanism built for it – e.g. mixle.reason.CrossModalJoint (a shared mixture-latent
joint over named modalities) or Bayesian evidence fusion via mixle.reason.reason() – not
independently-fit embeddings.
Torch is imported lazily; mixle.reason exposes this via a deferred attribute.
- class ScaledEmbedding(in_dim, max_dim=16, *, hidden=(64,), beta=1.0, kl_tau=1e-2, seed=0)[source]
Bases:
objectA beta-VAE-style rate-adaptive embedding with an ARD gate giving a data-dependent active dim.
One instance covers a single fixed input feature space – see the module docstring’s CONTRACT (MXR-080-0282): codes are comparable within one fitted instance, never across independently-fit instances (e.g. one per modality has its own, unrelated latent rotation).
- Parameters:
in_dim (int) – input feature width.
max_dim (int) – the embedding’s maximum width (upper bound on active dimension).
hidden (tuple[int, ...]) – hidden widths shared by the encoder and decoder trunks.
beta (float) – rate weight in the ELBO (larger -> tighter rate budget -> fewer active dims). Must be finite and non-negative.
kl_tau (float) – per-coordinate KL threshold (nats) above which a coordinate counts as active. Must be finite and non-negative.
seed (int) – torch RNG seed.
- fit(X, *, epochs=400, lr=3e-3, weight_decay=0.0)[source]
Train the embedding on unlabeled inputs
X((n, in_dim)) by the beta-VAE ELBO.Xmust be non-empty, finite, and exactlyin_dimwide (MXR-080-0280).epochsmust be a positive int: at least one optimizer step must run before the network’s (otherwise still at its random initialization) weights are certifiedfitted.
- encode(X)[source]
The embedding means
(n, max_dim)– this instance’s code (use as a store’s keys).Only comparable to other codes from THIS fitted instance – see the module docstring’s CONTRACT (MXR-080-0282).
- coordinate_kl(X)[source]
Per-coordinate KL from the prior,
(n, max_dim)(nats) – how much each coord encodes.
- active_dim(X)[source]
Per-input active dimension: number of coordinates whose KL exceeds
kl_tau.