mixle.represent.embed module

Embeddings – learned maps from a unit to the shared R^dim space, discrete OR continuous, one interface.

An Embedding is anything with a dim and a .module() – a lazily-built nn.Module mapping a batch of units to (n_units, dim). A discrete unit (an id) is embedded by a lookup table (CategoricalEmbedding); a continuous unit (a patch, a window, an element-feature vector) by a small parametric encoder (FeatureEmbedding). Because both expose the same .module() handle, either can be shared across models (pass the same instance) exactly like CategoricalEmbedding – one code path ties discrete or continuous representations.

This is the “embedding” half of the tokenizer/embedding pair; the segmenter (mixle.represent.segment) produces the units, this maps them into the shared space, and an optional quantizer (mixle.represent.quantize) discretizes in that space when discrete tokens are wanted.

class CategoricalEmbedding(num_categories, dim, *, name=None)

Bases: object

A lazily-built learned embedding of shape (num_categories, dim); every consumer gets the same module.

Parameters:

• num_categories (int)

• dim (int)

• name (str | None)

module()

The underlying nn.Embedding – built on first call, the identical instance thereafter.

Return type:

Any

class FeatureEmbedding(in_features, dim, *, hidden=(), name=None)

Bases: object

A continuous unit encoder: (n_units, in_features) -> (n_units, dim) via a linear or small-MLP module.

The continuous analogue of CategoricalEmbedding – same dim / .module() contract, so it shares and trains identically. hidden=() is a single linear projection (a learned patch/ window/element embedding); non-empty hidden inserts ReLU layers.

Parameters:

• in_features (int)

• dim (int)

• hidden (Sequence[int])

• name (str | None)

module()

Return type:

Any