mixle.reason.model module

Trainable cross-modal reasoning model with a shared latent.

This module learns per-modality encoders and decoders jointly from unlabeled multimodal records. The latent is not observed; training maximizes a Product-of-Experts variational lower bound for a multimodal VAE-style model.

  • Each modality m has an encoder q_m(z | x_m) = N(mu_m, diag(sig_m^2)).

  • The belief given any subset of modalities is the product of experts with the prior: precisions add, so more modalities produce a sharper belief. This matches mixle.inference.belief.GaussianBelief.fuse() with learned experts.

  • Each modality has a decoder p(x_m | z); training reconstructs every modality from the fused latent. Modality-subset subsampling lets inference work from one modality, all modalities, or any subset between them.

After training: belief(obs) returns q(z | available modalities) as a GaussianBelief, and predict(obs, target) generates a missing modality from the available ones. Uncertainty remains part of the object: the returned belief is a distribution, sharpened by each modality in proportion to its learned precision.

Torch is imported lazily; mixle.reason exposes this via a deferred attribute.

class CrossModalModel(latent_dim, *, seed=0)[source]

Bases: object

A multimodal Product-of-Experts VAE: one shared latent learned from many modalities, unsupervised.

Parameters:

  • latent_dim (int) – dimension of the shared latent z.

  • seed (int) – torch RNG seed.

add_modality(name, in_dim, *, hidden=(64,))[source]

Register a NEW modality with a learned encoder q(z|x) and decoder p(x|z).

MXR-080-0276: a duplicate name is rejected rather than silently replaced. Silently replacing a registered modality’s encoder/decoder previously left _fitted and any stored conformal radii intact, so a later prediction could combine a fresh, untrained replacement encoder/decoder with calibration computed against the OLD one and still advertise calibrated coverage. Use replace_modality() to deliberately replace an existing modality – it invalidates fitted/calibration state as part of the same operation.

Raises:

ValueError – name is already registered.

Parameters:
Return type:

CrossModalModel

replace_modality(name, in_dim, *, hidden=(64,))[source]

Deliberately replace a registered modality with a fresh, untrained encoder/decoder pair.

MXR-080-0276: unlike add_modality() (which rejects a duplicate name outright), this method exists specifically to allow overwriting an existing modality’s encoder/decoder – but doing so invalidates every dependent fitted/calibration artifact in the SAME operation (see _invalidate_fitted_state()): _fitted is cleared, training metadata is cleared, and every stored conformal radius is cleared, since each was computed against this modality’s prior encoder/decoder. Call fit() (and re-calibrate() any target) again before relying on beliefs/predictions/intervals.

Raises:

KeyError – name is not already registered.

Parameters:
Return type:

CrossModalModel

fit(data, *, epochs=600, lr=3e-3, beta=0.5, subsample=True)[source]

Train encoders and decoders jointly on unlabeled multimodal data.

data maps each registered modality name to an (N, in_dim) array (all modalities share the same N rows – row i is one record’s several views). beta weights the KL rate; with subsample=True the ELBO is also evaluated on each single-modality subset so the model can infer z from any one modality alone (the MVAE training trick).

MXR-080-0277: the complete aligned training table – equal row counts across every modality, each modality’s declared in_dim matching its data’s actual width, non-empty data, and finite values – is validated up front, before any per-modality normalization statistic or the model’s fitted state is touched. epochs is validated as a genuine positive int: a zero or negative value previously still set _fitted = True on encoders/decoders that were never actually updated (their random init weights, dressed up as a “trained” model).

Raises:

  • ValueError – data’s modalities don’t match the registered set; a modality’s data is empty, has the wrong width, contains non-finite values, or has a row count that disagrees with the other modalities; or epochs is not positive.

  • TypeError – epochs is not a plain int (a bool or a float, whole-valued or not, is rejected).

Parameters:
Return type:

CrossModalModel

belief(obs)[source]

The belief q(z | available modalities) as a GaussianBelief (product of experts).

Requires a prior fit() call. _fitted was tracked but never checked: a freshly constructed model’s encoders carry their random init weights, so every downstream consumer of this method – encode(), predict(), calibrate(), predict_interval() – could silently return a meaningless, randomly-initialized “belief” with no indication anything was wrong. Checked here once, since all four route through this method.

obs carries exactly ONE observation per modality: each value must be a 1-D array of length in_dim (MXR-080-0278). Previously np.atleast_2d silently accepted a (B, in_dim) batch, fused it through the product-of-experts as if it were legitimate – modalities with unequal batch sizes could broadcast into unintended cross-row pairings – and then returned only row 0, discarding the rest with no signal anything was dropped. No caller in this codebase passes a genuine batch here; call this once per observation.

Raises:

ValueError – some modality’s observation is not a 1-D array of the registered width.

Parameters:

obs (dict[str, Any])

Return type:

GaussianBelief

encode(obs)[source]

The posterior-mean latent code for obs (a shared-space embedding usable as store keys).

Parameters:

obs (dict[str, Any])

Return type:

ndarray

predict(obs, target)[source]

Generate the target modality from the modalities in obs (cross-modal generation).

obs takes exactly one observation per modality; see belief() (MXR-080-0278) – this method inherits that contract entirely through its call below.

Parameters:
Return type:

ndarray

calibrate(cal_data, target, *, alpha=0.1)[source]

Calibrate cross-modal prediction of target for finite-sample coverage (split conformal).

The holdout is SPLIT internally: the first half estimates the per-dimension residual scales, and only the second half is scored and ranked against those FIXED scales – the same fixed scales a fresh query is scored with, which is what makes the query’s score exchangeable with the ranked ones. Estimating the scales on the same residuals that get ranked broke that exchangeability, and the break was not cosmetic: with i.i.d. residuals and every stated assumption holding, measured joint coverage at nominal 0.90 was 0.854 in two dimensions, 0.584 in ten, and 0.179 in fifty (STAT-RR21-06) – each calibration residual helped set its own yardstick, so query scores ran systematically larger than the ranked ones. The radius is the standard conformal ceil((n_rank+1)(1-alpha))-th SMALLEST ranked score (ascending order statistic; an earlier docstring said “largest”, contradicting the implementation). Using the max over dimensions makes the guarantee simultaneous: predict_interval() returns a box whose joint coverage over the whole target vector is >= 1 - alpha – distribution-free, regardless of model specification (unlike the Gaussian posterior interval). Scope: that coverage is marginal over the calibration draw and the query jointly, under exchangeability of the calibration pairs and served traffic; it is not a per-query certainty, and distribution shift voids the statement silently – re-measure on drifted traffic. The split halves the effective ranking sample, so small holdouts reach +inf radii (an unbounded box) sooner – that is the honest price of a guarantee that actually holds.

MXR-080-0279: every split-conformal precondition is validated and the method fails closed – raises without storing anything – before a single score is computed. Previously, alpha outside (0, 1) could select a negative-indexed or otherwise unrelated order statistic instead of raising; an empty or wrong-width holdout produced an opaque broadcasting ValueError (or worse, silently proceeded on a mismatched pairing); and non-finite holdout values propagated into a NaN radius stored exactly like a valid one. A stored radius is later presented, via predict_interval(), as carrying a genuine finite-sample coverage guarantee, so it must never come from anything but a valid computation on a valid holdout.

Raises:

  • KeyError – target is not registered, or cal_data is missing a modality needed to predict it.

  • ValueError – alpha is not strictly between 0 and 1; the holdout is empty; a modality’s holdout width doesn’t match its registered in_dim; a modality’s holdout row count disagrees with the others; or any holdout value is non-finite.

Parameters:
Return type:

CrossModalModel

predict_interval(obs, target)[source]

A conformally-calibrated prediction box (lower, upper) for target given obs.

Requires a prior calibrate() call for target. Coverage is distribution-free and simultaneous: P(y in box) >= 1 - alpha jointly over the whole target vector – FOR THE PREDICTOR THAT PRODUCED THE CALIBRATION SCORES. Refitting clears the stored radii (STAT-RR22-08); the guarantee is additionally conditional on the encoders/decoders not being mutated in place between calibration and serving, which no runtime check can detect – recalibrate after any parameter update.

Parameters:
Return type:

tuple[ndarray, ndarray]

property modalities: Sequence[str]

Return modality names known to the cross-modal model.