mixle.experimental.cvi module¶
P3 (experimental) – conjugate-computation variational inference (CVI), the theory under D-track.
Khan & Lin’s conjugate-computation VI updates a model in natural-parameter space: it takes a natural-gradient step of the ELBO, which for a conjugate exponential-family part reduces to the exact conjugate (Bayes / EM) update, and for a non-conjugate part becomes an ordinary gradient step – one update rule for the whole model. This is the principle the D-track’s closed-form / gradient split should be derived from rather than scheduled heuristically.
The self-contained, provable core (this module) is the first half of the card’s experiment: a
single natural-gradient CVI step with unit step size reproduces the exact conjugate update for
exponential-family leaves. We show it for three conjugate pairs – Normal-Normal, Beta-Bernoulli,
Gamma-Poisson – in their natural parameterization, plus that streaming the data in any chunking
gives the identical posterior (natural parameters are additive), and that a damped step
(rho < 1) converges to the same fixed point over iterations. The mixed conjugate+GradLeaf-flow
model and the iterations-to-F comparison are the torch follow-up.
Exploratory mixle.experimental code (P3 card).
- cvi_step(family, prior, data, *, rho=1.0, obs_var=1.0)[source]
One natural-gradient CVI step from
priorgivendata; returns the updated parameters.The step is
eta <- eta_prior + rho * (sum of data sufficient statistics). Withrho = 1this is exactly the conjugate posterior update (the EM M-step for a conjugate leaf).
- conjugate_posterior(family, prior, data, *, obs_var=1.0)[source]
The closed-form conjugate posterior parameters (the reference the CVI step must reproduce).
- damped_to_convergence(family, prior, data, *, rho=0.3, iters=200, obs_var=1.0)[source]
Repeated damped natural-gradient steps toward the posterior (rho<1 converges to the same fixed point).
Each step moves the natural parameter a fraction
rhoof the way from the current value to the full posterior natural parameter – a convex combination that converges to the posterior.