mixle.ppl.guide module

User-declared structured variational inference: a Guide of per-latent variational factors.

This is the general counterpart to the packaged latent models (mixture/HMM/LDA): instead of calling a built-in model, you (1) write the generative model from PPL primitives, sharing a latent’s RandomVariable handle wherever it appears, (2) declare a Guide naming each latent and the variational factor that approximates it (the q-family), and (3) fit by mean-field VMP / coordinate-ascent VI. The result is a StructuredVIPosterior over the named latents, with a monotone ELBO.

The factorization is mean-field: each declared latent is an independent q-factor – that independence is the variational projection. The q-families are the conjugate exponential-family factors the VMP engine carries:

• 'gaussian' – a Normal latent (its mean),

• 'gamma' – a precision / positive latent (a Gamma in a scale slot),

• 'dirichlet'– a simplex latent (a Dirichlet).

Declare a family to make the projection explicit and have it checked against the model’s conjugate structure; omit it to take the conjugate default.

mu = Normal(0, 10) # shared latent handle (its mean ~ Gaussian q) tau = Gamma(1, 1) # precision latent (~ Gamma q) post = structured_vi([(Normal(mu, tau), data)], Guide(mu=mu, tau=tau)) post.mean(“mu”); post.posterior(“tau”); post.elbo

Latents shared across several observation factors combine their evidence (pass several (model, data) pairs). Coverage today is conjugate-exponential structured VI over Gaussian/Gamma/Dirichlet factors, including hierarchies and shared latents.

Admixtures / LDA-class models – a latent per-token categorical that indexes shared topics, drawn from a per-group Dirichlet – are expressed through admixture() (LDA is its categorical-word-emission special case), built from the same Dirichlet primitives by mean-field VI. Remaining gaps: non-conjugate q-families, and latent-feature (IBP-style) factors – those raise a clear error rather than a wrong answer.

class AdmixturePosterior(lam, gamma, topic_handles, ll_trace)

Bases: object

Posterior from admixture() (LDA-class mean-field VI). posterior(topic_handle) returns that topic’s fitted Dirichlet {'alpha','mean'}; topics() is E[beta] (K x V); doc_topics is the per-document mixing E[theta_d]; log_likelihood is the fitted-model corpus LL trace.

topics()

doc_topics(d=None)

posterior(topic)

summary()

Return type:

dict

class Guide(**latents)

Bases: object

A declared mean-field variational approximation: named latents, each an independent q-factor.

Build it with Guide(name=handle, ...) where each handle is the same RandomVariable object used in the model (latents are matched by object identity). To pin and validate the variational family, pass name=(handle, 'gaussian'|'gamma'|'dirichlet'); otherwise the conjugate factor implied by the latent’s prior is used.

Parameters:

latents (Any)

names()

Return type:

tuple[str, …]

class StructuredVIPosterior(gres, guide)

Bases: object

Posterior from structured_vi(): per-latent variational factors + the ELBO trace.

posterior(name) returns the factor’s hyperparameters (e.g. {'mean','sd'} for a Gaussian, {'alpha','mean'} for a Dirichlet, {'shape','rate','mean'} for a Gamma); mean / samples give the posterior mean / exact draws of that latent. Names are the guide’s keys (the latent handle itself is also accepted).

Parameters:

guide (Guide)

posterior(name)

Return type:

dict

mean(name)

samples(name, n=4000, rng=None)

Parameters:

n (int)

summary()

Return type:

dict

admixture(docs, topics, *, alpha=1.0, max_its=100, inner_its=40, tol=1e-5, seed=0)

Fit an admixture (LDA-class) model by mean-field VI built from this surface’s Dirichlet primitives.

docs is a corpus – a list of documents, each a sequence of integer word ids over the vocabulary. topics are the Dirichlet RandomVariable handles you declare as the per-topic variational factors q(beta_k); their prior alpha vectors set the vocabulary size V and the topic-word prior eta. alpha is the per-document topic Dirichlet prior. Returns an AdmixturePosterior.

This is “LDA via the guide”: LDA = an admixture whose emission is Categorical over words. The same coordinate-ascent (q(theta_d), q(beta_k) Dirichlet + categorical responsibilities q(z)) fits any admixture over a categorical vocabulary – no LDADistribution involved.

Parameters:

• max_its (int)

• inner_its (int)

• tol (float)

• seed (int)

structured_vi(observations, guide, *, max_its=300, tol=1e-8)

Fit a structured model by mean-field VMP / coordinate-ascent VI under a declared Guide.

Parameters:

• observations – a list of (model, data) pairs – each model is a PPL RandomVariable observation factor, data its observed values. Factors that reuse the same latent handle share that latent (their evidence is combined). A single factor may be passed as (model, data) directly.

• guide (Guide) – the Guide naming the latents to approximate and (optionally) their q-families.

• tol (float) – CAVI sweep budget and ELBO convergence tolerance.

• max_its (int)

• tol

Returns:

A StructuredVIPosterior over the guide’s latents (monotone ELBO).

Raises:

ValueError – if a guide latent is not actually an inferred latent of the model, or its declared q-family does not match the model’s conjugate factor (the projection constraint is checked).

Return type:

StructuredVIPosterior