mixle.ppl.diagnostics module¶
Predictive model-comparison diagnostics for the mixle PPL: WAIC and PSIS-LOO.
Both estimate the expected log pointwise predictive density (elpd) – how well a fitted Bayesian model
predicts new data – from the pointwise log-likelihood matrix loglik of shape (n_draws, n_obs)
(the log-density of each observation under each posterior draw of the parameters).
waic– the Widely Applicable Information Criterion (Watanabe):elpd = lppd - p_waicwith the effective parameter countp_waicthe per-observation posterior variance of the log-likelihood.psis_loo– Pareto-Smoothed Importance-Sampling Leave-One-Out cross-validation (Vehtari, Gelman & Gabry 2017). Importance-reweights the full-data posterior to each leave-one-out posterior, smoothing the heavy importance-weight tail with a generalized-Pareto fit; it reports the diagnostic shapekhat(values above ~0.7 flag unreliable estimates).
Both return results on the deviance scale (waic/loo = -2 * elpd, lower is better) with a
standard error, matching the conventions of Stan / ArviZ / the R loo package.
- split_rhat(draws)[source]
Rank-normalized split-R-hat for one parameter (
drawsis(n_chains, n_draws)).Splits each chain in half (catching within-chain non-stationarity), rank-normalizes, then takes the potential scale reduction. Values within ~0.01 of 1.0 indicate convergence; > 1.01 is a warning.
- bulk_ess(draws)[source]
Bulk effective sample size: ESS of the rank-normalized draws (efficiency in the distribution body).
- tail_ess(draws)[source]
Tail effective sample size: the smaller of the 5% and 95% quantile-indicator ESS (tail efficiency).
- convergence_diagnostics(draws)[source]
Return
{'split_rhat', 'bulk_ess', 'tail_ess'}for one parameter’s(n_chains, n_draws)draws.
- waic(loglik)[source]
Return the WAIC of a
(n_draws, n_obs)pointwise log-likelihood matrix.
- psis_loo(loglik)[source]
Return PSIS-LOO of a
(n_draws, n_obs)pointwise log-likelihood matrix.
- loo_stacking_weights(pointwise_lpd, iters=2000, tol=1.0e-10)[source]
Return LOO stacking weights (Yao, Vehtari, Simpson & Gelman, 2018).
pointwise_lpdis an(n_obs, K)matrix of per-model pointwise LOO log-predictive densities (each column ispsis_loo(model_k)["pointwise"]). The returned simplex weightswmaximize the LOO log-score of the weighted predictive distribution,sum_i log(sum_k w_k * exp(lpd_ik)). This is concave inwand solved here by the standard mixture-weight EM update (no external optimizer), which respects the simplex by construction.
- loo_stack(logliks)[source]
Stack K candidate models by LOO predictive performance.
logliksis a sequence of(n_draws_k, n_obs)pointwise log-likelihood matrices over the same, aligned observations. Returns the stackingweights, the(n_obs, K)per-model pointwise LOO densities, each model’selpd_loo, and thestacked_elpd_looof the weighted predictive (which is >= the best single-model elpd_loo, since a one-hot weight is feasible).