mixle.doe.active module

Active learning and Bayesian optimal design: sequential design to learn, not to optimize.

Where Bayesian optimization places points to find an optimum, active learning places points to make a surrogate accurate everywhere, and Bayesian optimal design places them to learn model parameters.

Active learning (GP surrogate): * alm_scores() – Active Learning MacKay: the posterior predictive variance (pick the most

uncertain point). Cheap but myopic.

  • alc_scores() – Active Learning Cohn / IMSE: the integrated reduction in posterior variance a candidate would buy over a reference set – the principled criterion.

  • active_learning_design() – the sequential loop that grows an accurate surrogate.

Bayesian optimal design (parametric model): * expected_information_gain_linear() – the exact EIG of a linear-Gaussian model (= Bayesian

D-optimality), in closed form.

  • expected_information_gain_nmc() – the nested-Monte-Carlo EIG for a general nonlinear simulator, from a prior sampler and a log-likelihood.

The GP-based functions fit the torch surrogate; the EIG functions are pure NumPy.

alm_scores(gp, x, y, candidates)[source]

Active Learning MacKay scores: the GP posterior predictive variance at each candidate.

Parameters:
Return type:

ndarray

alc_scores(gp, x, y, candidates, reference)[source]

Active Learning Cohn / IMSE scores: integrated posterior-variance reduction per candidate.

Adding candidate c reduces the posterior variance at a reference point r by cov_post(r, c)^2 / var_post(c); this returns the sum over the reference set (the negative change in integrated MSE), so the maximizer is the most globally informative next point.

Parameters:
Return type:

ndarray

propose_active_learning(x, y, bounds, *, method='alc', n_candidates=512, n_reference=256, seed=None, gp=None, fit_kwargs=None)[source]

Propose the next active-learning point (method='alc' IMSE, or 'alm' max variance).

Parameters:
Return type:

ndarray

active_learning_design(objective, bounds, *, n_init=None, max_evals=40, method='alc', seed=None, fit_kwargs=None)[source]

Sequentially place points to maximize surrogate accuracy, returning the design and its responses.

Starts from a Latin-hypercube design, then repeatedly fits the GP and adds the most informative point (by method) until max_evals evaluations. Returns {'X', 'Y'} – a design tailored to learn objective well everywhere, not just near an optimum.

Parameters:
Return type:

dict[str, Any]

expected_information_gain_linear(model_matrix, *, noise=1.0, prior_cov=None)[source]

Exact expected information gain of a linear-Gaussian design y = F.theta + eps (= Bayesian D-opt).

For a Gaussian prior theta ~ N(0, Sigma0) and observation noise eps ~ N(0, noise^2 I), the mutual information between the data and theta is 0.5 * log det(I + noise^-2 Sigma0 F^T F). model_matrix is the (n, p) design matrix F (e.g. from mixle.doe.optimal.polynomial_features()). Higher EIG = a more informative design.

Parameters:
Return type:

float

expected_information_gain_nmc(prior_sampler, log_likelihood, simulate, *, n_outer=256, n_inner=256, seed=None)[source]

Nested-Monte-Carlo expected information gain for a general (nonlinear) design.

Estimates EIG = E_{theta, y}[ log p(y|theta) - log E_{theta'}[p(y|theta')] ] (Ryan 2003): draw outer theta_i ~ prior and y_i ~ p(y|theta_i) via simulate; the inner expectation is a mean over n_inner prior draws of exp(log_likelihood(theta', y_i)). prior_sampler(rng, n) returns (n, k) parameter draws; log_likelihood(thetas, y) returns a log-density per row of thetas at the single observation y; simulate(theta, rng) draws one y given theta.

Parameters:
Return type:

float