mixle.doe.entropy module¶
Information-theoretic Bayesian optimization: Max-value Entropy Search (MES).
The improvement-based acquisitions (EI/PI/UCB) score a candidate by how much it might beat the current
best. MES (Wang & Jegelka 2017) instead scores a candidate by how much evaluating it would reduce the
entropy of the global optimum value y* = max f – the mutual information I(y; y* | x). It is
often more sample-efficient and is cheap: with a GP, I(y; y*|x) has a closed form per sampled
y* (a truncated-Gaussian entropy), and plausible y* are drawn by fitting a Gumbel to the
distribution of the maximum over a candidate set.
max_value_entropy_search is pure NumPy (given posterior moments and y* samples); the
propose_mes() driver fits the torch GP surrogate, so it needs PyTorch.
- sample_max_values(mean, std, n_samples=64, *, seed=0)[source]
Sample plausible global-max values
y*via the Gumbel approximation (Wang & Jegelka 2017).The CDF of the maximum over the candidate cloud is
P(max <= y) = prod_i Phi((y - mu_i)/sd_i); a Gumbel is fit to its 25/50/75 percentiles (found by bisection) and sampled. Returns an(n_samples,)array ofy*draws (never below the best posterior mean).
- max_value_entropy_search(mean, std, max_samples, *, maximize=True)[source]
Max-value Entropy Search acquisition at candidates with posterior
mean/std.Given samples
max_samplesof the global optimum valuey*, returns the per-candidate mutual informationI(y; y*) = (1/M) sum_m [ gamma_m phi(gamma_m)/(2 Phi(gamma_m)) - log Phi(gamma_m) ]withgamma_m = (y*_m - mu)/sd(maximization; for minimization the sense is flipped by the caller). Higher is better – it favors uncertain candidates near the believed optimum.
- propose_mes(x, y, bounds, *, n_candidates=512, max_samples=64, maximize=False, seed=None, gp=None, fit_kwargs=None)[source]
Propose the next point by Max-value Entropy Search.
Fits the GP to
(x, y), samplesmax_samplesoptimum values viasample_max_values(), scores Latin-hypercube candidates bymax_value_entropy_search(), and returns the maximizer as a(d,)array.maximizeselects the optimization sense (default minimize, matching the rest of the BO layer).