mixle.doe.entropy module

Information-theoretic Bayesian optimization: Max-value Entropy Search (MES).

The improvement-based acquisitions (EI/PI/UCB) score a candidate by how much it might beat the current best. MES (Wang & Jegelka 2017) instead scores a candidate by how much evaluating it would reduce the entropy of the global optimum value y* = max f – the mutual information I(y; y* | x). It is often more sample-efficient and low-overhead: with a GP, I(y; y*|x) has a closed form per sampled y* (a truncated-Gaussian entropy), and plausible y* are drawn by fitting a Gumbel to the distribution of the maximum over a candidate set.

max_value_entropy_search is pure NumPy (given posterior moments and y* samples); the propose_mes() driver fits the torch GP surrogate, so it needs PyTorch.

sample_max_values(mean, std, n_samples=64, *, seed=0)[source]

Sample plausible global-max values y* via the Gumbel approximation (Wang & Jegelka 2017).

The CDF of the maximum over the candidate cloud is P(max <= y) = prod_i Phi((y - mu_i)/sd_i); a Gumbel is fit to its 25/50/75 percentiles (found by bisection) and sampled. Returns an (n_samples,) array of y* draws (never below the best posterior mean).

Raises:

ValueError – if mean/std are empty, mismatched in shape, non-finite, or std is negative; or if n_samples is not a positive integer.

Parameters:
Return type:

ndarray

max_value_entropy_search(mean, std, max_samples, *, maximize=True)[source]

Max-value Entropy Search acquisition at candidates with posterior mean / std.

Given samples max_samples of the global optimum value y*, returns the per-candidate mutual information I(y; y*) = (1/M) sum_m [ gamma_m phi(gamma_m)/(2 Phi(gamma_m)) - log Phi(gamma_m) ] with gamma_m = (y*_m - mu)/sd (maximization; for minimization the sense is flipped by the caller). Higher is better – it favors uncertain candidates near the believed optimum. A candidate with sd == 0 is deterministic and scores exactly 0: its observation is a constant, which carries no mutual information with y* (MXR-080-1901).

Raises:

ValueError – if mean/std are empty, mismatched in shape, non-finite, or std is negative; if max_samples is empty or non-finite; or if the resulting per-candidate information is non-finite (e.g. an extreme-but-individually-finite mean/std/ max_samples combination can overflow gamma = (y* - mu)/sd to +/-inf, and inf * 0 in the gamma * pdf term then silently yields NaN).

Parameters:
Return type:

ndarray

propose_mes(x, y, bounds, *, n_candidates=512, max_samples=64, maximize=False, seed=None, gp=None, fit_kwargs=None)[source]

Propose the next point by Max-value Entropy Search.

Fits the GP to (x, y), samples max_samples optimum values via sample_max_values(), scores Latin-hypercube candidates by max_value_entropy_search(), and returns the maximizer as a (d,) array. maximize selects the optimization sense (default minimize, matching the rest of the BO layer).

Parameters:
Return type:

ndarray