Home / Curated / Algorithms / Max-value Entropy Search

Max-value Entropy Search

Algorithms AIMech:MaxValueEntropySearch DRAFT

A Bayesian global optimization algorithm that chooses evaluations using an approximate information acquisition for the latent maximum function value, averaging local truncated-Gaussian entropy differences over approximate maximum-value samples.

Identity Notes & Concept Scope The original MES Equation 6 acquisition targets a finite attained latent maximum value with local truncated-latent entropy, including MES-G and MES-R as sampling options. ES/PES target optimizer location; OPES shares the ideal value-information target but uses observed-latent EP constraints and noisy Gaussian moment-matched entropy. The common target does not establish operational identity, universal acquisition differences or superior utility. No acronym alias is adopted. Current full normalized, semantic and ignored-inclusive evidence is assessed before creation.

Claims & Grounded Evidence

6 Claims

Mechanism #maximum-value-information-target

On a compact domain, assuming sample objectives have a finite attained maximum, the method uses a Gaussian-process posterior for an objective observed through independent Gaussian noise. Its information target is the scalar latent maximum M=max_x f(x), rather than the location of a maximizer targeted by Entropy Search and Predictive Entropy Search. At fixed data, candidate x and hyperparameters, the ideal acquisition I(M;Y|D,x) can be written as H(Y|D,x)-E_M[H(Y|D,x,M)] when those entropy terms are well-defined. The maximum value remains defined when multiple locations attain it. Changing the information target does not establish that it is more useful on every problem or that it equals optimizer-location information.

supports Source: Max-value Entropy Search for Efficient Bayesian Optimization · Locator: Sections 2–3, PDF p. 2, Equations 1–5; original finite value/location information examples in research/discovery/mes-boundaries.py

Mechanism #local-latent-truncation-acquisition

Let mu_t(x) and s_t(x)>0 be the posterior mean and standard deviation of the latent function. For each approximate maximum sample m, set gamma=(m-mu_t(x))/s_t(x). Equation 6 averages gamma*phi(gamma)/(2*Phi(gamma))-log(Phi(gamma)) over these samples, where phi and Phi are the standard-normal density and cumulative distribution. This expression subtracts the entropy of a Gaussian truncated only by f(x)<=m from the ordinary latent Gaussian entropy. Conditioning on the actual global maximum is stronger than this necessary local inequality. Moreover, noisy Y=f(x)+epsilon may exceed M and has predictive variance s_t(x)^2 plus the observation-noise variance. The source uses latent variance and local truncation here, so this is an acquisition approximation, not exact noisy maximum-value information even with exact maximum samples.

supports Source: Max-value Entropy Search for Efficient Bayesian Optimization · Locator: Section 3, PDF pp. 2–3, Equations 4–6 and the paragraph following Equation 6; original conditional-Gaussian and noise examples in research/discovery/mes-boundaries.py

Mechanism #independent-grid-and-gumbel-maximum-sampling

The MES-G option first represents the domain by a finite set and approximates its posterior function values as independent Gaussians. Its approximate maximum CDF is the product of the marginal Gaussian CDFs. The source fits a Gumbel distribution by matching the 0.25 and 0.75 quantiles, then samples through its inverse CDF. These are distinct discretization, independence and distribution-fit approximations. The supplement’s Slepian comparison requires nonnegative posterior covariances on the finite set; nonnegative prior covariances alone do not suffice. Even when the independent maximum stochastically dominates under that comparison, the fitted Gumbel need not preserve the ordering. A stochastic distribution bound is also different from a pathwise bound for every sampled value.

supports Source: Max-value Entropy Search for Efficient Bayesian Optimization · Locator: Section 3.1 and Algorithm 1, PDF p. 3; original covariance and percentile-fit boundaries in research/discovery/mes-boundaries.py
supports Source: Max-value Entropy Search for Efficient Bayesian Optimization: Supplementary Material · Locator: Section 2, supplement PDF pp. 1–2, Lemma 2.1 and the Gumbel approximation discussion

Mechanism #finite-feature-posterior-function-sampling

The MES-R option approximates a suitable stationary Gaussian-process covariance with finite random Fourier features, samples Gaussian posterior feature weights, and numerically maximizes each resulting function to obtain an approximate maximum value. A stationary covariance has spectral mass k(0); drawing frequencies from a probability distribution requires normalization and the corresponding kernel-amplitude factor in the features. The displayed sqrt(2/D) cosine feature scaling uses unit amplitude. For fixed features the posterior belongs to the finite feature model. Finite features and numerical maximization do not guarantee exact maximum draws from the original Gaussian process.

supports Source: Max-value Entropy Search for Efficient Bayesian Optimization · Locator: Section 3.2, PDF p. 4; spectral-amplitude boundary in research/discovery/mes-boundaries.py

Mechanism #sequential-evaluations-and-model-adaptation

Each iteration numerically maximizes the approximate acquisition, evaluates the noisy objective at that location, appends the observation and recomputes the posterior acquisition. The paper also discusses maximum-likelihood hyperparameter fitting or aggregation over sampled hyperparameter settings and associated maximum samples. With fixed positive sample counts, omitting a common normalization does not change the acquisition argmax, although its numerical value is then a sum rather than an average. An average of fixed-hyperparameter information approximations is not generally exact information under the fully marginalized model. The displayed regression formulas use the zero-mean convention adopted in the experiments; a general prior mean requires consistent centering.

supports Source: Max-value Entropy Search for Efficient Bayesian Optimization · Locator: Algorithm 1, PDF p. 3; Sections 2.1, 3.5 and 5, PDF pp. 2, 4–5

Scope #single-sample-ranking-and-guarantee-scope

For one shared maximum sample and positive posterior standard deviations, the scalar Equation 6 score decreases with the standardized gap. Its maximizer therefore minimizes that gap, giving the source’s limited PI and EST relation with matching parameters. The displayed UCB coefficient min_x gamma can be negative, so equivalence to a standard nonnegative UCB exploration coefficient requires an additional domain restriction. There is no general such equivalence for multiple distinct maximum samples. The stated regret bound concerns a specific single-sample regime and maximum-sampling distribution. It does not automatically cover finite Gumbel or feature approximations, arbitrary multi-sample scores, learned hyperparameters or the additive extension.

supports Source: Max-value Entropy Search for Efficient Bayesian Optimization · Locator: Sections 3.3–3.4, PDF p. 4, Lemma 3.1 and Theorem 3.2; the finite score grid in research/discovery/mes-boundaries.py is illustrative and is not a proof of global monotonicity
supports Source: Max-value Entropy Search for Efficient Bayesian Optimization: Supplementary Material · Locator: Section 3, supplement PDF pp. 2–3, Theorem 3.1, its proof and the explicit absence of an additive regret bound

⚠️ Curated Limitations & Negative Results

This record covers the original Equation 6 acquisition and its MES-G/MES-R sampling options. It does not identify every maximum-value information method, or add-MES, constrained PES and parallel PES, with this algorithm. The paper’s PDF p. 2 credits output-space predictive entropy search with the same ideal information objective while distinguishing its operational approximation; that earlier method is not assigned as an exact alias here.
The local-truncation and observation-noise qualifications are part of the mechanism’s scope. The printed use of p(y|D,x) with latent variance and y<m is not silently repaired into a different noisy acquisition. Degenerate posterior variances, extreme Gaussian tails, finite-sample ties and numerical maximization require explicit implementation choices.
A finite-domain approximation is not automatically an upper bound on a continuous-domain maximum. The source’s covering argument uses an epsilon-cover and a valid Lipschitz constant to add epsilon times that constant. Neither such a bound nor posterior nonnegative covariance is guaranteed merely by choosing an arbitrary grid or a familiar covariance kernel. Gumbel inversion requires 0<r<1 and a positive scale for finite real outputs; degenerate fitted quantiles need handling.
The general regret theorem was not independently established by this record’s curator calculations. Integer sampling-block lengths, confidence terms at selected iteration times versus the final displayed index, and extensions to adaptive maximum distributions require a separate proof audit. No repaired theorem, universal regret rate or finite-budget global-optimum certificate is asserted.
The additive extension is outside this identity. Independent component priors generally become dependent after summed observations; separate component acquisitions are not automatically the joint information of the actual summed noisy evaluation. This contextual boundary does not establish that the additive heuristic lacks practical utility.
The original curator calculator evaluates 201 scalar scores and seven constructed boundary categories. Rational probability and covariance arithmetic is exact; Gaussian/Gumbel functions and entropy use floating arithmetic with explicit absolute tolerances. Its corrected CDF crossing witness and preserved initial failed assertion are research provenance, not a source-code repair. No Gaussian-process fitting, feature sampling, acquisition optimization, author implementation or benchmark was executed. These checks establish no general accuracy or runtime advantage over ES or PES.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:ActiveLearning` Active Learning	related_match	The pinned class describes interactive querying of an information source for outputs. MES chooses expensive objective evaluations for information about a latent maximum. This relation records a query-learning association, without identifying the algorithm with annotation or pool-based classification, asserting subclass membership, or inheriting a neural architecture.

Evidence Sources & Primary Literature

2 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`wang2017`	Max-value Entropy Search for Efficient Bayesian Optimization Zi Wang, Stefanie Jegelka	primary paper full_text	sha256:20e712e00165…
`wang2017-supplement`	Max-value Entropy Search for Efficient Bayesian Optimization: Supplementary Material Zi Wang, Stefanie Jegelka	primary paper full_text	sha256:3681a6cd7e60…

Curation Provenance

Curator: Codex

Method: ai_assisted

Model: GPT-6

Created: 2026-10-15T01:49:16Z

Updated: 2026-10-15T01:49:16Z

Source File: data/curated/MaxValueEntropySearch.yaml