Output-space Predictive Entropy Search
Algorithms AIMech:OutputSpacePredictiveEntropySearch DRAFT
A Bayesian global optimization algorithm that selects evaluations using an approximation to information about the latent maximum function value, combining Gaussian expectation-propagation constraints at observed inputs with candidate truncation, moment matching and predictive observation entropy.
Identity Notes & Concept Scope
The pinned author-hosted OPES report defines observed-latent upper constraints via EP, candidate truncation and moment matching, then approximate noisy predictive entropy for a finite attained maximum value. MES shares its ideal target but not this conditional-inference pathway and entropy approximation; PES targets a maximizer location. A one-threshold diagnostic distinguishes formulas without proving different decisions for all data. The 2015 catalog attribution and 2016 PDF build are separately retained. No acronym alias is adopted; current canonical, frontier and ignored-inclusive evidence is refreshed before creation.
Claims & Grounded Evidence
6 Claims
Mechanism
#maximum-value-information-target
For a model and domain with a finite attained latent maximum M=max_x f(x), the method targets information between M and a prospective observation Y at candidate x. At fixed data and model hyperparameters, its ideal acquisition is I(M;Y|D,x)=H(Y|D,x)-E_M[H(Y|D,x,M)] when those entropy terms are well-defined. This targets the maximum value rather than the maximizer location used by Predictive Entropy Search. Max-value Entropy Search later uses the same ideal value-information target but a different operational approximation; shared ideal information does not identify their evaluated acquisitions.
Mechanism
#observed-latent-ep-constraints
In the Gaussian-observation formulation, a zero-mean Gaussian-process model gives a Gaussian posterior over latent function values at previously queried inputs. For each sampled maximum value m, the method applies step factors requiring every one of these latent values to be at most m. Expectation propagation replaces the factors with Gaussian site approximations having diagonal site covariance, which are combined with the posterior. The finite collection of inequalities is a necessary condition for a global maximum m, not the exact event that the function maximum equals m. The source uses these constraints as an approximation to that conditioning.
Mechanism
#candidate-truncation-and-moment-matching
The approximate constrained latent Gaussian is propagated to a candidate using Gaussian-process prediction, giving mean mu_bar and variance v_bar. For v_bar>0, the method then truncates that scalar Gaussian above at m and moment-matches it to a Gaussian. With alpha=(m-mu_bar)/sqrt(v_bar) and r=phi(alpha)/Phi(alpha), the matched moments are mu_hat=mu_bar-sqrt(v_bar)*r and v_hat=v_bar*(1-r*(r+alpha)). Here phi and Phi denote the standard-normal density and cumulative distribution. These are moments of the selected local truncation, not exact global-maximum-conditioned moments. A nondegenerate moment-matched Gaussian itself retains positive probability above the truncation threshold.
Mechanism
#noisy-gaussian-entropy-acquisition
Let v(x|D) be the ordinary posterior latent variance, sigma^2 the independent Gaussian observation-noise variance, and m_1,...,m_K a positive number K of approximate maximum samples. Equation 7 computes 0.5*(log(v(x|D)+sigma^2) - (1/K)*sum_j log(v_hat(x|D,m_j)+sigma^2)). Observation noise is added after the latent moment matching. The score uses Gaussian observation entropies and remains an approximation to noisy maximum-value information. Its fixed-threshold expression can differ from the local truncated-latent entropy expression of Max-value Entropy Search even without observed-latent EP sites. A difference in one term does not imply different selected queries for every data set.
Mechanism
#maximum-sampling-and-hyperparameter-averaging
The finite average requires approximate draws of the latent maximum value. The report refers to a sampling strategy similar to Predictive Entropy Search rather than specifying a new exact maximum sampler. It also describes sampling Gaussian-process hyperparameters and using a maximum sample for each setting. These sampling and model approximations remain part of the acquisition. Averaging fixed-hyperparameter information approximations does not in general give exact information under a fully marginalized hyperparameter model. The abbreviated report does not establish exact maximum sampling or universal acquisition accuracy.
Scope
#bernoulli-observation-extension
The paper sketches an extension for binary observations with a logistic or probit link to a latent Gaussian process. Laplace or expectation-propagation inference provides Gaussian latent approximations, with maximum constraints incorporated into the candidate prediction. Integrating the link against the approximate latent distribution gives an observation success probability, whose Bernoulli entropy is used in the acquisition. This is observation entropy, not the differential entropy of the latent Gaussian. The source describes closed-form probit integration and approximate logistic integration. This extension sketch does not guarantee exact inference or the same treatment for every non-conjugate likelihood.
⚠️ Curated Limitations & Negative Results
- The source is the pinned five-page author-hosted report. The author catalog cites a December 2015 workshop, while the PDF has an embedded February 28, 2016 build timestamp. Its advertised longer derivation was not located in bounded primary-source searches.
- Finite upper constraints, Gaussian EP sites, candidate moment matching and maximum sampling are separate approximations. They do not establish exact conditioning on a global maximum or exact noisy mutual information.
- The displayed matrix formulas use inverse noise variance and matrix inverses. Literal zero noise, singular matrices, zero candidate variance and extreme inverse-Mills ratios need separate numerical or limiting treatment. The retained zero-noise scalar diagnostic only evaluates the entropy-stage limit, not those matrix formulas at zero noise.
- Output-value targeting can avoid a single coordinate-space entropy representation on a disjoint domain. The source statement that PES cannot apply to differently dimensional domains is not adopted as an impossibility result for mutual information under suitable probability measures.
- The final binary-entropy formula concerns the induced Bernoulli observation distribution. The source wording about entropy of the latent Gaussian approximation is not interpreted literally as equality of those two entropies.
- The preliminary experiments and unresolved Figure ?? reference do not support a general performance, convergence or efficiency claim here. The retained original arithmetic diagnostic does not implement EP, optimize an acquisition, fit a model, or reproduce author benchmarks.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ActiveLearning
Active Learning
|
related_match | The relationship is through choosing informative queries and obtaining their outputs. The pinned AIO term describes a labeling-oriented machine-learning task, whereas this algorithm selects objective evaluations for optimization. This is a related mapping, not exact equivalence or an asserted subclass, neural-network or layer relationship. |
Evidence Sources & Primary Literature
2 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
hoffman2015
|
Matthew W. Hoffman, Zoubin Ghahramani
|
primary paper
full_text
|
sha256:56241d9e8a49…
|
wang2017
|
Zi Wang, Stefanie Jegelka
|
primary paper
full_text
|
sha256:20e712e00165…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-15T01:49:16Z
Updated: 2026-10-15T01:49:16Z
Source File:
data/curated/OutputSpacePredictiveEntropySearch.yaml