AIMech AI

Predictive entropy search for multi-objective Bayesian optimization

Algorithms AIMech:PredictiveEntropySearchForMultiObjectiveOptimization DRAFT
A Bayesian multi-objective optimization algorithm that selects evaluations using an approximate information gain about the Pareto set, computed from sampled finite Pareto-set approximations and objective-factorized Gaussian expectation-propagation approximations to conditional predictions.
Identity Notes & Concept Scope This is the original PESMO procedure for information about a Pareto set under the stated independent objective models, finite sampled Pareto-set representation and EP approximations. Scalar PES targets one optimizer location; PESC adds feasibility to a scalar objective. Neither defines the multi-objective dominance factors or approximate objective-wise information allocation of PESMO. The proposed PESMOC, PPESMOC and PPES records preserve their different constrained or joint-batch targets.

Claims & Grounded Evidence

6 Claims
Mechanism #pareto-set-information-target
PESMO targets information about the Pareto set in input space for several objectives, using minimization conventions. A point dominates another when it is no worse in every objective and strictly better in at least one; the Pareto set contains the nondominated points. Assume a well-defined posterior over nonempty Pareto sets and finite mutual information. At fixed surrogate hyperparameters, the ideal acquisition is I(X_star; y(x) | D), equivalently the predictive entropy of the objective observations minus their expected conditional entropy given the Pareto set. This is a target for approximation; it is not information about only a scalar optimum value or a chosen scalarization.
Mechanism #independent-objective-surrogates
The model assigns independent Gaussian-process priors to the objectives and independent zero-mean Gaussian observation noise. At fixed independent model hyperparameters, the ordinary predictive distribution at a candidate factorizes across objectives. The reported experiments average acquisition values over sampled surrogate hyperparameters; such conditional averaging does not by itself compute the entropy of a hyperparameter mixture. Conditioning on a shared Pareto set need not preserve objective independence.
Mechanism #approximate-pareto-set-samples
To approximate the expectation over Pareto sets, sample finite random-feature linear approximations to each objective posterior, solve the resulting multi-objective problem approximately, and retain a finite representative set of nondominated locations. The paper describes a grid search and NSGA-II for higher-dimensional problems. Its choices of ten draws and fifty retained locations are reported numerical settings, not exact posterior sampling or fixed mathematical requirements of the algorithm. Finite search and set compression can omit parts of the true Pareto set.
Mechanism #finite-domain-ep-projection
For each retained Pareto-set sample, replace the continuous-domain conditioning calculation by a finite set containing previous evaluation locations, the sampled Pareto locations and the candidate. Approximate each non-Gaussian comparison factor by a product of objective-specific bivariate Gaussian factors, fitted through expectation propagation. Reuse the factors independent of the candidate, while the paper updates candidate-dependent factors once. The Gaussian projection matches the retained within-objective moments and discards cross-objective dependence; it does not reproduce all moments of the joint tilted distribution or certify exact whole-domain Pareto-set conditioning.
Mechanism #noisy-gaussian-variance-acquisition
Within the factorized Gaussian approximation, the contribution of objective k is one half of the difference between log(v_k(x)+sigma_k_squared) and the sample average of log(v_k_conditional(x; X_star_sample)+sigma_k_squared). Here v denotes latent predictive variance and sigma_k_squared is the observation-noise variance; the entire log-variance difference is multiplied by one half. Summing these contributions yields the coupled acquisition. All logarithm arguments must be positive. This additive score follows the Gaussian factorization and finite sample approximation, and is not generally an exact additive decomposition of joint Pareto-set mutual information.
Mechanism #coupled-and-decoupled-selection
In the coupled setting, maximize the sum of the objective contributions to choose a location for evaluating all objectives. If objectives can be evaluated separately, maximize each objective contribution over location and select the objective/location pair with the largest resulting score. The same sampled Pareto sets link the objective-specific approximations. The displayed decoupled rule compares information scores and does not include a cost divisor; unequal evaluation costs do not turn it into a guaranteed optimal scheduler. Coupled and decoupled use are modes of this record, not separate concept identities.

⚠️ Curated Limitations & Negative Results

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes
AIO Target Class Mapping Relation Mapping Rationale
AIO:ActiveLearning
Active Learning
related_match PESMO adaptively chooses informative objective evaluations. This resembles the pinned task of querying an information source, but multi-objective optimization and Pareto-set inference are not identical to labeling data for predictive learning. The mapping is a qualified curator relation, not an exact match or an upstream subclass assertion.

Evidence Sources & Primary Literature

3 Sources
Source ID Publication / Document Type / Access Reference & Checksum
hernandez2016-pesmo
Daniel Hernández-Lobato, José Miguel Hernández-Lobato, Amar Shah, Ryan P. Adams
primary paper
full_text
sha256:e85486acb4a5…
hernandez2016-pesmo-supplement
Daniel Hernández-Lobato, José Miguel Hernández-Lobato, Amar Shah, Ryan P. Adams
primary paper
full_text
sha256:2932a6f431f8…
pesmo-implementation
Daniel Hernández-Lobato and HIPS/Spearmint contributors
official documentation
documentation
sha256:613bec578f01…

Curation Provenance

Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-15T04:14:51Z
Updated: 2026-10-15T04:14:51Z
Source File: data/curated/PredictiveEntropySearchForMultiObjectiveOptimization.yaml