Predictive entropy search for multi-objective Bayesian optimization with constraints
Algorithms AIMech:PredictiveEntropySearchForMultiObjectiveOptimizationWithConstraints DRAFT
A Bayesian optimization algorithm for multiple objectives and unknown constraints that selects evaluations using approximate information gain about the feasible Pareto set, estimated through sampled feasible Pareto representatives and task-factorized Gaussian expectation propagation.
Identity Notes & Concept Scope
PESMOC targets a feasible Pareto set through objective dominance and constraint-feasibility factors. This differs from unconstrained PESMO and from scalar PESC even though the methods share approximate predictive-entropy reasoning. The single-query mechanism is distinct from the joint-batch covariance construction of PPESMOC. Version-specific paper, appendix and implementation limitations remain explicit.
Claims & Grounded Evidence
6 Claims
Mechanism
#feasible-pareto-set-information
PESMOC selects evaluations to learn the Pareto set in input space for multiple objectives under unknown inequality constraints. In the source minimization convention, a point is feasible when every constraint c_j(x) is nonnegative; a feasible point dominates another when it is no worse in all objectives and strictly better in at least one. Assume a well-defined posterior over attained, nonempty feasible Pareto sets and finite mutual information, with ties handled explicitly. At fixed surrogate hyperparameters the ideal acquisition is I(X_star; y(x) | D), equivalently predictive entropy minus its expected value conditional on the feasible Pareto set. The implemented score approximates this target; it is not entropy of only the Pareto front or a scalar optimum value.
Mechanism
#objective-and-constraint-surrogates
The method models each objective and each constraint with an independent Gaussian process and Gaussian observation noise. Conditional on fixed independent model hyperparameters, the ordinary predictive distribution factors over the objective and constraint tasks. Conditioning all tasks on a shared feasible Pareto set generally introduces dependence, which the later Gaussian projection does not retain across tasks. Averaging conditional acquisition values over sampled hyperparameters is also an approximation, rather than the entropy of the resulting predictive mixture.
Mechanism
#sampled-feasible-pareto-representatives
Approximate the expectation over Pareto sets by sampling random-feature approximations to both objective and constraint posteriors. For each draw, search candidate locations, discard locations violating a sampled constraint, and retain a finite representation of the nondominated feasible locations. The paper uses a grid and discusses evolutionary search for higher dimensions. Finite features, finite search and compression of the Pareto set make these approximate samples. The paper does not specify a general acquisition policy for posterior draws with no feasible solution; relaxing a reported recommendation threshold is a separate operation.
Mechanism
#finite-feasibility-and-comparison-ep
For a retained feasible Pareto-set sample, approximate the conditioning calculation on the union of sampled Pareto locations, previous evaluation locations and the candidate. Feasibility factors constrain sampled Pareto locations; comparison factors couple objective comparisons with feasibility of competing locations. Expectation propagation replaces these non-Gaussian factors by task-specific Gaussian factors: univariate constraint factors and bivariate within-objective factors for location pairs. Reuse the factors independent of the candidate and update candidate-dependent factors once in the stated approximation. This projection discards cross-task dependence and is not exact whole-domain feasible Pareto-set conditioning.
Mechanism
#noisy-taskwise-variance-score
Within the factorized Gaussian approximation, each task contributes one half of its ordinary log predictive observation variance minus the sample average of its conditional log predictive observation variance. Add that task’s observation-noise variance to the latent variance in both terms and require positive logarithm arguments. Summing over objective and constraint tasks gives the coupled score. The common factor one half follows Gaussian entropy; the main acquisition display omits it, which preserves uncosted rankings but not numerical information units. Task additivity belongs to the approximation and is not generally an exact decomposition of joint feasible-Pareto mutual information.
Mechanism
#coupled-and-decoupled-task-selection
In coupled use, maximize the sum of task contributions to choose one location for evaluating all objectives and constraints. When tasks can be evaluated separately, maximize their individual approximate contributions and compare the maxima to select a task and location. Coupled and decoupled selection are modes of PESMOC. The displayed comparison contains no evaluation-cost divisor and does not guarantee an optimal scheduler or lower wall-clock cost when task costs differ. Multi-location parallel batch selection requires a separate acquisition construction.
⚠️ Curated Limitations & Negative Results
- Existence and measurability of the feasible Pareto target require assumptions beyond boundedness of the input domain. Distinguish no observed feasible incumbent, no feasible grid point in an approximate search, and a posterior function draw whose feasible region is empty. The paper’s recommendation relaxation on PDFp.33 is not a specified posterior-empty-draw acquisition policy. Per-constraint probability thresholds also do not imply the same simultaneous feasibility probability.
- The 2019 institutional author manuscript is dated May3,2019 and has a visible gap from printedp.8 to printedp.10; the separate official supplement is dated May13,2019. The six-page September2016 arXivv2 is an earlier version. No claim of reading the inaccessible final publisher layout or the missing page is made; relevant accessible methods and the earlier independent-prior assumptions are cited explicitly.
- Main Eq8 on PDFp.15 includes self comparisons with Theta(0)=1, making the literal product zero for any nonempty feasible supplied set; removing self comparisons still requires tie handling and does not make nondominated-membership factors enforce equality to the complete Pareto set. Eq10 has the minimization comparison orientation stated in the main text. The prose saying that every supplied Pareto point dominates each other point is too strong: being better in one objective does not imply dominance.
- Main PDFp.17 Eq13 and supplement PDFp.9 Eq15 place constraint factors at the sampled Pareto location although their exact comparison factors depend on competitor feasibility. Main Eq7 lacks a half coefficient on the constraint log-variance terms; Eq15 lacks a common half coefficient; Eqs17–18 bind task indices instead of the Pareto-sample index in the expectation. These displays are not certified literal executable formulas.
- The supplement contains further algebraic and notation defects, including the feasibility log-normalizer curvature in PDFp.12 Eq22, the observation-first difference orientation in PDFp.15 Eq30, exponential grouping in PDFpp.16–17 Eqs40/43, singular difference-Hessian inverses in PDFp.18 Eqs45–48, and covariance/precision mixing in PDFp.22 Eq61. The same section first describes a precision diagonal as a variance and then gives an inverse diagonal on p.23. Its damping prose on p.24 ends with non-positive definiteness as the stopping state. The record preserves the supported EP mechanism without claiming a complete repaired implementation or guaranteed convergence.
- Selective inspection of a separately pinned May2020 author implementation provides context, not an official erratum: it uses competitor constraint values, a minimization sign, zeroed self sites, half-log entropy differences and inverse precision construction. It also caps retained observations, changes candidate covariance, damps one candidate pass, and clips sampled task scores before averaging. The empty-sample and numerical safeguards are version-specific. No author code was executed, and no equivalence to every paper equation is asserted.
- Finite features, approximate search, finite Pareto representatives, EP factorization, limited candidate updates and hyperparameter averaging introduce approximation. The original bounded checks in research/discovery/pesmoc-boundaries.py establish selected finite and Gaussian counterexamples only; they are not a PESMOC implementation, empirical validation, efficiency claim or universal convergence proof.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ActiveLearning
Active Learning
|
related_match | PESMOC adaptively queries informative objective and constraint evaluations. This relates to the pinned task of querying an information source, but constrained multi-objective optimization is not identical to labeling data for predictive learning. The mapping is a qualified curator assessment, not an exact match or an asserted upstream subclass relation. |
Evidence Sources & Primary Literature
4 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
garrido2019-pesmoc
|
Eduardo C. Garrido-Merchan, Daniel Hernandez-Lobato
|
primary paper
full_text
|
sha256:61320cfbed37…
|
garrido2019-pesmoc-supplement
|
Eduardo C. Garrido-Merchan, Daniel Hernandez-Lobato
|
primary paper
full_text
|
sha256:2d19afc6f7d5…
|
garrido2016-pesmoc
|
Eduardo C. Garrido-Merchan, Daniel Hernandez-Lobato
|
primary paper
full_text
|
sha256:7c1637319e8d…
|
pesmoc-implementation
|
Eduardo Garrido-Merchan and Spearmint contributors
|
official documentation
documentation
|
sha256:6ac60919d376…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-15T04:14:51Z
Updated: 2026-10-15T04:14:51Z