Predictive Entropy Search with Constraints
Algorithms AIMech:PredictiveEntropySearchWithConstraints DRAFT
A Bayesian optimization algorithm for an unknown objective and unknown inequality constraints that selects evaluations using an approximation to information about the constrained minimizer location, combining posterior optimizer samples, finite feasibility and optimality indicators, Gaussian expectation propagation and taskwise predictive observation entropies.
Identity Notes & Concept Scope
PESC is a constrained-minimizer-location information acquisition whose operational approximation uses finite feasibility/optimality indicators, task-factorized EP and noisy candidate moment matching. These distinguish it from unconstrained PES and maximum-value MES/OPES; merely setting the constraint count to zero does not establish operational PES equivalence. Taskwise acquisition is not the separate PESC-F scheduling policy. No acronym alias is adopted. Fresh complete canonical, semantic, frontier and ignored-inclusive assessments establish bounded distinctness before creation.
Claims & Grounded Evidence
6 Claims
Mechanism
#constrained-optimizer-information-target
The problem minimizes f(x) subject to c_k(x)>=0 for each constraint, with objective and constraints defined throughout the search domain. Assume posterior function draws have an attained feasible minimum and an almost surely unique minimizer or a specified measurable tie selection. At fixed data and model hyperparameters, the ideal acquisition targets I(X_star;Y|D,x), where X_star is that constrained minimizer location and Y contains the prospective objective and constraint observations. When the entropy terms are well-defined, this equals H(Y|D,x)-E_X_star[H(Y|D,x,X_star)]. The acquisition does not require an already observed feasible incumbent. That property does not specify how to handle a posterior function draw with no feasible point or no attained minimum.
Mechanism
#approximate-constrained-optimizer-sampling
The practical method models the objective and constraints with independent Gaussian processes and independent Gaussian observation noise. It approximates the expectation over X_star with a finite average: draw approximate posterior functions and numerically minimize the sampled objective under the sampled constraints. The described sampler uses a finite basis approximation for stationary covariance functions, so the generated optimizer samples are approximate. Sampling GP hyperparameters and averaging their acquisition scores is an additional approximation, not in general the information obtained by first marginalizing predictive distributions over hyperparameters. The source does not guarantee that its numerical searches find every global optimizer.
Mechanism
#finite-feasibility-and-optimality-indicators
For a proposed optimizer x_star, define Gamma(z)=product_k 1[c_k(z)>=0] and Psi(z)=Gamma(z)*1[f(z)>=f(x_star)]+1-Gamma(z). Gamma(x_star) requires feasibility and Psi(z) excludes a feasible point with strictly smaller objective value. The method replaces the domain-wide product of Psi factors with a finite collection at observed inputs and the candidate, with x_star also represented in the latent Gaussian variables. This finite restriction is not exact global optimality conditioning. Even on a finite domain the indicators express membership in the minimizer set, not a normalized optimizer density or a rule selecting one tied minimizer. Equality c_k(z)=0 is feasible under the source step convention.
Mechanism
#factorized-ep-and-candidate-moments
Expectation propagation replaces the finite non-Gaussian feasibility and optimality factors with Gaussian sites. The resulting approximation factorizes between the objective and constraint functions while retaining covariances between represented values of each function. The calculation separates the observed-input and optimizer-dependent approximation from the candidate calculation. At a candidate, it applies the remaining optimality factor and matches marginal moments to a product of Gaussians. Independent prior functions do not remain independent merely because the optimizer location is conditioned on; this factorization discards conditional dependencies. Independent observation noise is added to the latent marginal variances afterward.
Mechanism
#taskwise-noisy-log-variance-score
For positive predictive observation variances and M>0 optimizer samples, the Gaussian entropy approximation assigns function i the score a_i(x)=0.5*log(v_i(x)) -(1/(2*M))*sum_j log(v_i(x|x_star_j)), where the conditioned variances come from the approximation and include observation noise. The coupled score sums a_i over the objective and constraints; an evaluated subset can use the corresponding partial sum. This supports comparing objective and constraint queries. Additivity comes from the conditional factorization approximation, not exact mutual-information additivity. The 2015 Equations 8 and 16 omit the log-variance factor 1/2; the later 2016 Equations 9 and 11–14 include it. The common factor changes the information scale, though not the maximizer of that same score.
Scope
#boundary-optima-and-pes-distinction
PESC does not impose zero objective gradient or a positive-definite objective Hessian at every constrained optimizer: a feasible-region boundary optimum need not satisfy those unconstrained interior conditions. It instead uses feasibility and objective-comparison indicators. Removing all constraints gives an unconstrained PESC acquisition, but the 2016 report explicitly distinguishes this operational approximation from Predictive Entropy Search, including their derivative conditions and treatment of observed objective comparisons. Their shared optimizer-information target does not make them the same algorithm.
⚠️ Curated Limitations & Negative Results
- The source scope is the full nine-page 2015 paper and nine-page supplement, plus only pages 1–3, 12–22 and 41–49 of the 53-page 2016 report. PESC-F and general resource scheduling are outside this record.
- Feasible attainment and a unique optimizer or explicit tie selection are assumptions of the optimizer random variable. The inspected descriptions do not specify a complete policy for empty feasible posterior draws. No observed feasible incumbent is a different issue. The informal domain-wide indicator product is not a normalized density for an equality event at one continuous optimizer location.
- Finite basis sampling, numerical optimization, finite optimality constraints, Gaussian sites, task factorization, candidate moment matching and hyperparameter score averaging are separate approximations. They do not establish exact noisy information or universally additive task information.
- The printed appendices contain formula caveats. The 2015 supplement cavity means (Equations 18 and 28), propagation variance after Equation 34, feasibility site Equation 30, candidate variance after Equation 37 and damping Equation 38 fail the retained bounded Gaussian or endpoint checks. The 2016 report is not a wholesale repair: Equation 39 retains a reciprocal cavity-mean issue, Equation 54 gives the wrong derivative sign in the checked feasible-Gaussian case, and the propagation expression after Equation 56, candidate Equation 62 and damping Equation 65 retain corresponding defects. These are source caveats, not a claim to have implemented a corrected EP algorithm.
- The 2016 main Equation 10 describes its training covariance as noise-free while omitting training noise inside the inverse; Appendix Equations 28–29 include that noise. The record does not certify the literal main predictive formula or numerical behavior for singular matrices and degenerate variances.
- The 2015 Section 4.1 rejection-sampling comparison also approximates conditional marginals as independent Gaussians. Its ground-truth terminology does not establish exact mutual information or validate factorization universally. The noncore expected-improvement Equation 2 also has a sign mismatch with the stated minimization convention.
- The original retained calculator checks ten bounded categories using exact rational arithmetic and elementary floating-point functions. It does not run EP, fit GPs, optimize acquisitions, inspect author software, reproduce benchmarks or establish general performance or convergence.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ActiveLearning
Active Learning
|
related_match | The connection is choosing informative queries and obtaining their outputs. The pinned AIO term describes a labeling-oriented machine-learning task, whereas PESC chooses objective and constraint evaluations for optimization. This related mapping asserts neither exact equivalence nor a subclass, neural-network or layer relationship. |
Evidence Sources & Primary Literature
3 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
hernandez2015
|
José Miguel Hernández-Lobato, Michael A. Gelbart, Matthew W. Hoffman, Ryan P. Adams, Zoubin Ghahramani
|
primary paper
full_text
|
sha256:7ca4b6853ba7…
|
hernandez2015-supplement
|
Predictive Entropy Search for Bayesian Optimization with Unknown Constraints: Supplementary Material
José Miguel Hernández-Lobato, Michael A. Gelbart, Matthew W. Hoffman, Ryan P. Adams, Zoubin Ghahramani
|
primary paper
full_text
|
sha256:e16f3bcd49d4…
|
hernandez2016
|
José Miguel Hernández-Lobato, Michael A. Gelbart, Ryan P. Adams, Matthew W. Hoffman, Zoubin Ghahramani
|
primary paper
full_text
|
sha256:bf1df88f4242…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-15T02:42:19Z
Updated: 2026-10-15T02:42:19Z