Efficient Global Optimization
Algorithms AIMech:EfficientGlobalOptimization DRAFT
The original sequential optimization algorithm of Jones, Schonlau and Welch for expensive deterministic scalar functions on bounded continuous boxes, combining a space-filling initial design, fitted and diagnostically assessed DACE response models, expected-improvement acquisition search, objective evaluation and model refitting.
Identity Notes & Concept Scope
EGO is the original sequential procedure for expensive deterministic objectives, combining a design, Kriging model fitting, expected-improvement maximization and stopping. It is not just the EI functional. Existing SMAC uses a random-forest configuration model and instance intensification; ES/PES use optimizer-information acquisitions. Efficient Global Optimization in later paper titles does not establish identity with the original Jones procedure.
Claims & Grounded Evidence
6 Claims
Mechanism
#space-filling-initial-design
The original EGO procedure begins with evaluations of an expensive deterministic scalar objective on a bounded rectangular search region using a space-filling Latin-hypercube design. The described design search aims at nearly uniform one- and two-dimensional projections. These initial evaluations supply the data for the first DACE fit; the suggested initial sample size is a heuristic, not a defining fixed count or convergence condition.
Mechanism
#plug-in-dace-surrogate
Fit a constant-mean DACE response model with a product-exponential distance correlation, exp(-sum_h theta_h |x_h - x_prime_h|^p_h). Maximum likelihood estimates its parameters. The predictor uses the correlation matrix R and candidate correlation vector r; its prediction-error expression includes the constant-mean estimation term. The ordinary-inverse formulas require nonsingular R. Estimated correlation and process-variance parameters are plugged in rather than fully integrated out.
Mechanism
#diagnostics-and-response-scale
Inspect cross-validated predictions and standardized residuals before the sequential search. In the stated practical leave-one-out procedure, covariance parameters fitted to the full sample are retained. If diagnostics are unsatisfactory, refit after a suitable response transformation, such as the displayed log or reciprocal transformations on their applicable domains. Continue on the chosen response scale only if diagnostics are satisfactory; the source would abandon the method if no satisfactory transformation were found.
Mechanism
#expected-improvement-box-search
Choose an evaluation location by maximizing Gaussian expected improvement below the best observed value on the modeled response scale. The full acquisition search uses rectangular branch and bound: a lower bound on predictive mean and an upper bound on predictive standard deviation provide an EI upper bound. The paper derives surrogate bounds through relaxations of the correlation constraints. Its implemented strong predictor-bound construction and reported examples use p_h = 2; this does not establish a ready implementation for every correlation exponent.
Mechanism
#evaluate-refit-and-stop
Evaluate the objective at the selected location, augment the data and re-estimate the DACE parameters by maximum likelihood before the next acquisition. A tolerance on maximum expected improvement controls stopping. The reported relative-change rule and its log-scale approximation are heuristics: one-step expected gain under a fitted surrogate is not a certificate of the true error or the total gain available from all further evaluations.
Scope
#limited-memory-acquisition-variant
The reported six-dimensional Hartman run does not use the full acquisition search to convergence. It retains only the best 50 branch-and-bound nodes, caps that search at 500 iterations, then refines the result by local search. The source explicitly recognizes that this limited-memory variant can miss the true EI maximum. Its behavior must be distinguished from the full bounding procedure; the stated numerical limits are experimental settings, not a definition of every EGO run.
⚠️ Curated Limitations & Negative Results
- This identity denotes the original deterministic scalar procedure, not every Bayesian optimizer or every method using EI. No noisy-objective, nonlinear-constraint, derivative-enabled, multifidelity, batch or multiobjective extension is absorbed. Section 6, PDF p.33 / printed p.487 discusses such extensions separately and leaves noisy-function EGO unresolved. EI itself is a distinct acquisition rule; TPE and SMAC use different model and search machinery.
- The source does not prove global convergence. Conjecture 1 assumes correlation parameters theta_h bounded away from zero, a continuous objective on a finite box and an indefinitely continued search; it remains explicitly unproved (Section 4.2, PDF p.22 / printed p.476). Acquisition bounds, empirical examples and a model-based stopping threshold do not establish finite-budget optimality or universal performance.
- Nonlinear response transformations can change improvement magnitudes and query rankings. The literal 1% stopping rule needs a sign and scale convention when the incumbent is zero or negative; no absolute-value replacement or numerical floor is silently supplied. The log-scale tolerance is described as an approximate relative change. Plug-in uncertainty and one-step gain also limit interpretation of that stopping rule (Section 4.2, PDF pp.19–21 / printed pp.473–475).
- Very smooth responses and closely spaced inputs can make R nearly singular. The authors report singular-value decomposition with small singular values set to zero, but the record does not certify a particular truncation threshold, ordinary-inverse validity after truncation, or preservation of all interpolation and bounding properties (Section 6, PDF p.32 / printed p.486).
- Appendix 1, PDF p.36 / printed p.490, changes the negative cross term of the displayed quadratic into a positive term in its derivative. Differentiation requires a minus sign to recover the stated predictor Eq.7. The inspected discrepancy and separate exact sign witness are retained as source algebra context, not an asserted author erratum or a repaired EGO implementation.
- For fixed real symmetric positive-definite R, fixed nonnegative process variance and exact arithmetic, the Eq.16 Hessian is positive semidefinite with a zero eigenvalue, so the Eq.17 alpha correction is zero. The separate curator derivation uses A = R_inverse - (R_inverse 1)(R_inverse 1)^T/(1^T R_inverse 1). This qualifies the source discussion of a possible negative eigenvalue. Nonlinear constraints linking r and x remain nonconvex; neither EI concavity, convexity of the full search problem nor a numerical failure in the authors’ implementation follows (Section 4.3, PDF pp.22–24 / printed pp.476–478; Appendix 2, PDF p.36 / printed p.490).
- Primary method evidence is the 38-page journal-layout mirror of the 1998 paper. The distinct 43-page author manuscript was cover-inspected only; exact equivalence to that manuscript or an inaccessible publisher download was not established. The cited Mockus et al. (1978) chapter was not read, so no earliest-invention claim is made.
- The supporting original calculations and separate root reproductions check bounded EI, interpolation and source-sign examples. Exact rational checks are distinct from floating Gaussian, quadrature and transformation calculations. The separate Hessian assessment is an analytic derivation under stated assumptions. None executes author code, implements the complete EGO search, replicates its benchmarks or supplies a formal erratum or convergence proof.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ActiveLearning
Active Learning
|
related_match | Original EGO adaptively queries an objective and updates a data-derived response model, relating it to the pinned concept of querying an information source for outputs. Deterministic black-box optimization is not identical to a data-labeling task, and this qualified association imposes neither an annotation requirement nor an upstream subclass or exact-match assertion. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
jones1998
|
Donald R. Jones, Matthias Schonlau, William J. Welch
|
primary paper
full_text
|
ID: doi:10.1023/A:1008306431147
sha256:356260d0303a…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-15T04:14:51Z
Updated: 2026-10-15T04:14:51Z
Source File:
data/curated/EfficientGlobalOptimization.yaml