Separable Natural Evolution Strategies
Algorithms AIMech:SeparableNaturalEvolutionStrategies DRAFT
The Gaussian population natural evolution strategy that samples independent normal coordinates, ranks their evaluated fitness, and adapts the search mean and positive coordinate standard deviations using rank-shaped standardized directions and componentwise exponential scale updates in a fixed coordinate system.
Identity Notes & Concept Scope
The original 2011 Gaussian population Algorithm1 uses independent-coordinate sampling and scalar exponential coordinate-scale updates. This diagonal Gaussian finite procedure is distinct from full-covariance xNES and the broader NES/IGO frameworks; diagonal sampling alone does not identify every independent-coordinate optimizer with SNES. Current bounded identity assessment covers730native and442pinnedAIO records and four ignored/hidden-inclusive checkout scans; all2900matches were reconciled. Existing frontier owner194 is retained in place, without a duplicate leaf. The premerge collection and actual creation freshness checks remain separately recorded.
Claims & Grounded Evidence
6 Claims
Mechanism
#diagonal-gaussian-search-family
Maintain a mean vector mu in d-dimensional real space and a vector of strictly positive standard deviations sigma. The Gaussian population version uses a product of independent normal coordinates, with covariance diag(sigma_i squared), in a chosen coordinate system. It adapts d means and d scales rather than a general covariance matrix. The diagonal restriction concerns the search distribution; the objective need not itself factor into independent coordinate functions.
Mechanism
#population-sampling-and-rank-shaping
In each generation, draw lambda independent standard normal vectors s_k, form z_k = mu + sigma * s_k componentwise, and evaluate the black-box fitness at each z_k. Sort the sample pairs by fitness and assign rank utilities u_k, with the source formulated for fitness maximization. The update uses search-distribution scores and fitness rankings, not derivatives of the objective. Rank utility weights replace raw fitness values; they are not generally an unbiased estimator of the raw expected-fitness gradient.
Mechanism
#standardized-mean-direction
Using the ranked standardized samples from the current generation, compute g_mu = sum_k u_k s_k. The mean update is mu_new = mu + eta_mu * sigma * g_mu, with coordinatewise multiplication by the generation’s original standard-deviation vector. This transports the standardized direction back to the search coordinates; it is not an unscaled step directly in the objective coordinates.
Mechanism
#componentwise-exponential-scale-update
Compute g_sigma = sum_k u_k (s_k squared - 1) coordinatewise and update sigma_new = sigma * exp((eta_sigma/2) * g_sigma), with coordinatewise products and exponentials. The mean and scale directions come from the same pre-update samples. This is an additive step in each log standard deviation, not an additive variance update or a dense matrix exponential. Strictly positive scales remain positive in exact real arithmetic when the exponent is finite.
Scope
#coordinate-dependent-linear-parameterization
Restricting the covariance to its diagonal gives linear parameter storage and linear work per sampled vector and per final coordinate update. The explicit population sampling and weighted-moment loops require O(lambda*d) arithmetic, with fitness sorting and objective evaluations additional. The method does not represent arbitrary rotated correlations: selecting a distinguished coordinate system sacrifices the general rotation invariance of an unrestricted Gaussian family.
Scope
#gaussian-population-variant-boundary
The displayed Gaussian population SNES is a concrete variant of the NES search-distribution framework. The paper separately describes full-covariance exponential NES, a multivariate Cauchy hill climber, and a (1+1)-SNES variant whose pseudocode is not shown. The broader product-distribution discussion does not make these procedures, or every independent-coordinate optimizer, identical to Gaussian population Algorithm 1.
⚠️ Curated Limitations & Negative Results
- This record denotes the original Gaussian population Algorithm 1. It is not the whole NES or information-geometric optimization framework, full-covariance xNES, full CMA-ES, the Cauchy hill climber or every separable search distribution. The source’s Lennard-Jones experiment is labeled (1+1)-SNES and is not evidence for universal performance of the population algorithm.
- Algorithm 1 leaves the precise utility curve, tie policy, initialization and stopping rule to the setup. Section 2 mentions normalized nonnegative and centered utility choices; neither is silently imposed as the only rule. Strictly increasing objective transformations preserve rankings with consistent tie treatment, but decreasing transformations, arbitrary tie changes and noisy evaluations do not support an unqualified identical-trajectory claim.
- For a positive Gaussian scale, the location and log-scale scores are s/sigma and s squared - 1; their population Fisher block is diag(1/sigma squared, 2), giving the one-half log-scale factor. This bounded analytical explanation requires the Gaussian family. Raw expected-fitness gradient identities additionally require suitable regularity and integrability. Finite sampled updates, separate learning rates and nonlinear reparameterizations do not inherit an unconditional objective-improvement, convergence or exact finite-step invariance guarantee.
- A rotated diagonal covariance is generally not diagonal, so favorable coordinate alignment matters. The O(d) description in Section 3 is per individual or final coordinate update, not a population-size-free bound on an entire generation. The default lambda in Section 5 depends on dimension; sorting, evaluations and retaining a batch of vectors add costs. No benchmark speedup, global optimum or universal high-dimensional advantage is asserted.
- Positive scales are preserved by finite exact-real exponentials, but finite-precision overflow, underflow and numerically degenerate scales require implementation-specific safeguards. The source does not supply a complete numerical-stability or stopping guarantee, and no author implementation or benchmark has been executed for this record.
- Section 4, PDF p. 4 / printed p. 848, broadly links heavy-tailed infinite variance to unavailable Fisher geometry. That implication is not used here: the unit-scale Cauchy location score 2x/(1+x squared) is bounded and nonzero on positive probability, so its Fisher information is finite and positive despite undefined variance. This scoped curator counterexample is not an author-issued erratum or a complete audit of the separate Cauchy algorithm.
- Evidence locators refer to the eight-page paginated 2011 proceedings copy. The author-hosted manuscript has different abstract/conclusion wording, reference numbering, an added complexity footnote and different pagination, although the visually compared Algorithm 1 agrees. The 2014 journal comparison and the full-covariance factor conventions in Section 2 are separate context, not alternative evidence for a repaired implementation. Author-code access returned HTTP 500; code behavior is not certified.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ExponentialFunction
Exponential Function
|
related_match | The mandatory componentwise update of each standard deviation uses the scalar exponential e^x, a defining component matching the pinned Exponential Function concept. This relation does not equate the optimizer with a scalar function, assert a subclass or neural-network layer, or interpret a dense matrix exponential as elementwise exponentiation. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
schaul2011
|
Tom Schaul, Tobias Glasmachers, Jürgen Schmidhuber
|
primary paper
full_text
|
sha256:7576759a0b08…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-15T04:48:29Z
Updated: 2026-10-15T04:48:29Z