BOHB
Algorithms AIMech:BOHB DRAFT
Aliases: Bayesian Optimization and Hyperband
A hyperparameter optimization algorithm combining Hyperband resource brackets with model-guided configuration proposals from performance-split multivariate kernel density estimates and a continuing random-sampling component.
Identity Notes & Concept Scope
BOHB denotes the KDE-based method of Falkner, Klein and Hutter. It differs from pure random-sampling Hyperband and the original tree-structured one-dimensional TPE estimator. It is not every combination of Bayesian optimization and early stopping, a Gaussian-process surrogate, a Bayesian network, or the entire HpBandSter software framework. Paper and pinned implementation conventions are distinguished. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 256 preceding native IDs/names/aliases. Full-checkout fixed-string searches included ignored and hidden files in both worktree and main checkout, returning 2 lines in each checkout. These were prior random-search candidate leads, not existing records for these identities. Nine exact AIO CLASS nodes with synonyms and outgoing edges and six neighboring native mechanisms were inspected. Absence is bounded to this snapshot and semantic neighborhood.
Claims & Grounded Evidence
2 Claims
Mechanism
#method
Keep observations separated by evaluation budget. For proposals, use the highest budget with enough data to build better- and worse-performing configuration densities. The paper uses multivariate KDEs, a loss-quantile split with minimum set sizes, and finite candidate sampling from an expanded better-performing density, selecting a candidate with favorable l(x)/g(x). Retain a positive fraction of random configurations, and fall back to random sampling when no model is available. Hyperband brackets and successive halving then allocate increasing resources to surviving candidates. Observations are shared across brackets, but the proposal density is not fitted by pooling all fidelity levels indiscriminately. The method can schedule multiple brackets and concurrent evaluations; the paper’s worker policy favors waiting smaller-budget jobs. Numerical model-building and sampling details differ in the pinned implementation, as recorded separately.
Mechanism
#pinned-implementation
The pinned scheduler uses geometrically spaced budgets b_max*eta^(−s) and computes its initial bracket count with int(floor((s_max+1)/(s+1))*eta^s), which differs from the paper’s ceiling convention. The proposal generator selects the largest available model budget, minimizes a numerically floored g/l ratio over finite samples, expands continuous-coordinate bandwidths, and samples randomly from ConfigSpace when needed. KDEs fit disjoint best/remainder slices and are created only when each actual slice contains more rows than dimensions. This implementation does not reproduce the paper’s small-sample overlapping-set rule. These are static source observations, not a runtime or compatibility result.
⚠️ Curated Limitations & Negative Results
- The paper’s Equation(2) reverses the conditioning relative to configuration densities fitted from better/worse subsets. Its Algorithm1 prints eta^s*b_max as the initial budget, exceeding b_max for s>0. The recorded density interpretation and bounded geometric budgets follow the method prose, original Hyperband and inspected code; these are curator qualifications, not an author-issued erratum.
- Paper Equation(3) permits overlapping fit sets with few observations; the pinned code instead takes disjoint sorted slices and requires both actual slices to have more observations than dimensions. Its bandwidth widening applies to continuous proposal coordinates. Paper defaults and statements are not silently substituted for that implementation.
- Low-budget rankings can mislead search. Random exploration, density estimation and finite candidate sets do not give an unconditional finite-budget optimum, universal speedup or independent generalization guarantee. No author implementation was executed.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:TrainingStrategy
TrainingStrategy
|
related_match | This procedure organizes hyperparameter selection or allocation of model-training resources. The pinned AIO target describes training strategies as preprocessing; the relation connects these scopes without asserting that this search algorithm is preprocessing, an identical concept, or a subclass. |
Evidence Sources & Primary Literature
4 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Stefan Falkner, Aaron Klein, Frank Hutter
|
primary paper
full_text
|
sha256:774cc0ce9588…
|
schedule-code
|
AutoML / HpBandSter contributors
|
official documentation
documentation
|
sha256:23e134540679…
|
density-code
|
AutoML / HpBandSter contributors
|
official documentation
documentation
|
sha256:fc00e64eec04…
|
hyperband-source
|
Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, Ameet Talwalkar
|
primary paper
full_text
|
sha256:09c14cb28ec2…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T11:28:05Z
Updated: 2026-10-11T11:32:06Z
Source File:
data/curated/BOHB.yaml