Diverse beam search
Algorithms AIMech:DiverseBeamSearch DRAFT
Aliases: DBS
The inference procedure of Vijayakumar and colleagues that partitions a sequence beam budget into groups and updates each group using model log-probability scores augmented by a weighted dissimilarity to the already-updated earlier groups at the same decoding step.
Identity Notes & Concept Scope
This record denotes the group-based diversity-augmented method published at AAAI 2018, first submitted under a different title in 2016. It is distinct from Gumbel-based stochastic beam sampling, inter-sibling rank penalties, repeated complete beam runs with final-output diversity, and earlier algorithms using the phrase diversity beam search. The diversity objective is integrated into group beam updates; the procedure requires no retraining of the scoring model. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 328 preceding native IDs/names/aliases. Full-checkout fixed-string searches included ignored and hidden files in worktree and main, returning 49 and 49 lines respectively. All 2 unique hit contents were inspected: a saved stochastic-beam/Gumbel discovery lead and a search-result title about coalition-formation scatter search, not existing candidate records. Exact checkout counts and differences are retained in the ledger. Nine exact AIO CLASS nodes with scoped synonyms/comments and outgoing edges and four full native neighboring records were inspected. Absence is bounded to this snapshot and semantic neighborhood.
Claims & Grounded Evidence
3 Claims
Mechanism
#method
Divide a total beam budget B into G ordered groups, conventionally each of width B′=B/G when divisible; the paper also allows unequal allocations. At each decoding step, update the first group by ordinary beam search. For every later group in order, extend its retained prefixes by one token and score each candidate by its cumulative model log probability plus lambda times its dissimilarity to the current-step beams already selected in all earlier groups, with lambda nonnegative. For the additive family, dissimilarity to a group is the sum of pairwise sequence dissimilarities. Select the group’s top B′ distinct candidate extensions under that augmented score and proceed to the next group. Repeat through the chosen maximum length or termination condition, then combine group outputs and sort by model log probability as described in the source prose. The method is greedy across both time and group order; it encourages diversity between groups rather than imposing a joint exact sequence-set optimum.
Mechanism
#diversity-choice
The source’s simple Hamming-style option penalizes a candidate token according to how often earlier groups used that token at the same position. Other discussed choices include cumulative positional mismatch, n-gram overlap and embedding-based dissimilarity. These are configurable scoring choices, not interchangeable definitions or guaranteed measures of semantic diversity.
Application
#inference-context
The method operates during inference on sequence-model scores without requiring retraining, and is developed for image captioning, visual question generation and machine translation. Its empirical comparisons do not establish universal improvements.
⚠️ Curated Limitations & Negative Results
- The first group is an ordinary beam of width B′. Its baseline inclusion is not a guarantee of matching or beating an ordinary beam of total width B, nor a guarantee of global MAP or globally optimal diversity.
- Setting G=1 recovers an ordinary width-B beam. With G>1, setting lambda=0 leaves separate smaller beams; it does not generally reconstruct one width-B beam and can reproduce outputs across groups.
- Distinctness is enforced within each selected group. The soft inter-group diversity term does not impose hard global output uniqueness, semantic correctness or a minimum pairwise distance. Large diversity weight can overwhelm model scores.
- The diversity measure and strength, group allocation/order, score ties, EOS handling and maximum length are part of the configuration. Prefix-based diversity cannot automatically use an arbitrary sentence-level metric that requires completed outputs.
- The paper reports empirical quality/diversity gains in specified generation experiments. No universal superiority, runtime speedup, numerical benchmark result or current library behavior is asserted here.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:NaturalLanguageProcessing
Natural Language Processing
|
related_match | The inspected primary paper develops this decoder for natural-language sequence generation, including machine translation. The exact AIO CLASS denotes the language-processing task. This is an algorithm-to-application-domain relationship, not equivalence or an is-a assertion; the decoder is separate from the model supplying probabilities and is not restricted to natural-language sequences. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Ashwin K. Vijayakumar, Michael Cogswell, Ramprasaath R. Selvaraju, Qing Sun, Stefan Lee, David Crandall, Dhruv Batra
|
primary paper
full_text
|
ID: doi:10.1609/aaai.v32i1.12340
sha256:c002058498ea…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T16:13:12Z
Updated: 2026-10-11T16:13:12Z
Source File:
data/curated/DiverseBeamSearch.yaml