Bounded adaptive reward for sequence scoring
Mathematical Methods AIMech:BoundedAdaptiveReward DRAFT
A beam-dependent sequence rescoring rule that adds per-step rewards derived from the beam’s mean negative log token probability, accumulating them only up to a predicted output-length cap.
Identity Notes & Concept Scope
Yang et al.’s bounded adaptive-reward changes the reward rate with the decoding step, unlike fixed-r bounded word-reward. It is a beam-conditioned scoring method rather than a reinforcement-learning reward-training algorithm or a beam-independent probability model. No normalized identity match was found across the 442 AIO and 146 prior native records; both checkout text searches included ignored and hidden files and returned no matches for the selected exact IDs, full labels, aliases and source titles.
Claims & Grounded Evidence
1 Claim
Mechanism
#method
For a b-item beam at step t, compute r_t=−(1/b)·sum_i log p(word_i), using the last generated token probability of each beam item in its own context. Add accumulated r_t values to the sequence log probability up to the smaller of the candidate length and predicted length. After the cap is passed, no additional reward accrues, so the adjusted score cannot increase with further nonpositive log-probability increments. The source then permits stopping when the best current adjusted score is below the best completed adjusted score.
⚠️ Curated Limitations & Negative Results
- The reward sequence depends on the beam and model probabilities. Changing beam policy can change the score assigned to a completed hypothesis.
- Removing a manually tuned constant reward rate does not remove the trained translation model, length predictor, beam size or other implementation choices.
- The printed sum uses a predicted length as its upper limit without specifying a noninteger rounding policy. A discrete implementation must define the effective integer cap and EOS/token counting; the record does not assert an author-approved rounding convention.
- Equation (10) assumes b available beam entries and defined token log probabilities. Empty or shrunken beams and zero-probability tokens require an explicit policy.
- The stated stopping argument applies after reward saturation with a completed incumbent. It is limited to retained beam continuations, not global sequence optimality or guaranteed quality improvement.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:NaturalLanguageProcessing
Natural Language Processing
|
related_match | The primary paper defines this sequence scoring, stopping or pruning method for natural-language generation, with neural machine translation as its central setting. The exact AIO CLASS describes the language-processing task. This is an method-to-application-domain relationship, not equivalence or an is-a assertion; the method is not a trained language-model instance and may also apply outside NLP. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Yilin Yang, Liang Huang, Mingbo Ma
|
primary paper
full_text
|
ID: doi:10.18653/v1/D18-1342
sha256:019ba14d74ad…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T06:36:44Z
Updated: 2026-10-11T06:37:11Z
Source File:
data/curated/BoundedAdaptiveReward.yaml