Trust region policy optimization
Algorithms AIMech:TrustRegionPolicyOptimization DRAFT
Aliases: TRPO
A policy-optimization method that optimizes a surrogate objective while constraining the change in policy by a trust region based on Kullback–Leibler divergence.
Identity Notes & Concept Scope
TRPO's constrained policy update differs from PPO's clipped surrogate or adaptive penalty variants. It is an optimization procedure, not a trained policy instance.
Claims & Grounded Evidence
1 Claim
Mechanism
#learning-method
The practical method estimates a surrogate objective from sampled trajectories and constrains the average divergence from the old policy, using an approximate constrained update rather than evaluating a maximum-divergence constraint at every state.
⚠️ Curated Limitations & Negative Results
- The paper distinguishes its theoretical improvement scheme from practical approximations; this record does not guarantee monotonic improvement for every practical update.
- Single-path and vine sampling have different access requirements; the latter assumes the ability to restore sampled states.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ReinforcementLearning
Reinforcement Learning
|
related_match | This algorithm implements reinforcement-learning updates. The AIO target is a learning task, not the same method or an asserted algorithm superclass. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
original-paper
|
John Schulman, Sergey Levine, Philipp Moritz, Michael Jordan, Pieter Abbeel
|
primary paper
full_text
|
sha256:8e959ae88cae…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T03:44:41Z
Updated: 2026-10-11T03:44:41Z
Source File:
data/curated/TrustRegionPolicyOptimization.yaml