Home / Curated / Model Families / Markov decision process

Markov decision process

Model Families AIMech:MarkovDecisionProcess DRAFT

Aliases: MDP

A sequential decision model with states, actions, Markov transitions, and rewards.

Identity Notes & Concept Scope Model does not require controlled Markov transitions and reward-based sequential decisions.

Claims & Grounded Evidence

1 Claim

Mechanism #mechanism

Actions condition transitions between states and yield rewards used to define a policy objective.

supports Source: Planning and acting in partially observable stochastic domains · Locator: Kaelbling, Littman and Cassandra (1998), section 2.1, printed pp. 101–102 / PDF pp. 3–4: states, actions, transitions and rewards

This record supports method identity and the stated mechanism only; no universal performance or biological efficacy claim is curated.
Proposed placement is an inference from the existing AIO hierarchy; maintainers must resolve task, algorithm and model scope before acceptance.

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:Model` Model	broad_match	Model does not require controlled Markov transitions and reward-based sequential decisions. Mapping is local curation, not an upstream subclass assertion.

1 Source

Source ID	Publication / Document	Type / Access	Reference & Checksum
`method-source`	Planning and acting in partially observable stochastic domains Leslie Pack Kaelbling, Michael L. Littman, Anthony R. Cassandra	primary paper full_text

Curator: Codex

Method: ai_assisted

Model: GPT-6

Created: 2026-10-11T03:01:34Z

Updated: 2026-10-11T03:01:34Z

Source File: data/curated/MarkovDecisionProcess.yaml