FlashAttention
Algorithms AIMech:FlashAttention DRAFT
An exact attention computation algorithm using tiling and recomputation to reduce transfers between GPU memory levels.
Identity Notes & Concept Scope
The original dense exact-attention algorithm, not a distinct trained model, a generic attention layer, or its separately described block-sparse approximation.
Claims & Grounded Evidence
1 Claim
Mechanism
#mechanism
Blocks of queries, keys and values are processed with running softmax statistics; intermediate attention values can be recomputed during the backward pass.
⚠️ Curated Limitations & Negative Results
- The block-sparse extension and later FlashAttention versions are outside this record.
- Reduced memory traffic does not establish a universal wall-clock speedup; no hardware-independent performance claim is made.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:AttentionLayer
Attention Layer
|
related_match | FlashAttention changes how attention is computed and stored, while this AIO target denotes a layer implementing attention. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
original-paper
|
Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, Christopher Ré
|
primary paper
full_text
|
ID: arXiv:2205.14135v2
sha256:ca7f9fda10b9…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T03:02:19Z
Updated: 2026-10-11T03:02:19Z
Source File:
data/curated/FlashAttention.yaml