Home / Curated / Layers & Modules / Bottleneck Attention Module

Bottleneck Attention Module

Layers & Modules AIMech:BottleneckAttentionModule DRAFT

Aliases: BAM (bottleneck attention)

A convolutional feature refinement component that combines parallel channel and spatial attention branches and adds the resulting gated features back to its input, originally proposed for network downsampling bottlenecks.

Identity Notes & Concept Scope This record denotes Park and colleagues’ 2018 bottleneck attention module, scoped to the summation convention of arXiv1807.06514v2. The pinned author implementation uses a different fusion operation, recorded explicitly below. BAM is an ambiguous acronym also used by the 2021 Balanced Attention Mechanism paper; only the qualified bottleneck alias is assigned here. The module is distinct from sequential CBAM, channel-only SE and a complete CNN. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 279 preceding native IDs/names/aliases. Full-checkout fixed-string searches included ignored and hidden files in worktree and main, returning 6 and 6 lines respectively. Every text hit was a prior candidate lead or a closely related variant lead. Seven exact AIO CLASS nodes with scoped synonyms/comments and outgoing edges and three native neighboring mechanisms were inspected. Absence is bounded to this snapshot and semantic neighborhood.

Claims & Grounded Evidence

2 Claims

Mechanism #method

From the same input feature tensor, form a channel branch using global spatial averaging and a bottleneck MLP, and a spatial branch using a 1-by-1 channel reduction, two dilated 3-by-3 convolutions and a 1-by-1 projection to one spatial map. The selected paper equations place batch normalization at the outputs of both branches. Broadcast the channel and spatial logits to the input tensor shape, sum them and apply sigmoid to obtain the attention map. Multiply that map by the input and add the result back to the input. The source commonly places modules at downsampling bottlenecks and also studies other placements; its reduction ratio 16 and dilation 4 are experimental settings. The learned attention computation operates in both training and inference.

supports Source: BAM: Bottleneck Attention Module · Locator: Section3, Equations1–4, PDF pp.3–5; branch fusion and placement comparisons Section4.1, PDF pp.6–8; other placements Section4.6 and conclusion, PDF pp.9–10

Limitation #paper-code-boundary

The selected paper adds broadcast channel and spatial branch logits before sigmoid. The inspected BAM.forward instead multiplies those branch outputs before applying sigmoid and adding one. The code places batch normalization and ReLU in intermediate branch blocks rather than the terminal batch-normalization positions displayed in the paper equations. These are explicit source-version differences, not equivalent rewritings; the concept record uses the paper convention and does not certify the code as an exact implementation of it.

supports Source: BAM: Bottleneck Attention Module · Locator: Section3 Equations2–4 and Section4.1 Combining methods, PDF pp.4–5,7
supports Source: Official attention-module implementation: MODELS/bam.py · Locator: ChannelGate.__init__, SpatialGate.__init__ and BAM.forward in MODELS/bam.py

⚠️ Curated Limitations & Negative Results

The record distinguishes the selected paper’s SUM fusion from the pinned author code’s product fusion and normalization placement. These yield different attention maps; neither the repository README nor a shared module name proves that code reproduces the selected paper equations.
For finite real logits the residual multiplier 1+sigmoid(logit) lies between one and two. The module therefore cannot directly attenuate the absolute magnitude of a nonzero input element; suppression in source prose is relative to other features or downstream computation. Floating-point sigmoid can reach numerical endpoints.
The branch maps are constrained by their channel/spatial structure rather than arbitrary independent attention values at every tensor position. Learned attention is not automatically a calibrated explanation or semantic object mask. Placement and hyperparameters affect overhead and empirical results.
The inspected standalone bam.py references gate_activation in ChannelGate without a parameter or binding for that name in the module. Its average-pooling call also assumes square spatial maps to produce a single descriptor. Static reading is not a successful runtime validation; no author code, model or supplementary benchmark was executed. The final BMVC PDF was inaccessible during this assessment.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:Layer` Layer	broad_match	The native concept is a particular learned input-to-output feature-map component inside a neural network. The pinned generic Layer class is broader; the pinned AttentionLayer specifically denotes dot-product attention and is not used as a generic attention parent.

Evidence Sources & Primary Literature

2 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`method-source`	BAM: Bottleneck Attention Module Jongchan Park, Sanghyun Woo, Joon-Young Lee, In So Kweon	primary paper full_text	ID: arXiv:1807.06514v2 sha256:56235ca53612…
`pinned-code`	Official attention-module implementation: MODELS/bam.py Jongchan Park, Sanghyun Woo	official documentation documentation	sha256:e2efa83e0e08…

Curation Provenance

Curator: Codex

Method: ai_assisted

Model: GPT-6

Created: 2026-10-11T12:43:35Z

Updated: 2026-10-11T12:43:35Z

Source File: data/curated/BottleneckAttentionModule.yaml