Residual Attention Network
Architectures AIMech:ResidualAttentionNetwork DRAFT
Aliases: RAN (residual attention)
A convolutional-network architecture that stacks attention modules combining trunk features with learned bottom-up and top-down masks through residual attention gating.
Identity Notes & Concept Scope
This record covers the 2017 architecture with its canonical mixed sigmoid masks and attention residual learning. It is a network construction rather than a single attention block, trained model or generic ResNet. Named depths such as Attention-56 and Attention-92 configure this architecture; constrained channel/spatial mask variants are not silently substituted for the selected mixed-mask procedure. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 282 preceding native IDs/names/aliases. Full-checkout fixed-string searches included ignored and hidden files in worktree and main, returning 2 and 2 lines respectively. Every text hit was a prior candidate lead or a closely related variant lead. Ten exact AIO CLASS nodes with scoped synonyms/comments and outgoing edges and five native neighboring mechanisms were inspected. Absence is bounded to this snapshot and semantic neighborhood.
Claims & Grounded Evidence
1 Claim
Mechanism
#method
Stack attention modules at successive network stages. Within a module, a trunk branch computes convolutional features while a mask branch pools down to broader context and upsamples through a top-down path, with skip connections between scales. The source uses max pooling, bilinear interpolation, intermediate feature-processing units, two final 1-by-1 convolutions and an elementwise sigmoid to form a mask matching the trunk tensor in spatial positions and channels. Combine them as one-plus-mask multiplied elementwise by the trunk output. This retains the trunk signal when the mask approaches zero while allowing input-dependent relative emphasis. The source composes these modules with feature-processing units and a classifier; numbers of modules, stage widths, branch depths and basic units are architecture choices. The bottom-up/top-down process is implemented in a single feedforward computation, not an iterative recurrent feedback loop.
⚠️ Curated Limitations & Negative Results
- Attention residual learning uses (1+M)*T, not raw-input addition x+M*T and not the naive product M*T. A near-zero mask preserves the trunk output T, which need not equal the module input x. Residual units within the trunk are a separate architectural choice.
- For the selected sigmoid mask, one-plus-mask lies between one and two for finite real logits. Immediate trunk magnitudes are therefore not attenuated; noise suppression in the source is a relative or network-level interpretation. The source’s L2-normalized channel-mask variant does not necessarily obey these sigmoid bounds.
- The gradient formula with factor M in Equation2 describes the naive product. For fixed mask and separate trunk parameters, the residual formulation instead has factor1+M. It does not by itself prevent an erroneous-label gradient or guarantee robustness, gradient stability or monotonically improving accuracy with depth.
- Masks are learned feature weights, not calibrated object probabilities or causal explanations. The paper’s classification and noisy-label experiments have specific conditions; detection/segmentation are proposed follow-up applications. No benchmark superiority, unlimited depth or author-code execution is asserted.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:DeepConvolutionalNetwork
Deep Convolutional Network
|
broad_match | The source explicitly constructs a deep convolutional image-recognition network, which fits this broader architecture category. Its learned multiscale attention modules provide the narrower identity. |
AIO:ResidualNeuralNetwork
Residual Neural Network
|
related_match | The source uses residual building blocks and relates attention residual learning to ResNet. Its (1+mask)*trunk formulation is not itself an identity skip over the module input, and the framework permits different basic units, so the relationship is not equivalence. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Fei Wang, Mengqing Jiang, Chen Qian, Shuo Yang, Cheng Li, Honggang Zhang, Xiaogang Wang, Xiaoou Tang
|
primary paper
full_text
|
sha256:51451a968e81…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T12:54:15Z
Updated: 2026-10-11T12:54:15Z
Source File:
data/curated/ResidualAttentionNetwork.yaml