Attention-based Dropout Layer
Layers & Modules AIMech:AttentionBasedDropoutLayer DRAFT
Aliases: ADL
A training-time convolutional feature-map layer that stochastically applies either a thresholded spatial drop mask or sigmoid importance weights derived from channel-averaged activations, developed for weakly supervised object localization.
Identity Notes & Concept Scope
This is the 2019 ADL component, not a complete localization network or a trained checkpoint. Its self-attention map is a channel average of feature activations, not dot-product query/key attention. It differs from random-block DropBlock, whole-channel dropout and Cutout by selecting strongly activated spatial regions from the current feature map and alternating their suppression with continuous importance gating. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 273 preceding native IDs/names/aliases. Full-checkout fixed-string searches included ignored and hidden files in worktree and main, returning 18 and 18 lines respectively. Every text hit was a prior candidate lead. Eleven exact AIO CLASS nodes with scoped synonyms/comments and outgoing edges and five native neighboring mechanisms were inspected. Absence is bounded to this snapshot and semantic neighborhood.
Claims & Grounded Evidence
1 Claim
Mechanism
#method
Average the input convolutional feature tensor over channels to obtain one spatial attention map. Set a threshold to a configured fraction gamma of that map’s maximum; construct a binary drop mask that suppresses positions above the threshold. Separately apply a sigmoid to the attention map to obtain importance weights. On a training iteration, choose the drop mask with the configured drop rate and otherwise choose the importance map, then multiply the selected spatial map into the input features across channels. These map operations add no trainable parameters; gradients through the importance branch train the surrounding classifier. ADL is disabled at testing and returns the feature map unchanged, without a train/test distribution compensation step. The source uses image-level classification labels and a separate heatmap extraction/localization procedure; ADL itself does not output an object bounding box.
⚠️ Curated Limitations & Negative Results
- The drop rate controls how often the drop-mask branch is selected, not the fraction of pixels removed. Thresholding depends on the activation distribution; the source prose specifies greater-than and less-than comparisons but leaves equality at the threshold unresolved. The record does not invent a tie rule or implementation-specific batching convention.
- Sigmoid importance weights lie between zero and one and reweight regions relative to each other; they do not necessarily amplify raw activation magnitudes. The averaged attention map is a heuristic signal, not a calibrated explanation or guaranteed object mask.
- No added trainable parameters does not mean zero training computation. Placement, threshold and branch probability are choices; classification and localization can trade off, and low-level maps need not capture class-specific object regions.
- The source explicitly shows that attention suppression can encourage reliance on co-occurring background rather than the object. The inspected evaluation is single-object localization; broader segmentation compatibility is a proposed application, not an established universal result. No benchmark superiority or complete-object-coverage guarantee is asserted.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:Layer
Layer
|
broad_match | ADL is a specific input-to-output feature-map component inside a classifier, so the generic AIO Layer has broader scope. |
AIO:DropoutLayer
Dropout Layer
|
related_match | ADL stochastically suppresses activations during training, which relates it to dropout. Its activation-dependent mask, alternative sigmoid branch and lack of the pinned fixed-rate dropout rescaling prevent identity or an unqualified subclass assertion. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Junsuk Choe, Hyunjung Shim
|
primary paper
full_text
|
sha256:5c9518189d80…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T12:27:02Z
Updated: 2026-10-11T12:27:02Z
Source File:
data/curated/AttentionBasedDropoutLayer.yaml