Home / Curated / Layers & Modules

Layers & Modules

18 Concepts

Defined neural network components that transform representations between stages.

Showing 18 of 18

Concept	Origin	Kind / Branch	Identifier	Evidence / Status
Attention-based Dropout Layer aka ADL A training-time convolutional feature-map layer that stochastically applies either a thresholded spatial drop mask or sigmoid importance wei…	Curated	Layers & Modules	`AIMech:AttentionBasedDropoutLayer`	1 claim 1 source
Balanced Attention Mechanism aka BAM (balanced attention) A convolutional feature-gating component for single-image super-resolution that computes average-pooled channel attention and maximum-pooled…	Curated	Layers & Modules	`AIMech:BalancedAttentionMechanism`	6 claims 1 source
Bottleneck Attention Module aka BAM (bottleneck attention) A convolutional feature refinement component that combines parallel channel and spatial attention branches and adds the resulting gated feat…	Curated	Layers & Modules	`AIMech:BottleneckAttentionModule`	2 claims 2 sources
Compositional spatial transformer A learnable image-alignment module that retains an original source image and accumulated warp state, samples that source under the current w…	Curated	Layers & Modules	`AIMech:CompositionalSpatialTransformer`	5 claims 1 source
Convolutional Block Attention Module aka CBAM A convolutional feature refinement component that sequentially applies learned channel attention and then spatial attention to an intermedia…	Curated	Layers & Modules	`AIMech:ConvolutionalBlockAttentionModule`	2 claims 2 sources
DropBlock A training-time feature-map transformation for convolutional networks that suppresses randomly placed contiguous spatial blocks of activatio…	Curated	Layers & Modules	`AIMech:DropBlock`	1 claim 2 sources
DropConnect A stochastic neural-network layer that masks individual weights and biases during training, so each example is processed through a randomly…	Curated	Layers & Modules	`AIMech:DropConnect`	1 claim 1 source
Dynamic Convolutional Layer aka Dynamic Convolution Layer A neural layer that convolves feature maps with filters generated from the current input by a learned subnetwork, allowing the applied kerne…	Curated	Layers & Modules	`AIMech:DynamicConvolutionalLayer`	6 claims 1 source
Feature-wise Linear Modulation aka FiLM A neural conditioning layer that applies a feature-wise affine transformation whose scale and shift are generated from conditioning informat…	Curated	Layers & Modules	`AIMech:FeatureWiseLinearModulation`	5 claims 1 source
Gaussian Differentiable Pooling A spatial pooling and unpooling component whose nonnegative, unit-L2 weights are square-root normalized axis-aligned Gaussian samples, with…	Curated	Layers & Modules	`AIMech:GaussianDifferentiablePooling`	5 claims 1 source
Hierarchical softmax A neural categorical-output component that assigns a probability to each vocabulary leaf by multiplying normalized branch probabilities alon…	Curated	Layers & Modules	`AIMech:HierarchicalSoftmax`	1 claim 3 sources
Hungry Hungry Hippos layer aka H3 A sequence layer that combines projected queries, keys and values with shift and diagonal state-space models, using multiplicative interacti…	Curated	Layers & Modules	`AIMech:H3Layer`	1 claim 1 source
Linear state-space layer aka LSSL A sequence-to-sequence layer defined by discretizing a linear continuous-time state-space system, with a state recurrence and, for zero init…	Curated	Layers & Modules	`AIMech:LinearStateSpaceLayer`	1 claim 1 source
Non-local block aka Non-local neural-network block A neural-network component that aggregates input-dependent pairwise contributions from positions across a representation, projects the aggre…	Curated	Layers & Modules	`AIMech:NonLocalBlock`	1 claim 1 source
Relation network A reusable neural module for relational reasoning that applies a shared learned function to pairs of object representations, sums the result…	Curated	Layers & Modules	`AIMech:RelationNetwork`	6 claims 1 source
Spatial Transformer aka Spatial Transformer module A learnable neural-network component that predicts an input-dependent spatial transformation and uses the resulting sampling grid to warp an…	Curated	Layers & Modules	`AIMech:SpatialTransformer`	2 claims 2 sources
Squeeze-and-Excitation block aka SE block A neural-network feature recalibration component that globally averages each channel, transforms the resulting descriptor through a learned…	Curated	Layers & Modules	`AIMech:SqueezeAndExcitationBlock`	1 claim 1 source
Stochastic Pooling A spatial neural pooling layer that samples one activation per region with probability proportional to its nonnegative value during training…	Curated	Layers & Modules	`AIMech:StochasticPooling`	5 claims 1 source