Home / Curated / Architectures

Architectures

71 Concepts

Structural blueprints and macro-designs for deep learning models.

Showing 71 of 71

Concept	Origin	Kind / Branch	Identifier	Evidence / Status
Adaptive Deconvolutional Network An unsupervised hierarchical image-representation architecture that alternates convolutional sparse synthesis with spatial and cross-map poo…	Curated	Architectures	`AIMech:AdaptiveDeconvolutionalNetwork`	5 claims 2 sources
Attention-Based Configurable Convolutional Neural Network aka ABC-CNN A visual-question-answering architecture that derives a convolution kernel from an encoded question to compute normalized spatial attention,…	Curated	Architectures	`AIMech:AttentionBasedConfigurableConvolutionalNeuralNetwork`	6 claims 1 source
Attention-Based Convolutional Neural Network aka ABCNN A sentence-pair neural architecture that uses cross-sentence distance affinities to add attention feature channels before convolution, rewei…	Curated	Architectures	`AIMech:AttentionBasedConvolutionalNeuralNetwork`	6 claims 1 source
Attentive Pooling Network aka Attentive Pooling Networks A neural sentence-pair architecture that learns bilinear segment interactions, derives attention by row and column maxima followed by separa…	Curated	Architectures	`AIMech:AttentivePoolingNetwork`	6 claims 1 source
Autoregressive conditional restricted Boltzmann machine aka Autoregressive CRBM A neural sequence architecture that models each current vector with a conditional RBM whose visible and hidden biases are affine functions o…	Curated	Architectures	`AIMech:AutoregressiveConditionalRestrictedBoltzmannMachine`	1 claim 2 sources
Clockwork recurrent neural network aka CW-RNN, Clockwork RNN A recurrent architecture partitioning hidden units into modules with prescribed update periods, copying inactive module states and restricti…	Curated	Architectures	`AIMech:ClockworkRecurrentNeuralNetwork`	6 claims 1 source
Communication Neural Network aka CommNet, Communication Neural Net… A neural architecture for cooperative multi-agent tasks that alternates shared per-agent hidden-state updates with continuous communication…	Curated	Architectures	`AIMech:CommunicationNeuralNetwork`	6 claims 1 source
Conditional restricted Boltzmann machine aka CRBM, Conditional RBM A conditional neural probability model that represents an output vector with an RBM whose biases or interactions are determined by a supplie…	Curated	Architectures	`AIMech:ConditionalRestrictedBoltzmannMachine`	1 claim 1 source
Deep autoregressive network aka DARN A generative neural architecture that combines directed stochastic hidden layers with autoregressive within-layer conditional dependencies a…	Curated	Architectures	`AIMech:DeepAutoregressiveNetwork`	1 claim 1 source
Deep Output Recurrent Neural Network aka DO-RNN A recurrent neural architecture whose hidden-to-output function uses a multilayer perceptron with intermediate nonlinear feedforward layers…	Curated	Architectures	`AIMech:DeepOutputRecurrentNeuralNetwork`	4 claims 1 source
Deep Output-Transition Recurrent Neural Network aka DOT-RNN, Deep output, deep transition RNN A recurrent neural architecture that uses intermediate nonlinear feedforward layers in both the hidden-state transition from the previous st…	Curated	Architectures	`AIMech:DeepOutputTransitionRecurrentNeuralNetwork`	6 claims 1 source
Deep Sets A neural architecture family for unordered collections that combines shared element transformations with symmetric aggregation for invariant…	Curated	Architectures	`AIMech:DeepSets`	1 claim 1 source
Deep Sets++ aka DS++ A Deep Sets architecture revision whose encoder combines identity residual paths with set normalization while retaining the original aggrega…	Curated	Architectures	`AIMech:DeepSetsPlusPlus`	1 claim 1 source
Deep Transition Recurrent Neural Network aka DT-RNN A recurrent neural architecture whose state-transition function uses a multilayer perceptron with intermediate nonlinear feedforward layers…	Curated	Architectures	`AIMech:DeepTransitionRecurrentNeuralNetwork`	4 claims 1 source
Depth-Gated LSTM aka DGLSTM, Depth-gated long short-term memory A stacked LSTM architecture that adds a learned, elementwise gated contribution from the current lower-layer memory cell to the upper-layer…	Curated	Architectures	`AIMech:DepthGatedLSTM`	6 claims 1 source
Document NADE aka DocNADE A neural autoregressive document model that predicts successive words from shared additive embeddings of the preceding words, using normaliz…	Curated	Architectures	`AIMech:DocumentNADE`	1 claim 2 sources
Dynamic Memory Network aka DMN, Dynamic Memory Networks A recurrent neural architecture that encodes an input and a question, repeatedly forms question-conditioned episodes from input facts to upd…	Curated	Architectures	`AIMech:DynamicMemoryNetwork`	6 claims 1 source
Dynamic Memory Network Plus aka DMN+ A recurrent question-answering architecture that fuses text or image facts, aggregates them with question- and memory-conditioned attention…	Curated	Architectures	`AIMech:DynamicMemoryNetworkPlus`	6 claims 1 source
End-to-End Memory Network aka MemN2N, End-to-End Memory Networks A neural architecture that answers a query through one or more differentiable soft-attention reads over separately embedded input and output…	Curated	Architectures	`AIMech:EndToEndMemoryNetwork`	6 claims 1 source
Factored restricted Boltzmann machine language model aka FRBM A conditional neural word model that factorizes position-specific word-to-hidden weights through a shared word-embedding matrix and couples…	Curated	Architectures	`AIMech:FactoredRestrictedBoltzmannMachineLanguageModel`	1 claim 1 source
Fully visible sigmoid belief network aka FVSBN, Logistic autoregressive Bayesian network A fully observed binary autoregressive probabilistic model whose conditional probability at each position is a logistic function of a bias a…	Curated	Architectures	`AIMech:FullyVisibleSigmoidBeliefNetwork`	1 claim 3 sources
Gamma neural model aka Gamma neural network, Gamma net… A neural architecture family that represents delayed inputs or neural outputs by weighted gamma-kernel memory traces, realized through casca…	Curated	Architectures	`AIMech:GammaNeuralModel`	6 claims 2 sources
Gated Graph Neural Network aka GG-NN, Gated Graph Neural Networks A graph neural architecture that initializes node states from annotations, repeatedly combines edge-type- and direction-specific neighbor me…	Curated	Architectures	`AIMech:GatedGraphNeuralNetwork`	6 claims 1 source
Gated Graph Sequence Neural Network aka GGS-NN, Gated Graph Sequence Neural Networks A graph-to-sequence neural architecture that repeatedly runs gated graph propagation to predict an output and update node annotations, using…	Curated	Architectures	`AIMech:GatedGraphSequenceNeuralNetwork`	6 claims 1 source
Gated-feedback recurrent neural network aka GF-RNN A layered recurrent architecture in which learned input-dependent scalar gates modulate recurrent projections from every previous layer stat…	Curated	Architectures	`AIMech:GatedFeedbackRecurrentNeuralNetwork`	6 claims 1 source
GENET A connectionist architecture for finite constraint satisfaction that represents variable assignments by active label nodes and alternates in…	Curated	Architectures	`AIMech:GENET`	1 claim 1 source
Graph attention network aka GAT A graph neural architecture that updates node representations using learned attention weights over graph neighborhoods.	Curated	Architectures	`AIMech:GraphAttentionNetwork`	1 claim 1 source
Graph attention network v2 aka GATv2 A graph neural architecture whose neighborhood attention scorer applies a learned scalar projection after a nonlinear transformation of the…	Curated	Architectures	`AIMech:GATv2`	1 claim 1 source
Graph isomorphism network aka GIN A graph neural architecture using multilayer perceptrons on sums of neighboring representations and a separately weighted self-representatio…	Curated	Architectures	`AIMech:GraphIsomorphismNetwork`	1 claim 1 source
GraphSAGE An inductive graph representation architecture that learns neighborhood aggregation functions to generate node embeddings from features.	Curated	Architectures	`AIMech:GraphSAGE`	1 claim 1 source
Grid Long Short-Term Memory aka Grid LSTM A neural architecture family whose grid blocks couple hidden representations across computational dimensions while maintaining separate gate…	Curated	Architectures	`AIMech:GridLongShortTermMemory`	6 claims 1 source
Guarded discrete stochastic network A binary neural constraint-satisfaction architecture whose main assignment neurons are asymmetrically coupled to empty-row guards and update…	Curated	Architectures	`AIMech:GuardedDiscreteStochasticNetwork`	5 claims 2 sources
Helmholtz machine A hierarchical probabilistic neural model that pairs a top-down generative model with a separately parameterized bottom-up recognition model…	Curated	Architectures	`AIMech:HelmholtzMachine`	1 claim 2 sources
Hierarchical log-bilinear language model aka HLBL A next-word language architecture that computes one linear context feature vector and uses its dot products with internal-node vectors to as…	Curated	Architectures	`AIMech:HierarchicalLogBilinearLanguageModel`	1 claim 1 source
Hierarchical probabilistic neural network language model A neural next-word architecture that factors word probabilities along a binary vocabulary tree and shares a nonlinear branch predictor condi…	Curated	Architectures	`AIMech:HierarchicalProbabilisticNeuralNetworkLanguageModel`	1 claim 1 source
Identity-initialized rectified recurrent neural network aka IRNN A recurrent neural sequence architecture using rectified linear hidden units, zero initial recurrent biases, and a trainable recurrent matri…	Curated	Architectures	`AIMech:IdentityInitializedRecurrentNeuralNetwork`	6 claims 1 source
Interaction Network aka Interaction Networks, IN (interaction network) A graph-based neural architecture that applies a shared relation model to supplied directed attributed object relations, sums predicted effe…	Curated	Architectures	`AIMech:InteractionNetwork`	6 claims 1 source
Inverse compositional spatial transformer network A recurrent learned image-alignment architecture that repeatedly applies one shared geometric predictor to the original source sampled at th…	Curated	Architectures	`AIMech:InverseCompositionalSpatialTransformerNetwork`	5 claims 1 source
Jumping Knowledge network aka JK-Net, Jumping Knowledge networks A graph neural architecture that sends intermediate node representations from multiple neighborhood-aggregation depths to a final layer-aggr…	Curated	Architectures	`AIMech:JumpingKnowledgeNetwork`	1 claim 1 source
Key-Value Memory Network aka KV-MemNN, Key-Value Memory Networks A neural memory architecture that answers a query by attending to keys and reading corresponding values from supplied key-value memory pairs…	Curated	Architectures	`AIMech:KeyValueMemoryNetwork`	6 claims 1 source
List-augmented recurrent neural network aka List RNN A recurrent sequence architecture whose neural controller accesses an external list through a movable read/write head, learning continuous m…	Curated	Architectures	`AIMech:ListAugmentedRecurrentNeuralNetwork`	6 claims 1 source
Log-bilinear language model aka LBL A next-word language architecture that linearly combines position-specific transforms of context-word embeddings and normalizes their biline…	Curated	Architectures	`AIMech:LogBilinearLanguageModel`	1 claim 2 sources
Mamba sequence architecture aka Mamba A neural sequence architecture built from repeated blocks containing input-dependent selective state-space transformations.	Curated	Architectures	`AIMech:Mamba`	1 claim 1 source
Mixture density network aka MDN A neural conditional-density architecture whose input-dependent outputs parameterize the mixing weights and component distributions of a fin…	Curated	Architectures	`AIMech:MixtureDensityNetwork`	1 claim 1 source
Neural autoregressive distribution estimator aka NADE A fully visible autoregressive density model whose ordered variable conditionals are computed by neural networks that share input-to-hidden…	Curated	Architectures	`AIMech:NeuralAutoregressiveDistributionEstimator`	1 claim 1 source
Neural GPU aka Neural GPUs A recurrent neural architecture that embeds a complete symbolic sequence in a spatial state grid and repeatedly applies shared convolutional…	Curated	Architectures	`AIMech:NeuralGPU`	6 claims 2 sources
Neural network pushdown automaton aka NNPDA A recurrent state-controller architecture whose signed action output pushes or removes fractional quantities of input symbols on an external…	Curated	Architectures	`AIMech:NeuralNetworkPushdownAutomaton`	6 claims 1 source
Neural probabilistic language model aka NPLM A neural finite-context language architecture that jointly learns a shared word-embedding lookup and a nonlinear predictor of the next-word…	Curated	Architectures	`AIMech:NeuralProbabilisticLanguageModel`	1 claim 1 source
Noisy-OR belief network A directed acyclic binary probabilistic network that models a node’s inactivity as the product of a leak-failure factor and failure factors…	Curated	Architectures	`AIMech:NoisyORBeliefNetwork`	1 claim 1 source
Perspective Transformer Nets An image-to-volume neural architecture that combines a convolutional image encoder and volumetric decoder with a supplied-camera projection…	Curated	Architectures	`AIMech:PerspectiveTransformerNets`	6 claims 2 sources
PointNet A neural architecture for unordered point clouds that applies shared pointwise feature extraction and symmetric max pooling to produce a glo…	Curated	Architectures	`AIMech:PointNet`	1 claim 1 source
PointNet++ A hierarchical point-set neural architecture that repeatedly samples centroids, groups metric neighborhoods and applies local PointNet featu…	Curated	Architectures	`AIMech:PointNetPlusPlus`	1 claim 1 source
Real-valued neural autoregressive density estimator aka RNADE, Real-valued autoregressive density estimator A fully visible neural autoregressive density architecture for real-valued vectors that shares input-to-hidden parameters across ordered sca…	Curated	Architectures	`AIMech:RealValuedNeuralAutoregressiveDensityEstimator`	1 claim 2 sources
Recurrent temporal restricted Boltzmann machine aka RTRBM A recurrent neural sequence architecture whose deterministic state is a conditional RBM hidden-probability vector and sets the next step’s h…	Curated	Architectures	`AIMech:RecurrentTemporalRestrictedBoltzmannMachine`	1 claim 1 source
Replicated Softmax An undirected document model with binary hidden units and a length-dependent set of categorical word units that share visible-to-hidden weig…	Curated	Architectures	`AIMech:ReplicatedSoftmax`	1 claim 1 source
Residual Attention Network aka RAN (residual attention) A convolutional-network architecture that stacks attention modules combining trunk features with learned bottom-up and top-down masks throug…	Curated	Architectures	`AIMech:ResidualAttentionNetwork`	1 claim 1 source
Retentive network aka RetNet An autoregressive neural sequence architecture that stacks pre-normalized residual blocks combining gated multi-scale retention with feedfor…	Curated	Architectures	`AIMech:RetNet`	1 claim 1 source
RNN-NADE aka Recurrent neural network neural autoregressive distribution estimator A neural sequence architecture in which a deterministic recurrent state supplies parameters of a NADE output distribution, combining tempora…	Curated	Architectures	`AIMech:RecurrentNeuralNetworkNeuralAutoregressiveDistributionEstimator`	1 claim 2 sources
RNN-RBM aka Recurrent neural network restricted Boltzmann machine A neural sequence architecture in which a deterministic recurrent network with its own hidden state supplies parameters of a conditional RBM…	Curated	Architectures	`AIMech:RecurrentNeuralNetworkRestrictedBoltzmannMachine`	1 claim 1 source
Set Transformer A neural encoder-decoder architecture for permutation-invariant set-input prediction that uses attention to encode interactions among elemen…	Curated	Architectures	`AIMech:SetTransformer`	1 claim 1 source
Set Transformer++ aka ST++ A Set Transformer architecture revision that replaces its induced-attention encoder blocks with set-normalized blocks retaining identity res…	Curated	Architectures	`AIMech:SetTransformerPlusPlus`	1 claim 1 source
Sigmoid belief network aka SBN A directed acyclic probabilistic network of binary variables whose forward conditional probabilities are logistic functions of weighted pare…	Curated	Architectures	`AIMech:SigmoidBeliefNetwork`	1 claim 1 source
SimNet aka SimNets A neural architecture family built from learned template-similarity operations and MEX log-mean-exp operations that combine shifted similari…	Curated	Architectures	`AIMech:SimNet`	8 claims 2 sources
Stack-augmented recurrent neural network aka Stack RNN A recurrent sequence architecture whose neural controller reads the leading entries of one or more external stack states and learns continuo…	Curated	Architectures	`AIMech:StackAugmentedRecurrentNeuralNetwork`	6 claims 3 sources
Structurally constrained recurrent neural network aka SCRN, SCRNN A recurrent sequence architecture coupling a nonlinear recurrent hidden state with a separate linear context state whose recurrence is const…	Curated	Architectures	`AIMech:StructurallyConstrainedRecurrentNeuralNetwork`	6 claims 5 sources
Temporal factored restricted Boltzmann machine language model aka Temporal FRBM A sequential extension of the factored RBM language model that directs information from the preceding binary hidden state into the current h…	Curated	Architectures	`AIMech:TemporalFactoredRestrictedBoltzmannMachineLanguageModel`	1 claim 2 sources
Temporal restricted Boltzmann machine aka TRBM A neural sequence architecture that multiplies time-indexed conditional RBM distributions whose visible and hidden biases depend on precedin…	Curated	Architectures	`AIMech:TemporalRestrictedBoltzmannMachine`	1 claim 2 sources
Temporal-kernel recurrent neural network aka TKRNN A recurrent architecture that factors connections across time into learned spatial weights and source-specific exponential temporal kernels,…	Curated	Architectures	`AIMech:TemporalKernelRecurrentNeuralNetwork`	6 claims 1 source
Transforming autoencoder A deterministic neural architecture that learns first-level capsules from images, transformed target images and supplied transformations. Ea…	Curated	Architectures	`AIMech:TransformingAutoencoder`	6 claims 1 source
Variational Fair Autoencoder aka VFAE A variational autoencoder architecture for learning representations with reduced sensitive or nuisance information, combining sensitive-vari…	Curated	Architectures	`AIMech:VariationalFairAutoencoder`	1 claim 1 source
Vertex Attention Interaction Network aka VAIN A neural architecture for multi-agent predictive modeling that encodes each agent separately, aggregates communication vectors using learned…	Curated	Architectures	`AIMech:VertexAttentionInteractionNetwork`	6 claims 1 source