Attention-Based Convolutional Neural Network
Architectures AIMech:AttentionBasedConvolutionalNeuralNetwork DRAFT
Aliases: ABCNN
A sentence-pair neural architecture that uses cross-sentence distance affinities to add attention feature channels before convolution, reweight local pooling after convolution, or combine both stages when deriving pair-dependent sentence representations.
Identity Notes & Concept Scope
Compared all 442 pinned AIO and 608 native IDs, names, aliases and scoped synonyms; no normalized identity match. Ignored- and hidden-inclusive fixed-string searches of the recovery, older discovery and original Documents checkouts returned 7, 0, 0 lines. All distinct match contents were inspected. Semantic matching covered 104 AIO and 66 native definitions, with every matched definition read. 7 full annotated AIO node/edge/generated contexts and 3 full native records were inspected. This is Yin and colleagues’ named sentence-pair ABCNN, using reciprocal-distance affinities with input-feature and pooling attention variants. It is not the bilinear/max/softmax Attentive Pooling Network, CBAM channel/spatial refinement, PointNet set pooling, ABC-CNN visual QA or every later model reusing the acronym. ABCNN-1, -2 and -3 remain explicitly different variants.
Claims & Grounded Evidence
6 Claims
Mechanism
#shared-sentence-convolution
The sentence-pair backbone shares convolutional parameters across the two inputs. Wide zero-padded convolution with tanh produces local feature maps; window pooling returns a sequence of features for another block, and global average pooling supplies sentence representations used for similarity features. The formal presentation pads sentences to a common maximum length, while diagrams also illustrate unequal sentence lengths. This is an order-sensitive sentence encoder rather than an unordered set representation.
Mechanism
#input-attention-variant
ABCNN-1 computes cross-sentence affinities between input feature columns, using the reported reciprocal-one-plus-Euclidean-distance match function. Learned projections of the affinity matrix and its transpose produce attention feature maps. Each is stacked with its sentence representation as an extra convolution input channel. Although the notation uses W0 and W1, footnote 1 says their weights are shared in the implementation.
Mechanism
#pool-attention-and-combined-variants
ABCNN-2 computes affinities after convolution, sums each row or column to obtain per-unit weights, and uses these in local weighted-sum pooling. The displayed formula includes no softmax or division by the sum of weights, despite the attention-based average-pooling terminology. Duplicating fixed counterpart feature columns can therefore scale the other sentence’s weights and pooled values. ABCNN-3 combines the input-attention and pooling-attention stages. The printed square s-by-s affinity shape in the ABCNN-2 paragraph does not match the expanded convolution lengths shown in Figure 3; this record follows the dimensions of the compared feature maps and width-w pooling, without asserting uninspected code conventions.
Scope
#staged-training-and-feature-heads
The reported experiments keep pretrained word embeddings fixed, train convolution-pooling blocks layerwise while freezing preceding blocks, and collect similarity features from multiple representation levels. Depending on the task, a separately trained linear SVM or logistic regression classifier uses these neural features together with specified linguistic features. This staged, feature-augmented protocol is not a claim that every reported result comes from one jointly trained neural head or from word embeddings alone.
Limitation
#task-specific-comparison-boundaries
WikiQA answer selection, MSRP paraphrase identification and SICK textual entailment have different splits, filters, metrics and feature sets. WikiQA keeps questions with a correct answer and truncates long answers; MSRP uses swapped-pair augmentation, and SICK uses original and non-overlapping sentence versions. The strongest tabulated variants use extra features, and corresponding feature-removal results are weaker. The reported MSRP improvement over its comparator is not statistically significant, and additional depth is not uniformly better on every metric. These experiments do not establish universal architectural superiority or unanswerable-question detection.
Limitation
#attention-and-scope-limits
The attention values express cross-sentence similarity and feature weighting, not a normalized one-to-one word alignment. Feature maps depend on the counterpart sentence, so the construction does not guarantee independently precomputable sentence embeddings. The selected entailment visualization illustrates learned responses but does not prove causal explanation or general coreference resolution. ABCNN-1, ABCNN-2 and ABCNN-3 are variants of this named sentence-pair architecture; similarly named visual-question-answering or later attention-CNN systems are not automatically the same concept.
⚠️ Curated Limitations & Negative Results
- This record denotes Yin and colleagues’ named sentence-pair ABCNN, not every attention-based CNN. Its three numbered variants are not interchangeable aliases.
- Printed attention-matrix shapes and pooling-index notation are interpreted using the declared feature dimensions and window width. No author implementation was inspected to settle edge conventions.
- Training stages, feature augmentation, task-specific splits and non-significant comparisons limit empirical conclusions; attention visualization is not causal validation.
- All fourteen pages of arXiv1512.05193v4 were visually read, including figures, tables, footnotes and references. No author code, training, benchmark replication or other source version was inspected. DRAFT remains.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ArtificialNeuralNetwork
Artificial Neural Network
|
broad_match | Shared learned convolutional sentence encoders and attention-dependent feature computations form a specific neural architecture; the target is broader. |
AIO:ConvolutionalLayer
Convolutional Layer
|
related_match | Learned local convolutional filters are components of the architecture, not an equivalent sentence-pair model. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
yin2018
|
Wenpeng Yin, Hinrich Schütze, Bing Xiang, Bowen Zhou
|
primary paper
full_text
|
ID: arXiv:1512.05193v4
sha256:46b9425267ae…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-13T11:11:55Z
Updated: 2026-10-13T11:14:51Z