Home / Curated / Architectures / Attentive Pooling Network

Attentive Pooling Network

Architectures AIMech:AttentivePoolingNetwork DRAFT

Aliases: Attentive Pooling Networks

A neural sentence-pair architecture that learns bilinear segment interactions, derives attention by row and column maxima followed by separate softmax normalization, and compares the resulting pair-conditioned pooled representations for answer selection.

Identity Notes & Concept Scope Compared all 442 pinned AIO and 606 native IDs, names, aliases and scoped synonyms; no normalized identity match. Ignored- and hidden-inclusive fixed-string searches of the recovery, older discovery and original Documents checkouts returned 8, 0, 0 lines. All distinct match contents were inspected. Semantic matching covered 104 AIO and 64 native definitions, with every matched definition read. 7 full annotated AIO node/edge/generated contexts and 3 full native records were inspected. This is the named 2016 sentence-pair attentive pooling architecture, not every later method called attention pooling. AP-CNN and AP-biLSTM are instantiations rather than equivalent aliases of each other. The bilinear interaction/max/softmax construction differs from memory-key addressing, recurrent episodes and PointNet independent max pooling.

Claims & Grounded Evidence

6 Claims

Mechanism #pair-conditioned-alignment

For encoded question and answer segments Q and A, compute the interaction matrix G=tanh(Q-transpose*U*A), with a learned bilinear matrix U. Row maxima over answer segments and column maxima over question segments yield two score vectors, which are separately softmax-normalized. Each input is then pooled as a weighted sum of its own segment vectors. These representations depend on the current pair; the intermediate signed interaction scores are not probabilities or a one-to-one alignment.

supports Source: Attentive Pooling Networks · Locator: Section 3, PDF pp. 4–5, Equations 13–18 and Figure 2

Scope #notation-and-reduction-boundaries

With Q having M columns and A having L columns, G has M rows and L columns. The intended row/column reductions follow these dimensions and Figure 2; the printed bounds in Equations 14–15 swap M and L and use strict endpoint inequalities. This record uses the dimensionally consistent interpretation rather than asserting those literal bounds. Max reductions are piecewise smooth, and tie-gradient conventions are not established by an inspected implementation. For the displayed unscaled tanh scores, any two resulting softmax weights have ratio at most exp(2), a direct formula consequence rather than a learned-performance guarantee.

supports Source: Attentive Pooling Networks · Locator: Section 3, PDF p. 4, Equations 13–17; Figure 2, p. 5; constructed mathematical checks

Scope #encoder-and-ranking-variants

AP-CNN and AP-biLSTM are distinct instantiations using convolutional or bidirectional recurrent segment encoders. The question and answer encoder parameters are shared within each model. Cosine similarity compares the pair-conditioned pooled vectors, and a margin ranking loss contrasts positive and selected negative answers; the described training samples fifty negative answers and uses the highest-scoring one. The architecture ranks supplied candidates rather than generating answers. A symmetric final cosine does not by itself make the complete pair-conditioned scorer symmetric when the learned interaction matrix is unconstrained.

supports Source: Attentive Pooling Networks · Locator: Sections 2.1–2.3, PDF pp. 2–3; Section 3, pp. 4–5; constructed asymmetric-matrix example

Limitation #fixed-representation-pooling-limits

Permuting already encoded segment columns consistently leaves the corresponding pooled vectors unchanged, but this does not establish invariance to reordering the original words: convolutional and recurrent encoders can change their outputs. Duplicating a segment can change its side of the weighted read even when the relevant maxima are unchanged. Pair-conditioned attention is not ordinary independent global max pooling or a guarantee that candidate embeddings can be precomputed independently of their question.

supports Source: Attentive Pooling Networks · Locator: Sections 2–3, PDF pp. 2–5, Equations 13–18; direct constructed examples

Limitation #filtered-answer-selection-evaluation

InsuranceQA uses candidate answer pools and top-answer accuracy, while TREC-QA and WikiQA report MAP and MRR under specified filtering and splits. TREC excludes questions without both correct and incorrect answers, and WikiQA retains questions with a correct answer; these experiments do not establish unanswerable-question detection. Architecture configurations and tuning differ, including many fewer filters in AP-CNN than in the QA-CNN baseline. Reported runtime belongs to the stated Theano/GPU setup, not a universal efficiency guarantee.

supports Source: Attentive Pooling Networks · Locator: Sections 5–6, PDF pp. 5–8, Tables 1–5

Limitation #comparison-and-explanation-boundaries

The performance comparisons do not establish one best model across all tasks and feature regimes. AP-biLSTM has higher InsuranceQA test accuracy than AP-CNN, while the WikiQA footnote reports a hand-feature comparator exceeding AP-CNN despite its advantage over the word-embedding-only comparison. Length plots are cumulative accuracy for answers up to a threshold, not controlled length interventions. The two attention heatmaps show selected correctly answered examples and do not prove faithful causal explanations.

supports Source: Attentive Pooling Networks · Locator: Section 6, PDF pp. 6–9, Tables 3–5, footnote 6 and Figures 3–6

⚠️ Curated Limitations & Negative Results

The named 2016 sentence-pair construction is narrower than generic attention pooling; AP-CNN and AP-biLSTM are variants rather than interchangeable aliases.
Printed reduction bounds require an explicitly labelled dimensional interpretation. Empty sequences, zero-norm cosine inputs and gradient choices at max ties were not established from author code.
Invariance of pooling over fixed encoded vectors does not imply word-order invariance, and attention heatmaps do not certify causal explanations.
All ten pages of arXiv1602.03609v1 were visually read. Other versions, author code and training were not inspected; no benchmark or hardware result was replicated. DRAFT remains.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:ArtificialNeuralNetwork` Artificial Neural Network	broad_match	Learned neural encoders and a trained bilinear interaction/pooling scorer define a particular neural architecture; the target covers both convolutional and recurrent instantiations.

Evidence Sources & Primary Literature

1 Source

Source ID	Publication / Document	Type / Access	Reference & Checksum
`santos2016`	Attentive Pooling Networks Cicero dos Santos, Ming Tan, Bing Xiang, Bowen Zhou	primary paper full_text	ID: arXiv:1602.03609v1 sha256:16f2cd4db2b0…

Curation Provenance

Curator: Codex

Method: ai_assisted

Model: GPT-6

Created: 2026-10-13T10:45:21Z

Updated: 2026-10-13T10:53:01Z

Source File: data/curated/AttentivePoolingNetwork.yaml