PAIR Passage-Centric Similarity Relation for Dense Passage Retrieval
Algorithms AIMech:PAIRPassageCentricSimilarityRelationForDenseRetrieval DRAFT
Aliases: PAIR, PAIR Passage-Centric Dense Passage Retrieval, Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval
A dense passage retrieval training algorithm that jointly enforces query-centric similarity relations (pulling query q toward positive passage p^+ and pushing q away from negative passages p^- in N_P via L_Q(q, p^+, N_P) = -log(exp(f(q, p^+)) / (exp(f(q, p^+)) + sum_{p^- in N_P} exp(f(q, p^-))))) and passage-centric similarity relations (pushing positive passage p^+ away from negative passages p^- in N_P via L_P(q, p^+, N_P) = -log(exp(-f(p^+, p^+)) / (exp(-f(p^+, p^+)) + sum_{p^- in N_P} exp(f(p^+, p^-))))) under a combined objective L(q, p^+, N_P) = (1 - alpha) L_Q(q, p^+, N_P) + alpha L_P(q, p^+, N_P) trained in two stages: a pre-training stage on denoised synthetic pseudo-queries with decaying weight alpha > 0 followed by a fine-tuning stage on human-labeled queries with alpha = 0.
Identity Notes & Concept Scope
Distinct from DPR (AIMech:DensePassageRetrieval) and RocketQA (AIMech:RocketQADenoisedHardNegativeSampling), which optimize only query-passage similarity f(q, p) without explicitly penalizing passage-passage inner products f(p^+, p^-); and distinct from GPL (AIMech:GPLGenerativePseudoLabelingForDenseRetrieval), which distills cross-encoder query-passage margins on synthetic queries rather than optimizing a passage-centric contrastive repulsion loss L_P.
Claims & Grounded Evidence
4 Claims
Mechanism
#pair-passage-centric-contrastive-loss
Under the geometric separation condition (Assumption 1: max(||q - p^+||, ||p^+ - p^-||) < ||q - p^-||), query-centric similarity alone cannot prevent hard negative passages p^- from clustering near p^+; PAIR introduces the passage-centric contrastive loss L_P(q, p^+, N_P) = -log(exp(-f(p^+, p^+)) / (exp(-f(p^+, p^+)) + sum_{p^- in N_P} exp(f(p^+, p^-)))) (Equation 2) and combines it with L_Q via L = (1 - alpha) L_Q + alpha L_P (Equation 3) to explicitly minimize positive-negative passage inner products f(p^+, p^-).
Mechanism
#pair-two-stage-pseudo-query-training
To overcome data sparsity when estimating both query-centric and passage-centric similarity relations, PAIR generates synthetic queries from passages via a T5/BART generator, filters out noisy pseudo-labels and false negatives with a cross-encoder, pre-trains the dual-encoder with dynamic decay of alpha from alpha_0 > 0, and then fine-tunes on labeled target queries with alpha = 0 (Section 2.4).
Application
#pair-benchmark-applications
PAIR achieves 37.9 MRR@10 and 86.4% Recall@50 on MS MARCO Passage Ranking Dev and 74.9% Recall@5 / 83.5% Recall@20 on Natural Questions test, outperforming RocketQA and prior dense retrievers while reducing positive-negative passage cosine similarity (Tables 1, 3, and 4).
Limitation
#pair-unit-norm-and-pseudo-query-limitation
When passage embeddings are not L2-normalized, the exp(-f(p^+, p^+)) term in L_P can encourage growing the norm ||p^+|| unless unit-norm constraints or careful decay of alpha are applied, and the two-stage pipeline requires generating and cross-encoder-filtering millions of synthetic queries.
⚠️ Curated Limitations & Negative Results
- Requires generating millions of synthetic pseudo-queries and scoring them with a cross-encoder to provide sufficient training coverage for passage-centric similarity estimation.
- Sensitive to the weighting hyperparameter alpha and its decay schedule between Stage 1 pre-training and Stage 2 fine-tuning.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ArtificialNeuralNetwork
Artificial Neural Network
|
related_match | PAIR trains a dual-encoder Transformer dense retriever using a combined query-centric and passage-centric contrastive loss over neural embeddings. |
Evidence Sources & Primary Literature
3 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
r759-primary-pdf
|
Ruiyang Ren, Shangwen Lv, Yingqi Qu, Jing Liu, Wayne Xin Zhao, Qiaoqiao She, Hua Wu, Haifeng Wang, Ji-Rong Wen
|
primary paper
full_text
|
ID: https://arxiv.org/bibtex/2108.06027
sha256:2919e5f004f6…
|
r759-arxiv-abs
|
Ruiyang Ren, Shangwen Lv, Yingqi Qu, Jing Liu, Wayne Xin Zhao, Qiaoqiao She, Hua Wu, Haifeng Wang, Ji-Rong Wen
|
primary paper
abstract_only
|
ID: https://arxiv.org/abs/2108.06027
sha256:24813ef24115…
|
r759-metadata-cr
|
Ruiyang Ren, Shangwen Lv, Yingqi Qu, Jing Liu, Wayne Xin Zhao, Qiaoqiao She, Hua Wu, Haifeng Wang, Ji-Rong Wen
|
primary paper
abstract_only
|
ID: https://arxiv.org/bibtex/2108.06027
sha256:b100366c883b…
|
Ontology & Grounding Topology
Open in interactive network graph →Curation Provenance
Curator: Antigravity
Method: ai_assisted
Model: Gemini
Created: 2026-10-19T14:02:00Z
Updated: 2026-10-19T14:02:00Z
Source File:
data/curated/PAIRPassageCentricSimilarityRelationForDenseRetrieval.yaml
· Schema:
curated_concept.yaml
Explore across X-Mech
Related computational & biological Mechs:
Embedding Map
Network Graph
ProteinTraitsMech (ESM-2 pLM)
DUFMech (Domain pLM)
TraitMech
CultureMech
All 14 X-Mech Repositories