Home / Curated / Algorithms / DeepImpact Learning Passage Impacts for Inverted Indexes

DeepImpact Learning Passage Impacts for Inverted Indexes

Algorithms AIMech:DeepImpactLearningPassageImpactsForInvertedIndexes DRAFT

Aliases: DeepImpact, Learning Passage Impacts for Inverted Indexes, DeepImpact Passage Impact Model

A learned sparse impact-scoring and inverted-index retrieval method that expands each passage offline with DocT5Query predicted queries concatenated after a [SEP] token, encodes the expanded passage with BERT, takes the contextualized representation of the first occurrence of each unique term (merging WordPiece sub-tokens into space-separated words by averaging their contextual representations), passes each unique term embedding through a two-layer ReLU multi-layer perceptron trained end-to-end via pairwise softmax cross-entropy loss on the sum-of-impacts query-document score over intersecting terms, and linearly quantizes the resulting positive impact scores into b-bit integers (b = 8) stored directly in standard inverted-index postings for fast query-independent top-k retrieval.

Identity Notes & Concept Scope Distinct from DeepCT (AIMech:DeepCTContextAwareTermImportanceEstimationForFirstStageRetrieval), which trains per-token linear regression against static heuristic ground-truth term recall weights rather than end-to-end pairwise ranking over sum-of-impacts and requires BM25 scoring formulas at query time; and distinct from uniCOIL (AIMech:uniCOILUnifiedContextualizedInvertedList), which operates over WordPiece subword tokens and computes query-time contextualized query term weights rather than query-independent first-occurrence word-level impacts.

Claims & Grounded Evidence

4 Claims

Mechanism #deepimpact-first-occurrence-relu-mlp-and-sum-of-impacts-training

DeepImpact concatenates each passage d with DocT5Query expansion terms separated by a [SEP] token, feeds the sequence into BERT (averaging WordPiece sub-tokens for multi-piece words), feeds only the contextualized embedding of the first occurrence of each unique term t in d into a two-layer MLP with ReLU activations to produce a single non-negative scalar impact score s_{t,d}, defines the query-document score as the sum of impacts over intersecting query-document terms score(q, d) = sum_{t in q cap d} s_{t,d}, and optimizes the encoder and MLP end-to-end via pairwise softmax cross-entropy loss over (q, d+, d-) triples (Section 2, Figure 1).

supports Source: Learning Passage Impacts for Inverted Indexes · Locator: Section 2, Figure 1, pp. 2-3
supports Source: Learning Passage Impacts for Inverted Indexes · Locator: first_occurrence_positions(), compute_document_impacts(), sum_of_impacts_score(), and pairwise_softmax_ce_loss()

Mechanism #deepimpact-linear-quantization-into-inverted-index-postings

To support fast top-k query processing in standard inverted-index engines such as PISA using MaxScore without runtime BM25/TF-IDF computation, DeepImpact linearly scales positive real-valued impact scores (stored at 3 digits of precision) across the collection into b-bit integers in [1, 2^b - 1] (with b = 8 yielding [1, 255]) and stores each quantized integer directly in the frequency payload field of the inverted list alongside the document ID (Section 2).

supports Source: Learning Passage Impacts for Inverted Indexes · Locator: Section 2 (Indexing and Quantization & Query Processing), pp. 2-3
supports Source: Learning Passage Impacts for Inverted Indexes · Locator: quantize_impacts_linear() and main() verification

Application #deepimpact-msmarco-and-trec-dl-effectiveness-and-speedup-application

On MS MARCO Dev and TREC DL 2019/2020, DeepImpact achieves 0.326 MRR@10 on MS MARCO Dev (up to +17.7% over DocT5Query at 0.277 and +34.2% over DeepCT at 0.243) and, when used as a first-stage retriever feeding a ColBERT re-ranker over k = 100 candidates, matches the end-to-end effectiveness of re-ranking k = 1000 BM25 candidates while achieving up to a 5.1x query latency speedup (Section 3, Tables 1-2).

supports Source: Learning Passage Impacts for Inverted Indexes · Locator: Section 3, Tables 1-2, pp. 3-4

Limitation #deepimpact-offline-doct5query-expansion-dependency-limitation

Because DeepImpact only estimates impact scores for terms physically present in its input token sequence, bridging vocabulary mismatch depends on first running an external generative document-expansion model (DocT5Query) across the entire corpus prior to BERT impact scoring and inverted-index construction, and passages remain truncated at 300 tokens (Section 2).

supports Source: Learning Passage Impacts for Inverted Indexes · Locator: Section 2 (Eliminating the Vocabulary Mismatch & Implementation Details), pp. 2-3

⚠️ Curated Limitations & Negative Results

Vocabulary-mismatch mitigation relies on pre-expanding every passage in the corpus with an external generative sequence-to-sequence model (DocT5Query) before computing BERT passage impacts.
Input sequences (original passage plus expansion queries) are truncated at a maximum length of 300 tokens during training and inference.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:ArtificialNeuralNetwork` Artificial Neural Network	related_match	DeepImpact fine-tunes a contextualized BERT encoder coupled with a two-layer ReLU multi-layer perceptron to predict scalar passage-term impact scores.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r776-primary-pdf`	Learning Passage Impacts for Inverted Indexes Antonio Mallia, Omar Khattab, Torsten Suel, Nicola Tonellotto	primary paper full_text	ID: https://arxiv.org/bibtex/2104.12016 sha256:bb75e79dae3a…
`r776-arxiv-abs`	Learning Passage Impacts for Inverted Indexes (arXiv abstract) Antonio Mallia, Omar Khattab, Torsten Suel, Nicola Tonellotto	primary paper abstract_only	ID: https://arxiv.org/abs/2104.12016 sha256:ae21315454b8…
`r776-metadata-cr`	Learning Passage Impacts for Inverted Indexes (Crossref metadata) Antonio Mallia, Omar Khattab, Torsten Suel, Nicola Tonellotto	primary paper abstract_only	ID: https://arxiv.org/bibtex/2104.12016 sha256:d3b8bec5869b…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T14:19:00Z

Updated: 2026-10-19T14:19:00Z

Source File: data/curated/DeepImpactLearningPassageImpactsForInvertedIndexes.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories