Home / Curated / Architectures / EPIC Expansion via Prediction of Importance with Contextualization

EPIC Expansion via Prediction of Importance with Contextualization

Architectures AIMech:EPICExpansionViaPredictionOfImportanceWithContextualization DRAFT

Aliases: EPIC, Expansion via Prediction of Importance with Contextualization, EPIC Lexicon-Grounded Neural Ranking Model

A lexicon-grounded representation-based neural passage re-ranking architecture that maps a query q into a sparse |V|-dimensional vector phi_q where each present query term t_i receives contextualized log-softplus importance weight w_q(t_i) = ln(1 + softplus(theta_1^T f_i(q))) (summed across repeated occurrences), and maps a passage d offline at index time into a |V|-dimensional lexicon vector phi_d(tau) = c(d) * max_{t_j in d}(w_d(t_j) * psi_j(tau)) combining a projection psi_j = Theta_2 f_j(d) into the vocabulary space, token importance w_d(t_j) = ln(1 + softplus(theta_3^T f_j(d))), and a sigmoid [CLS] passage-quality gate c(d) = sigmoid(theta_4^T d_[CLS]), scoring relevance via the inner product sim(q, d) = phi_q^T phi_d over query terms with optional top-r document vector pruning.

Identity Notes & Concept Scope Distinct from SPLADE (AIMech:SPLADESparseLexicalAndExpansionModel), which applies log(1 + ReLU(.)) directly to MLM logits with explicit FLOPS regularization for first-stage inverted-index retrieval without separate token-salience factors w_d(t_j) or a sigmoid [CLS] document-quality gate c(d); and distinct from SPARTA (AIMech:SPARTASparseTransformerMatchingForRetrieval), which computes non-contextualized query-to-contextualized-document token max dot products followed by ReLU(b + .) and log(1 + .) without query contextualization or a learned vocabulary expansion projection matrix Theta_2.

Claims & Grounded Evidence

4 Claims

Mechanism #epic-log-softplus-term-importance-and-cls-gated-lexicon-expansion

EPIC encodes an n-term query q with BERT into contextualized vectors f_i(q) in R^e (e = 768) and forms a sparse vector phi_q in R^{|V|} (|V| = 30,522) assigning each query term t_i importance w_q(t_i) = ln(1 + softplus(theta_1^T f_i(q))) summed across duplicates (Equation 1), while encoding an m-term passage d at index time via lexicon projection psi_j = Theta_2 f_j(d) in R^{|V|} (with Theta_2 initialized from BERT's pre-trained MLM prediction matrix), token importance w_d(t_j) = ln(1 + softplus(theta_3^T f_j(d))) (Equation 2), overall passage quality c(d) = sigmoid(theta_4^T d_[CLS]) (Equation 3), and max-pooling phi_d(tau) = c(d) * max_{t_j in d}(w_d(t_j) * psi_j(tau)) (Equation 4), scoring pairs by sim(q, d) = sum_{tau in V} phi_q(tau) * phi_d(tau) (Equation 5).

supports Source: Expansion via Prediction of Importance with Contextualization · Locator: Section 2 & Section 3 (Training), Figure 1 & Equations 1-5, pp. 1-3
supports Source: Expansion via Prediction of Importance with Contextualization · Locator: term_importance(), build_query_vector(), build_doc_vector(), and epic_similarity()

Mechanism #epic-top-r-document-representation-pruning

Because approximately 74% of entries in the dense |V|-dimensional passage vector phi_d have values of 0.1 or below (Figure 2), EPIC prunes each pre-computed passage vector by retaining only its top r largest values (e.g., r = 2000 reducing the 30,522-term vocabulary by 93.4%, or r = 1000 reducing it by 96.7%), cutting per-passage storage from ~60 KiB (500 GiB for MS MARCO in fp16) to 3.9 KiB (34 GiB total) with no statistically significant difference in MRR@10 (Section 3, Table 1 & pp. 3-4).

supports Source: Expansion via Prediction of Importance with Contextualization · Locator: Section 3 (Effect of document representation pruning & Cost of pre-computing), Table 1 & Figure 2, pp. 3-4
supports Source: Expansion via Prediction of Importance with Contextualization · Locator: prune_top_r() and main() verification of pruned scoring preservation

Application #epic-msmarco-and-treccar-additive-reranking-application

On MS MARCO passage ranking Dev, re-ranking BM25 with EPIC raises MRR@10 from 0.198 to 0.273 (48ms latency with r = 1000 pruning), and re-ranking docTTTTTquery with EPIC achieves an additive gain to 0.304 MRR@10 at 78ms (0.303 MRR@10 at 68ms—only 5ms overhead over docTTTTTquery—when pruned at r = 1000, Table 1), while on TREC CAR 2017 EPIC increases BM25 MRR from 0.235 to 0.353, outperforming DeepCT at 0.332 (Section 3, p. 3).

supports Source: Expansion via Prediction of Importance with Contextualization · Locator: Section 3 (Ranking effectiveness & Ranking efficiency), Table 1, pp. 3-4

Limitation #epic-dense-vector-storage-and-aggressive-pruning-drop-limitation

Unpruned EPIC passage representations are dense |V|-dimensional vectors requiring ~60 KiB per passage (500 GiB across MS MARCO) and must be loaded from disk for candidate re-ranking rather than queried directly as a sparse inverted index, while aggressive pruning down to r = 100 terms causes a sharp drop in MRR@10 (from 0.273 to 0.241 on BM25 and from 0.304 to 0.285 on docTTTTTquery, Section 3, p. 4).

supports Source: Expansion via Prediction of Importance with Contextualization · Locator: Section 3 (Effect of document representation pruning & Cost of pre-computing), pp. 3-4

⚠️ Curated Limitations & Negative Results

Unpruned passage vectors are dense across the 30,522-term vocabulary (~500 GiB for MS MARCO in 16-bit floats), requiring top-r pruning for practical disk storage and re-ranking.
Pruning passage representations too aggressively (e.g., r = 100 terms per passage) substantially degrades MRR@10 effectiveness.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:ArtificialNeuralNetwork` Artificial Neural Network	related_match	EPIC fine-tunes a contextualized BERT-base neural network with learned linear salience heads, a vocabulary projection matrix initialized from the MLM head, and a sigmoid [CLS] quality gate.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r779-primary-pdf`	Expansion via Prediction of Importance with Contextualization Sean MacAvaney, Franco Maria Nardini, Raffaele Perego, Nicola Tonellotto, Nazli Goharian, Ophir Frieder	primary paper full_text	ID: https://arxiv.org/bibtex/2004.14245 sha256:1a1438fc18a1…
`r779-arxiv-abs`	Expansion via Prediction of Importance with Contextualization (arXiv abstract) Sean MacAvaney, Franco Maria Nardini, Raffaele Perego, Nicola Tonellotto, Nazli Goharian, Ophir Frieder	primary paper abstract_only	ID: https://arxiv.org/abs/2004.14245 sha256:9c827eab0448…
`r779-metadata-cr`	Expansion via Prediction of Importance with Contextualization (Crossref metadata) Sean MacAvaney, Franco Maria Nardini, Raffaele Perego, Nicola Tonellotto, Nazli Goharian, Ophir Frieder	primary paper abstract_only	ID: https://arxiv.org/bibtex/2004.14245 sha256:4420c142ee2a…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T14:22:00Z

Updated: 2026-10-19T14:22:00Z

Source File: data/curated/EPICExpansionViaPredictionOfImportanceWithContextualization.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories