SLEDGE (SciBERT Zero-Shot Domain-Transfer Scientific Literature Search)
Architectures AIMech:SLEDGESciBERTZeroShotScientificLiteratureSearch DRAFT
Aliases: SLEDGE, SLEDGE Scientific Knowledge Search, SLEDGE Zero-Shot SciBERT Scientific Literature Reranking Pipeline
A two-stage zero-shot domain-transfer scientific literature retrieval and neural re-ranking methodology (MacAvaney, Cohan, & Goharian, EMNLP 2020) that combines first-stage BM25 candidate generation over concatenated scientific article title, abstract, full-text paragraphs, and headings (with optional publication-date filtering and high-k_1 term-saturation tuning k_1 = 3.9, b = 0.55) with a second-stage SciBERT cross-encoder re-ranker trained in a zero-shot transfer regime on general-domain MS MARCO passage ranking pairs (or the MedSyn-filtered 78,895-query Med-MSMARCO medical subset in the fully automatic setting) to score natural-language topic questions against article titles and abstracts.
Identity Notes & Concept Scope
Distinct from general-domain BERT passage rerankers (Nogueira & Cho, 2019) and from CEDR (AIMech:CEDRContextualizedEmbeddingsForDocumentRanking). SLEDGE bridges emerging scientific domains lacking in-domain relevance judgments by pairing a scientific pre-trained language model (SciBERT) with out-of-domain relevance supervision transferred from MS MARCO (or lexicon-filtered Med-MSMARCO via the MedSyn layperson medical lexicon) and asymmetric field routing (searching full text with keyword queries or natural-language questions in Stage 1 while re-ranking title+abstract with natural-language questions in Stage 2).
Claims & Grounded Evidence
4 Claims
Mechanism
#sledge-two-stage-asymmetric-field-and-scibert-transfer
SLEDGE indexes concatenated scientific article titles, abstracts, full-text paragraphs, and headings (preferring cleaner PubMed XML over PDF text) to retrieve the top 500 candidates using BM25 (tuned via grid search to k_1 = 3.9, b = 0.55 with a >= Jan 1, 2020 date filter in run1), and re-ranks candidates using a Vanilla SciBERT cross-encoder trained on MS MARCO via pairwise cross-entropy L(q, d^+, d^-) = -log(R(q, d^+)) - log(1 - R(q, d^-)) (Equation 1) by scoring the topic's natural-language question field against each candidate's title and abstract.
Mechanism
#sledge-med-msmarco-lexicon-filtered-zero-shot-transfer
In the fully automatic zero-shot setting without human judgments or hyperparameter tuning (run2), SLEDGE mitigates domain shift by filtering the 808,531 MS MARCO training queries with the MedSyn medical terminology lexicon (excluding ambiguous common words such as gas, card, bing, died, map, and fall) to construct Med-MSMARCO (78,895 queries, 9.7% of MS MARCO) for training the SciBERT re-ranker.
Application
#sledge-trec-covid-leaderboard-performance
On Round 1 of the TREC-COVID shared task (51,045 CORD-19 articles, 56 participating teams), SLEDGE run1 tops the manual-run leaderboard with 0.6844 nDCG@10 and 0.6533 P@5 (fully relevant) (and reaches 0.7733 nDCG@10 when using the question field in both stages, Table 3), while automatic SLEDGE run2 achieves 0.6032 nDCG@10 and 0.5667 P@5 (fully relevant, Tables 1-2).
Limitation
#sledge-incidental-mention-and-term-count-limitation
Failure analysis on queries such as Query 23 (coronavirus hypertension) shows that Vanilla SciBERT can rank documents with incidental single mentions of a query term above documents with frequent term repetition that BM25 ranks highly, and applying a hard publication-date filter improves top-rank precision but caps total system recall by discarding pre-2020 coronavirus literature (Section 5.3).
⚠️ Curated Limitations & Negative Results
- Vanilla SciBERT re-ranking over title+abstract can over-score incidental single mentions of query terms compared to count-based models, and hard publication-date filtering caps recall of pre-2020 relevant articles.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ArtificialNeuralNetwork
Artificial Neural Network
|
broad_match | SLEDGE is a two-stage hybrid lexical and SciBERT neural network re-ranking architecture for zero-shot scientific literature retrieval. |
Evidence Sources & Primary Literature
3 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
r799-primary-pdf
|
Sean MacAvaney, Arman Cohan, Nazli Goharian
|
primary paper
full_text
|
ID: https://arxiv.org/bibtex/2005.02365
sha256:cebc7ef139ef…
|
r799-arxiv-abs
|
Sean MacAvaney, Arman Cohan, Nazli Goharian
|
primary paper
abstract_only
|
ID: https://arxiv.org/abs/2005.02365
sha256:54f5861f8980…
|
r799-metadata-cr
|
SLEDGE: A Simple Yet Effective Baseline for COVID-19 Scientific Knowledge Search (Crossref metadata)
Sean MacAvaney, Arman Cohan, Nazli Goharian
|
primary paper
abstract_only
|
ID: https://arxiv.org/bibtex/2005.02365
sha256:42f45bdd94a3…
|
Ontology & Grounding Topology
Open in interactive network graph →Curation Provenance
Curator: Antigravity
Method: ai_assisted
Model: Gemini
Created: 2026-10-19T14:42:00Z
Updated: 2026-10-19T14:42:00Z
Source File:
data/curated/SLEDGESciBERTZeroShotScientificLiteratureSearch.yaml
· Schema:
curated_concept.yaml
Explore across X-Mech
Related computational & biological Mechs:
Embedding Map
Network Graph
ProteinTraitsMech (ESM-2 pLM)
DUFMech (Domain pLM)
TraitMech
CultureMech
All 14 X-Mech Repositories