Home / Curated / Architectures / SPLADE (Sparse Lexical and Expansion Model for First Stage Ranking)

SPLADE (Sparse Lexical and Expansion Model for First Stage Ranking)

Architectures AIMech:SPLADESparseLexicalAndExpansionModel DRAFT

Aliases: SParse Lexical AnD Expansion Model, SPLADE First-Stage Sparse Retriever, SPLADE-FLOPS Learned Sparse Retrieval

A learned sparse neural information retrieval architecture for first-stage inverted-index ranking that maps an input query or document token sequence t = (t_1, ..., t_N) into a |V|-dimensional WordPiece vocabulary vector by projecting contextualized BERT hidden states h_i through the Masked Language Modeling (MLM) prediction head w_{ij} = transform(h_i)^T E_j + b_j, aggregating non-negative term importance with logarithmic saturation w_j = sum_{i in t} log(1 + ReLU(w_{ij})), and jointly optimizing in-batch negative ranking loss with asymmetric query and document sparsity regularizers (lambda_q L_reg^q + lambda_d L_reg^d, using l_1 or the batch-squared activation regularizer l_FLOPS).

Identity Notes & Concept Scope Introduced by Formal, Piwowarski, and Clinchant (SIGIR 2021, pp. 2288-2292, arXiv:2107.05720). Distinguished from DeepCT (Dai & Callan, 2019), which re-weights only existing document terms without vocabulary expansion, from doc2query/docTTTTTquery (Nogueira et al., 2019), which perform offline sequence-to-sequence text expansion before BM25 indexing, and from SparTerm (Bai et al., 2020), which uses un-saturated ReLU summation with a fixed pre-trained binary gating mask rather than end-to-end log-saturated FLOPS-regularized training.

Claims & Grounded Evidence

3 Claims

Mechanism #splade-log-saturation-and-flops-regularization-mechanism

For an input sequence t = (t_1, ..., t_N) with MLM token-to-vocabulary logits w_{ij} = transform(h_i)^T E_j + b_j (j in {1, ..., |V|}), SPLADE computes vocabulary weights w_j = sum_{i in t} log(1 + ReLU(w_{ij}))—preventing bursty terms from dominating and inducing natural sparsity without fixed binary gates—and jointly minimizes L = L_{rank-IBN} + lambda_q L_reg^q + lambda_d L_reg^d, where the FLOPS regularizer l_FLOPS = sum_{j in V} ((1/N) sum_{i=1}^N w_j^{(d_i)})^2 penalizes high batch-mean term activations to produce evenly distributed posting lists.

supports Source: SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking · Locator: Section 3.1-3.2 (Equations 1, 4, 5, 6), pp. 2-3
supports Source: SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking · Locator: splade_weights, flops_reg, l1_reg & main

Scope #splade-inverted-index-efficiency-and-compression-expansion-balance

Trained end-to-end in a single stage on MS MARCO passage ranking, SPLADE-l_FLOPS achieves 0.322 MRR@10 and 0.955 Recall@1000 on MS MARCO Dev (and 0.665 nDCG@10 on TREC DL 2019) at 0.73 expected FLOPS—a 3.9x FLOPS reduction compared to un-saturated SparTerm-exp-l_FLOPS (2.83 FLOPS, 0.312 MRR@10)—simultaneously dropping an average of 20 uninformative original terms while adding 32 latent expansion terms per document.

supports Source: SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking · Locator: Section 4 (Table 1, Figure 1, & Table 2), pp. 3-4

Limitation #splade-regularization-schedule-and-flops-tradeoff-limitation

SPLADE requires careful tuning and quadratic warmup scheduling (e.g., over 50k steps) of two separate regularization hyperparameters (lambda_q and lambda_d), as aggressive sparsity regularization to match pure bag-of-words retrieval cost (e.g., FLOPS = 0.05) reduces MS MARCO Dev MRR@10 from 0.322 to 0.296.

supports Source: SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking · Locator: Section 4 (Training, indexing and retrieval & Effectiveness-efficiency trade-off, Figure 1), pp. 3-4

⚠️ Curated Limitations & Negative Results

Requires separate tuning and quadratic warmup scheduling of query and document regularization weights (lambda_q, lambda_d) to avoid early representation collapse.
Strong sparsity regularization (e.g., FLOPS = 0.05, 6 non-zero query terms and 18 document terms) incurs a measurable effectiveness drop (MRR@10 from 0.322 to 0.296).

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:ArtificialNeuralNetwork` Artificial Neural Network	related_match	Uses a pre-trained neural transformer MLM head with log-saturated ReLU pooling to learn sparse high-dimensional representations.
`AIO:SparseLearning` Sparse Learning	related_match	Enforces explicit representation sparsity in the \|V\|-dimensional vocabulary space via L1 and FLOPS regularization.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r749-primary-pdf`	SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking Thibault Formal, Benjamin Piwowarski, Stéphane Clinchant	primary paper full_text	ID: https://dblp.org/search/publ/api?q=SPLADE+Sparse+Lexical+and+Expansion+Model+Formal&h=5&format=xml sha256:e791451960e4…
`r749-arxiv-abs`	SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking (arXiv abstract) Thibault Formal, Benjamin Piwowarski, Stéphane Clinchant	primary paper abstract_only	ID: https://arxiv.org/abs/2107.05720 sha256:85089c31d19e…
`r749-metadata-cr`	SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking (Crossref metadata) Thibault Formal, Benjamin Piwowarski, Stéphane Clinchant	primary paper abstract_only	ID: https://dblp.org/search/publ/api?q=SPLADE+Sparse+Lexical+and+Expansion+Model+Formal&h=5&format=xml sha256:405af6287b49…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T13:52:00Z

Updated: 2026-10-19T13:52:00Z

Source File: data/curated/SPLADESparseLexicalAndExpansionModel.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories