Promptagator Few-Shot Dense Retrieval
Algorithms AIMech:PromptagatorFewShotDenseRetrievalDataGeneration DRAFT
Aliases: Promptagator, Prompt-based Query Generation for Retriever, Promptagator Few-Shot Dense Retrieval and Round-Trip Filtering
A few-shot dense retrieval training method (Prompt-based Query Generation for Retriever) that prompts a large language model with task-specific document and query role descriptions alongside a small set (2 to 8) of in-domain relevant query-document pairs to generate synthetic queries across a target document corpus, trains an initial dual-encoder retriever on the noisy synthetic pairs to apply round-trip consistency filtering—retaining a synthetic pair only if its source passage is ranked in the top-K retrieved passages—and fine-tunes a compact task-specific dual encoder and optional cross-attention reranker (Promptagator++) on the filtered data.
Identity Notes & Concept Scope
Distinct from InPars (AIMech:InParsInContextDataAugmentationForInformationRetrieval), which filters synthetic queries by length-normalized generation log-probability and trains only a second-stage cross-encoder reranker over BM25 candidates, whereas Promptagator conditions on task-specific search-intent role prefixes (e.g., counter-argument vs. fact-check claim), performs round-trip retrieval consistency filtering using a self-trained initial dual encoder without external QA models, and produces a standalone first-stage dense dual encoder. Also distinct from zero-shot hypothetical document embedding at query time (AIMech:HyDEHypotheticalDocumentEmbeddings), which calls an LLM online for every test query rather than distilling synthetic queries offline into a fast 110M dual encoder.
Claims & Grounded Evidence
4 Claims
Mechanism
#promptagator-intent-conditioned-query-generation
Promptagator constructs a prompt of the form (e_doc(d_1), e_query(q_1), ..., e_doc(d_k), e_query(q_k), e_doc(d)) using task-specific document and query role prefixes and at most k=8 examples to sample synthetic queries q_hat from a large language model across target corpus passages d in D_T, capturing task-specific search intents such as counter-argument retrieval or claim verification.
Mechanism
#promptagator-self-supervised-round-trip-consistency-filter
Without relying on an external supervised question-answering model, Promptagator trains an initial dual-encoder retriever on the raw synthetic (q, d) pairs and filters the synthetic dataset by keeping (q, d) only when the initial retriever ranks the source passage d within the top-K (with K=1) passages for q.
Application
#promptagator-beir-dual-encoder-and-reranker-application
Evaluated across 11 heterogeneous retrieval datasets in BEIR, a 110M-parameter Promptagator dual encoder trained from at most 8 examples without MS MARCO supervision outperforms MS MARCO-supervised ColBERT v2 and SPLADE v2 by over 1.2 nDCG@10 on average, and adding a 110M Promptagator++ cross-attention reranker yields an additional 5.0 nDCG@10 gain.
Limitation
#promptagator-small-corpus-filtering-overfitting-limitation
On very small target corpora (such as NFCorpus and SciFact), aggressive top-1 round-trip consistency filtering can discard too many synthetic pairs and cause subsequent dual-encoder fine-tuning to overfit, slightly reducing retrieval accuracy.
⚠️ Curated Limitations & Negative Results
- Offline synthetic dataset generation across up to one million documents per corpus with a 137B-parameter language model incurs high upfront compute cost.
- Sensitive to noisy or misannotated examples in the few-shot prompt (such as 'not enough info' labels in Climate-FEVER).
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ArtificialNeuralNetwork
Artificial Neural Network
|
related_match | Promptagator distills a prompted large language model into a neural dual-encoder retriever and cross-attention reranker via round-trip filtered synthetic queries. |
Evidence Sources & Primary Literature
3 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
r740-primary-pdf
|
Zhuyun Dai, Vincent Y. Zhao, Ji Ma, Yi Luan, Jianmo Ni, Jing Lu, Anton Bakalov, Kelvin Guu, Keith B. Hall, Ming-Wei Chang
|
primary paper
full_text
|
ID: https://dblp.org/search/publ/api?q=Promptagator+Few-shot+Dense+Retrieval+From+8+Examples+Dai&h=5&format=xml
sha256:c2dec45d0b2d…
|
r740-arxiv-abs
|
Zhuyun Dai, Vincent Y. Zhao, Ji Ma, Yi Luan, Jianmo Ni, Jing Lu, Anton Bakalov, Kelvin Guu, Keith B. Hall, Ming-Wei Chang
|
primary paper
abstract_only
|
ID: https://arxiv.org/abs/2209.11755
sha256:ba6931e0fa26…
|
r740-metadata-cr
|
Zhuyun Dai, Vincent Y. Zhao, Ji Ma, Yi Luan, Jianmo Ni, Jing Lu, Anton Bakalov, Kelvin Guu, Keith B. Hall, Ming-Wei Chang
|
primary paper
abstract_only
|
ID: https://dblp.org/search/publ/api?q=Promptagator+Few-shot+Dense+Retrieval+From+8+Examples+Dai&h=5&format=xml
sha256:128123e6b128…
|
Ontology & Grounding Topology
Open in interactive network graph →Curation Provenance
Curator: Antigravity
Method: ai_assisted
Model: Gemini
Created: 2026-10-19T13:43:00Z
Updated: 2026-10-19T13:43:00Z
Source File:
data/curated/PromptagatorFewShotDenseRetrievalDataGeneration.yaml
· Schema:
curated_concept.yaml
Explore across X-Mech
Related computational & biological Mechs:
Embedding Map
Network Graph
ProteinTraitsMech (ESM-2 pLM)
DUFMech (Domain pLM)
TraitMech
CultureMech
All 14 X-Mech Repositories