Home / Curated / Algorithms / EPR Efficient Prompt Retrieval for In-Context Learning

EPR Efficient Prompt Retrieval for In-Context Learning

Algorithms AIMech:EPREfficientPromptRetrievalForInContextLearning DRAFT

Aliases: EPR, Efficient Prompt Retrieval, LM-Scored Contrastive Dense Prompt Retrieval

A supervised dense prompt retrieval method for in-context learning that first filters candidate training demonstrations for each training example using an unsupervised retriever, scores each candidate by the conditional probability of the gold target sequence under a scoring language model conditioned on that candidate demonstration and the input, labels the highest- and lowest-scoring candidates as positive and hard-negative examples, and trains a dual-encoder bi-encoder via contrastive InfoNCE loss for fast maximum inner-product search at inference time.

Identity Notes & Concept Scope Distinct from unsupervised or transfer sentence-embedding nearest-neighbor retrieval (AIMech:KATEkNNAugmentedInContextExampleSelection), which does not use language-model conditional decoding probabilities to supervise the retriever. Also distinct from determinantal point process subset selection (CEIL), which models pairwise diversity and joint subset interactions across multiple exemplars rather than scoring individual prompt candidates independently.

Claims & Grounded Evidence

4 Claims

Mechanism #epr-scoring-lm-candidate-labeling-mechanism

For each training pair (x, y), EPR retrieves a candidate pool E_bar of L examples using an unsupervised retriever on y, evaluates each candidate e_l independently using a scoring language model g_hat via s(e_l) = Prob_{g_hat}(y | e_l, x), and designates the top-k candidates as positive prompts E_pos and the bottom-k candidates as hard-negative prompts E_neg.

supports Source: Learning To Retrieve Prompts for In-Context Learning · Locator: Section 3.1, p. 3, Figure 2

Mechanism #epr-contrastive-biencoder-mips-mechanism

EPR trains an input encoder E_X(x) and a prompt encoder E_P(e) with inner-product similarity sim(x, e) = E_X(x)^T E_P(e) using an in-batch contrastive negative log-likelihood objective over positive and hard-negative examples, enabling sub-linear test-time prompt retrieval via FAISS maximum inner-product search.

supports Source: Learning To Retrieve Prompts for In-Context Learning · Locator: Section 3.2, p. 4, Equation 1

Application #epr-semantic-parsing-and-lm-proxy-application

EPR is applied to few-shot semantic parsing and meaning-representation generation on BREAK, MTOP, and SMCalFlow, where a retriever trained using a smaller proxy scoring language model (GPT-Neo 2.7B) transfers effectively to larger inference models including GPT-J, Codex, and GPT-3.

supports Source: Learning To Retrieve Prompts for In-Context Learning · Locator: Section 4.1-4.4, pp. 4-7, Tables 2-5
supports Source: Learning To Retrieve Prompts for In-Context Learning (Crossref metadata) · Locator: DBLP bibliographic record conf/naacl/RubinHB22

Limitation #epr-independent-singleton-scoring-limitation

Because EPR scores each candidate demonstration independently as a single-shot prompt during training data construction while concatenating multiple top-ranked demonstrations at inference time, it does not model redundancy or higher-order interactions among the jointly selected in-context examples.

supports Source: Learning To Retrieve Prompts for In-Context Learning · Locator: Section 3.2, p. 4

⚠️ Curated Limitations & Negative Results

Generating training supervision requires scoring L candidate prompts with a language model for every training instance in the dataset.
Retrieves top-L' prompts greedily by independent inner-product scores without optimizing joint set diversity or prompt ordering.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:ArtificialNeuralNetwork` Artificial Neural Network	related_match	EPR trains a neural dual-encoder dense retriever supervised by a neural language model to select in-context prompts.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r735-primary-pdf`	Learning To Retrieve Prompts for In-Context Learning Ohad Rubin, Jonathan Herzig, Jonathan Berant	primary paper full_text	ID: https://dblp.org/search/publ/api?q=Learning+To+Retrieve+Prompts+for+In-Context+Learning+Rubin&h=5&format=xml sha256:d18479bda48c…
`r735-arxiv-abs`	Learning To Retrieve Prompts for In-Context Learning (arXiv abstract) Ohad Rubin, Jonathan Herzig, Jonathan Berant	primary paper abstract_only	ID: https://arxiv.org/abs/2112.08633 sha256:3d473a7c3699…
`r735-metadata-cr`	Learning To Retrieve Prompts for In-Context Learning (Crossref metadata) Ohad Rubin, Jonathan Herzig, Jonathan Berant	primary paper abstract_only	ID: https://dblp.org/search/publ/api?q=Learning+To+Retrieve+Prompts+for+In-Context+Learning+Rubin&h=5&format=xml sha256:c9bedf435e59…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T13:38:00Z

Updated: 2026-10-19T13:38:00Z

Source File: data/curated/EPREfficientPromptRetrievalForInContextLearning.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories