Expando-Mono-Duo: Multi-Stage Text Ranking with Pretrained Sequence-to-Sequence Models

A four-stage retrieve-and-rerank architecture (Pradeep, Nogueira, & Lin, 2021) built on pretrained sequence-to-sequence Transformers (T5) that comprises: (1) Stage H_-1 ('Expando', doc2query-T5), which expands corpus documents prior to indexing with top-k sampled predicted queries; (2) Stage H_0, which retrieves top-k_0 candidates R_0 from the expanded inverted index via BM25 while passing unexpanded original texts downstream; (3) Stage H_1 ('Mono', monoT5), which performs pointwise target-word reranking on R_0 to select top-k_1 candidates R_1 (k_1 <= k_0); and (4) Stage H_2 ('Duo', duoT5), which formats all k_1(k_1 - 1) ordered candidate pairs (d_i, d_j) into 'Query: q Document0: d_i Document1: d_j Relevant:' to estimate pairwise preference probabilities p_{i,j} = P(d_i > d_j | d_i, d_j, q) via target-token ('true'/'false') logits and aggregates them using SUM, SUM-LOG, SYM-SUM (s_i = sum_{j != i} (p_{i,j} + (1 - p_{j,i}))), or SYM-SUM-LOG.