RocketQAv2 Joint Training for Dense Passage Retrieval and Passage Reranking

A joint training algorithm for dense passage retrieval and passage reranking that couples a bi-encoder dual-encoder retriever s_de(q, p) = E_Q(q)^T E_P(p) and a cross-encoder reranker s_ce(q, p) = w^T E_C([CLS] q [SEP] p [SEP]) over a shared candidate passage list C_q = {p^+, p^-_1, ..., p^-_{K-1}} via dynamic listwise distillation and hybrid data augmentation: it normalizes both models' listwise relevance scores via softmax (p~_de(q, p_i) = exp(s_de(q, p_i)) / sum_{j=1}^K exp(s_de(q, p_j)) and p~_ce(q, p_i) = exp(s_ce(q, p_i)) / sum_{j=1}^K exp(s_ce(q, p_j))), jointly minimizes the sum of the listwise Kullback-Leibler divergence and cross-encoder supervised NLL loss L_final = L_kl + L_ce where L_kl = sum_{i=1}^K p~_ce(q, p_i) log(p~_ce(q, p_i) / p~_de(q, p_i)) and L_ce = -log p~_ce(q, p^+), and refreshes C_q using a hybrid data augmentation strategy that retrieves top candidates with the dual-encoder and filters out high-confidence false negatives using the cross-encoder.