Reranking

Reranking is the second stage of a retrieve-then-rerank cascade. The first stage (BM25, dense retrieval or a hybrid) optimises recall over the whole corpus with scores that can be precomputed per document; the second stage spends far more compute on a short candidate list, typically 50 to 200 items, to optimise precision at the top, measured with nDCG@10 or MRR@10. The division of labour is strict: a reranker can only reorder what the first stage returned, so first-stage recall at the candidate depth is a hard ceiling on end-to-end quality.

The standard neural reranker is a cross-encoder. Query and passage are concatenated into one input, for BERT-style models [CLS] query [SEP] passage [SEP], and full self-attention runs across both, so every query token can attend to every passage token; a linear head on the pooled output gives a relevance logit. Nogueira and Cho (2019) showed with monoBERT that this substantially outperformed BM25 on the MS MARCO passage ranking task, and monoT5 later framed relevance as the probability of generating "true" versus "false". Because nothing can be precomputed, cost is one full forward pass per query-candidate pair, and latency grows linearly with candidate depth and passage length; inputs beyond the model's maximum length, often 512 tokens, are truncated, so long chunks may be judged on their beginning only. Training uses pointwise binary cross-entropy or pairwise and listwise losses, with hard negatives mined from the first-stage retriever.

Several alternatives sit around the cross-encoder. Late-interaction models such as ColBERT (Khattab and Zaharia, 2020) store one vector per document token and score with a sum of per-query-token maximum similarities (MaxSim), keeping document encoding offline at the cost of much larger indexes. LLM-based rerankers work pointwise, pairwise or listwise; RankGPT (Sun et al., 2023) prompts a model with a numbered list of passages and asks for a permutation, using sliding windows for long lists, but it is expensive and sensitive to the order in which candidates are presented. Before neural models, learning-to-rank methods such as LambdaMART combined hand-crafted features with gradient-boosted trees, and they remain common in e-commerce search.

In practice reranker scores are relative, not calibrated probabilities, so a fixed cut-off threshold needs validation on labelled data before it is used to drop passages or trigger an "I don't know" answer. Language coverage matters: an English-only reranker can reorder Danish passages worse than the first stage did. And because the reranker decides which few passages reach the generator, it is a natural place to enforce diversity, deduplicate near-identical chunks and, importantly, never to reintroduce passages that permission filtering removed earlier.