Semantic search

In current usage semantic search means dense retrieval: an embedding model maps documents (or chunks) to vectors at indexing time, the query is embedded with the same model at search time, and an approximate nearest-neighbour index returns the k vectors with the highest cosine similarity or inner product. The idea of matching on latent meaning is older: latent semantic indexing (Deerwester et al., 1990) applied a truncated singular value decomposition to the term-document matrix. The term is also used loosely for web-search features built on knowledge graphs and for the Semantic Web's RDF-based querying, which are different technologies.

The modern turning point was Dense Passage Retrieval (Karpukhin et al., 2020): two BERT encoders, one for questions and one for passages, trained with in-batch negatives, outperformed a strong Lucene BM25 baseline by 9-19 percentage points in top-20 passage retrieval accuracy across open-domain QA datasets. The BEIR benchmark (Thakur et al., 2021) then showed the limit of that result: evaluated zero-shot on domains they were not trained on, several dense retrievers fell below BM25, whereas lexical matching transferred robustly. Much of the progress since then, in large-scale contrastive pretraining of general-purpose embedding models, is aimed at that out-of-domain gap.

Two design distinctions matter in practice. Symmetric search compares texts of the same kind (duplicate detection, similar cases), whereas asymmetric search matches a short question with a longer passage that answers it; many embedding models are trained for one or the other and expect query and document prefixes or instructions accordingly. Query-side techniques can reduce the asymmetry: HyDE (Gao et al., 2022) has a language model write a hypothetical answer and embeds that instead of the question. Multilingual embedding models place translations close together, so a Danish query can retrieve an English document, which is useful but can surprise users and complicate relevance judgements.

The characteristic failure modes follow from compressing text into one vector. Embeddings are weak on negation and polarity ("allowed" and "not allowed" can score almost identically), on exact identifiers, numbers and dates, and on rare proper names; they measure topical relatedness rather than whether a passage answers the question. A dense retriever also always returns k nearest neighbours, even when nothing relevant exists, so an application must decide, with a calibrated threshold or a reranker, when to answer "not found". These properties are why semantic search is usually combined with keyword search in hybrid retrieval and followed by reranking, and why its quality should be measured with recall@k and nDCG on labelled queries from the target domain.