Cosine similarity

For vectors a and b the definition is cos(a, b) = (a · b) / (‖a‖ ‖b‖) = Σ aᵢbᵢ / (√Σ aᵢ² · √Σ bᵢ²). It is invariant to positive scaling of either vector, which is why it became the default in vector-space information retrieval: a long document with the same term proportions as a short one gets the same score. With non-negative vectors such as TF-IDF weights, as in the classic treatment in Manning, Raghavan and Schütze, the range is 0 to 1; only dense embeddings with signed components use the full −1 to 1 range. Pearson correlation is the cosine of mean-centred vectors.

For L2-normalised vectors, cosine similarity equals the dot product, and squared Euclidean distance is ‖a − b‖² = 2 − 2 cos(a, b), so ranking by cosine, by inner product and by Euclidean distance gives the same order. Vector databases exploit this: normalising at ingestion lets them use a fast inner-product kernel. The equivalence breaks if a model was trained for unnormalised inner product (DPR, for example, used the raw dot product) or if vectors are truncated, as with Matryoshka embeddings, without renormalising. Cosine distance, 1 − cos, is widely used as a dissimilarity but is not a true metric because it violates the triangle inequality; angular distance, arccos(cos)/π, is. Some index structures assume a metric, which is one reason engines expose the distance function as an explicit index parameter that must match the model.

The numeric value is less informative than the formal range suggests. Many transformer embeddings are anisotropic, occupying a narrow cone of the space (Ethayarajh, 2019), so unrelated texts can still score 0.6 or higher and negative scores are rare. Scores are therefore only comparable within one model and one version; a threshold such as "above 0.8 counts as a match" must be calibrated per model on labelled pairs and will not transfer when the model is changed. Steck, Ekanadham and Kallus (2024) showed that for some learned embeddings, cosine similarity can yield arbitrary results because the training objective leaves the scaling of individual dimensions undetermined, a reminder that the measure inherits whatever geometry the training imposed.

Computation is d multiply-adds per pair, cheap enough for brute-force comparison of a query with tens of thousands of vectors using SIMD or GPU kernels, but approximate nearest-neighbour indexes are needed at millions of vectors. Compressed representations change the arithmetic: int8 scalar quantization keeps cosine ranking close to float32, while binary quantization replaces it with Hamming distance on sign bits and is normally followed by rescoring the top candidates with full-precision vectors.