osmarks/documentation
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Edit ‘vector_indexing’

Browse Source
This commit is contained in:
osmarks 2024-11-28 20:43:57 +00:00 committed by wikimind
parent adfe60d681
commit 6d183e55b0
1 changed files with 3 additions and 1 deletions

4
vector_indexing.myco

@ -2,4 +2,6 @@
* graph-based
* product quantization (lossy compression)
* inverted lists (split vectors into clusters, search a subset of the clusters)
* inverted lists (split vectors into clusters, search a subset of the clusters)
Inverted list/product quantization was historically the most common way to search large vector datasets. However, recall is very bad in some circumstances (most notably when query/dataset vectors are drawn from significantly different distributions: see [[https://arxiv.org/abs/2305.04359]] and [[https://kay21s.github.io/RoarGraph-VLDB2024.pdf]]. The latter explains this phenomenon as resulting from the nearest neighbours being split across many more (and more widely distributed) clusters (cells) than with in-distribution queries.