osmarks/documentation
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Edit ‘vector_indexing’

Browse Source
This commit is contained in:
osmarks 2024-11-28 21:03:33 +00:00 committed by wikimind
parent 87b1cbf769
commit f4ad7531bd
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
vector_indexing.myco
View File

@ -6,4 +6,4 @@
IVF-DAC (for some reason), which is just inverted lists combined with product quantization, was historically the most common way to search large vector datasets. However, recall is very bad in some circumstances (most notably when query/dataset vectors are drawn from significantly different distributions: see [[https://arxiv.org/abs/2305.04359]] and [[https://kay21s.github.io/RoarGraph-VLDB2024.pdf]]). The latter explains this phenomenon as resulting from the nearest neighbours being split across many more (and more widely distributed) clusters (cells) than with in-distribution queries.
Graph-based approaches aim to create graphs such that a greedy search on the graph toward closer (by the vector distance metric) points rapidly converges on (most of the time) the best-matching point.
Graph-based approaches aim to create graphs such that a greedy search on the graph toward closer (by the vector distance metric) vertices rapidly converges on (most of the time) the best-matching vertex. These generally offer better search time/recall tradeoffs but have worse build time and are in some sense more [[cursed]] algorithmically.