osmarks/documentation
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Edit ‘osmarks.net_web_search_plan_(secret)’

Browse Source
This commit is contained in:
osmarks 2025-03-27 23:59:44 +00:00 committed by wikimind
parent 1514c0c264
commit 3f0190633f
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
osmarks.net_web_search_plan_(secret).myco
View File

@ -52,4 +52,4 @@ The job of a search engine is to retrieve useful information for users. This is
= Cost
Good high-performance vector index uses ~5TB RAM/billion documents (can cut this down decently if embedding vectors are shorter). DRAM is a bit under £3/GB now so ~£15000/billion documents for only index. DiskANN etc use disk instead for lower throughput for very big (~£0.1/GB) cost savings. Would also need server hardware but main cost is RAM. Could use Optane (<£1/GB, highly variable). Model training has fixed costs around (roughly) £10k for big BERT/CLIP/etc finetune - worse if training from scratch or using really long context. We probably do need at least one copy (in text and original format for later redesigns) of all documents used, though this can go on cold storage at ~£0.02/GB.
Good high-performance vector index uses ~5TB RAM/billion documents (can cut this down decently if embedding vectors are shorter). DRAM is a bit under £3/GB now so ~£15000/billion documents for only index. DiskANN etc use disk instead for lower throughput for very big (~£0.1/GB) cost savings. Would also need server hardware but main cost is RAM. Could use Optane (<£1/GB, highly variable). Model training has fixed costs around (roughly) £1k for big BERT/CLIP/etc finetune - worse if training from scratch or using really long context. We probably do need at least one copy (in text and original format for later redesigns) of all documents used, though this can go on cold storage at ~£0.02/GB.