Edit ‘osmarks.net_web_search_plan’

2025-04-30 22:56:20 +00:00
parent 03cecf841d
commit c71d000d74
1 changed files with 2 additions and 0 deletions
@@ -56,6 +56,8 @@ The job of a search engine is to retrieve useful information for users. This is

 Good high-performance vector index uses ~5TB RAM/billion documents (can cut this down decently if embedding vectors are shorter). DRAM is a bit under £3/GB now so ~£15000/billion documents for only index. DiskANN etc use disk instead for lower throughput for very big (~£0.1/GB) cost savings. Would also need server hardware but main cost is RAM. Could use Optane (<£1/GB, highly variable). Model training has fixed costs around (roughly) £1k for big BERT/CLIP/etc finetune - worse if training from scratch or using really long context. We probably do need at least one copy (in text and original format for later redesigns) of all documents used, though this can go on cold storage at ~£0.02/GB.

+Nontrivial cost: indexing each document requires a pass through one or more models.
+
 = Misc

 * https://gwern.net/oen