1
0
mirror of https://github.com/osmarks/meme-search-engine.git synced 2026-01-20 09:59:45 +00:00
Commit Graph

20 Commits

Author SHA1 Message Date
osmarks
c277b49dc1 fix resumption, oops 2024-11-07 20:43:26 +00:00
osmarks
b9bb629e6f performance improvements 2024-11-07 16:52:58 +00:00
osmarks
7fa14d45ae improve observability and fix up Reddit dump for full-scale run 2024-11-02 19:38:05 +00:00
43ff9215fb File metadata storage.
The backend now knows how big images are, so the frontend can size images correctly before they're loaded.
This should significantly improve the UI on slow connections.
Also fix bug where videos weren't erased from the index properly.
2024-06-26 20:02:12 +01:00
1ab254ff1d Adjust index storage for memory efficiency and fix SQLite interface type confusion 2024-06-25 08:23:30 +01:00
747058e254 misc fixes
- thumbnails/OCR off was broken
- problematic video files caused segfaults (I blame ffmpeg for this)
2024-05-30 19:05:54 +01:00
3257521068 Video search 2024-05-30 15:58:31 +01:00
74d91d52e5 probably I should do better testing 2024-05-29 21:25:50 +01:00
5eae8674ce video parsing basics 2024-05-28 22:28:41 +01:00
a8329e43fc more progress on Reddit 2024-05-27 15:22:28 +01:00
f8d68d9d54 WIP Reddit dump loader 2024-05-24 17:47:18 +01:00
d8c147df52 Predefined embedding modes in search 2024-05-22 20:17:13 +01:00
14387a61a3 refactor configuration 2024-05-22 19:02:34 +01:00
ffc3d648a6 basic monitoring implementation 2024-05-22 18:49:32 +01:00
ce590298a7 concurrent index queries and fix database typo yet again 2024-05-22 18:25:50 +01:00
30b1b72712 I really should test database queries better 2024-05-22 14:35:29 +01:00
24fbc0dd1f apparently, that quality value is too low 2024-05-21 20:09:28 +01:00
63a9b3d9a6 the consequences of my own actions 2024-05-21 12:39:04 +01:00
b7010b41dd oops 2024-05-21 01:58:50 +01:00
7cb42e028f Rewrite entire application (well, backend) in Rust and also Go
I decided I wanted to integrate the experimental OCR thing better, so I rewrote in Go and also integrated the thumbnailer.
However, Go is a bad langauge and I only used it out of spite.
It turned out to have a very hard-to-fix memory leak due to some unclear interaction between libvips and both sets of bindings I tried, so I had Claude-3 transpile it to Rust then spent a while fixing the several mistakes it made and making tweaks.
The new Rust version works, although I need to actually do something with the OCR data and make the index queryable concurrently.
2024-05-21 00:09:04 +01:00