diff --git a/autogollark.myco b/autogollark.myco index 825dea6..0c1a4d7 100644 --- a/autogollark.myco +++ b/autogollark.myco @@ -48,7 +48,9 @@ Autogollark currently comprises the dataset, the search API server and the [[htt * Context length issues, and subquadratic models are sort of bad, though maybe we can "upcycle" a midsized model to RWKV. This exists somewhere. Not sure of efficiency. Inference code will be awful. } * Train on e.g. Discord Unveiled (local copy available). -* Is vision practical? Probably not without a pretrained model for it, and those don't seem to be base models. Perhaps CLIP embeddings are good enough but I wouldn't count on it. +* {Is vision practical? Probably not without a pretrained model for it, and those don't seem to be base models. Perhaps CLIP embeddings are good enough but I wouldn't count on it. +* Vision "output" via retrieval over Nooscope/MSE. +} == Versions