From c7adb6f355d31f6d904968973f6bf97225aab96a Mon Sep 17 00:00:00 2001 From: osmarks Date: Fri, 7 Mar 2025 12:37:10 +0000 Subject: [PATCH] =?UTF-8?q?Edit=20=E2=80=98compression=5Fis=5Fprediction?= =?UTF-8?q?=5Fis=5Fintelligence=E2=80=99?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- compression_is_prediction_is_intelligence.myco | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/compression_is_prediction_is_intelligence.myco b/compression_is_prediction_is_intelligence.myco index dc35b4e..ea0208c 100644 --- a/compression_is_prediction_is_intelligence.myco +++ b/compression_is_prediction_is_intelligence.myco @@ -1 +1 @@ -Any model which assigns probabilities to sequences can be turned into a compression algorithm (e.g. with [[arithmetic coding]]). Symmetrically, any compression algorithm can be interpreted as assigning probabilities to sequences (splitting probability mass of 1/2, 1/4, 1/8, ... between all compressed-domain sequences of length 0, 1, 2, etc). [[Intelligence]] is (mostly) predicting what will happen next in the world. This is the thinking behind the [[http://prize.hutter1.net/|Hutter Prize]] and [[large language model|large language models]] (the [[cross-entropy loss]] used in pretraining is exactly identical to optimizing(tokenwise) compression). \ No newline at end of file +Any model which assigns probabilities to sequences can be turned into a compression algorithm (e.g. with [[arithmetic coding]]). Symmetrically, any compression algorithm can be interpreted as assigning probabilities to sequences (splitting probability mass of 1/2, 1/4, 1/8, ... between all compressed-domain sequences of length 0, 1, 2, etc). [[Intelligence]] is (mostly) predicting what will happen next in the world. This is the thinking behind the [[http://prize.hutter1.net/|Hutter Prize]] and [[large language model|large language models]] (the [[cross-entropy loss]] used in pretraining is exactly identical to optimising (tokenwise) compression). \ No newline at end of file