From 68880cadb986315914f59e20245c1969718db1b6 Mon Sep 17 00:00:00 2001 From: osmarks Date: Thu, 6 Mar 2025 23:06:41 +0000 Subject: [PATCH] =?UTF-8?q?Create=20=E2=80=98compression=5Fis=5Fprediction?= =?UTF-8?q?=5Fis=5Fintelligence=E2=80=99?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- compression_is_prediction_is_intelligence.myco | 1 + 1 file changed, 1 insertion(+) create mode 100644 compression_is_prediction_is_intelligence.myco diff --git a/compression_is_prediction_is_intelligence.myco b/compression_is_prediction_is_intelligence.myco new file mode 100644 index 0000000..fe45893 --- /dev/null +++ b/compression_is_prediction_is_intelligence.myco @@ -0,0 +1 @@ +Any model which assigns probabilities to sequences can be turned into a compression algorithm (e.g. with [[arithmetic coding]]). Symmetrically, any compression algorithm can be interpreted as assigning probabilities to sequences (splitting probability mass of 1/2, 1/4, 1/8, ... between all sequences of length 0, 1, 2, etc). [[Intelligence]] is (mostly) predicting what will happen next in the world. This is the thinking behind the [[http://prize.hutter1.net/|Hutter Prize]] and [[large language model|large language models]] (the [[cross-entropy loss]] used in pretraining is exactly identical to providing optimal (tokenwise) compression). \ No newline at end of file