neuralhash collisions

blog/online-safety-bill.md

@@ -2,7 +2,7 @@
 title: Against the "Online Safety Bill"
 description: In which I get annoyed at yet more misguided UK government behaviour.
 created: 08/07/2021
-updated: 06/08/2021
+updated: 19/08/2021
 slug: osbill
 ---
 I recently found out that the UK government aims to introduce the "[Online Safety Bill](https://www.gov.uk/government/publications/draft-online-safety-bill)" and read about it somewhat (no, I have not actually read much of the (draft) bill itself; it is 145 pages with 146 of impact assessments and 123 of explanatory notes, and so out of reach of all but very dedicated lawyers) and, as someone on the internet, it seems very harmful. This has already been detailed quite extensively and probably better than I [can](https://techcrunch.com/2021/05/12/uk-publishes-draft-online-safety-bill/) [manage](https://www.openrightsgroup.org/blog/access-denied-service-blocking-in-the-online-safety-bill/) [elsewhere](https://matrix.org/blog/2021/05/19/how-the-u-ks-online-safety-bill-threatens-matrix), so I'll just summarize my issues relatively quickly. 
@@ -19,4 +19,6 @@ If you are in fact in the UK, I hope this has convinced you to do something abou
 
 Update (19/07/2021): also consider reading [this](https://boingboing.net/2012/01/10/lockdown.html), which addresses this sort of thing as a result of more general problems.
 
-Update (06/08/2021): [Oh look, Apple just did the client-side scanning thing](https://appleprivacyletter.com/). I do not think this sets a good precedent; this is the most obviously defensible usecase for this technology, and now future extensions can just be portrayed as a natural extension of it. The best case is that this is a prelude to E2EE iCloud, but this is still a fundamental hole in the security of such a thing. Whatever happens, given government pressure, reverting this will be quite hard.
+Update (06/08/2021): [Oh look, Apple just did the client-side scanning thing](https://appleprivacyletter.com/). I do not think this sets a good precedent; this is the most obviously defensible usecase for this technology, and now future extensions can just be portrayed as a natural extension of it. The best case is that this is a prelude to E2EE iCloud, but this is still a fundamental hole in the security of such a thing. Whatever happens, given government pressure, reverting this will be quite hard.
+
+Update (19/08/2021): As it turns out, NeuralHash, which Apple intend to use for the above, is [easily collidable](https://github.com/anishathalye/neural-hash-collider) (using a fairly generic technique which should be applicable to any other neural-network-based implementation). This seems like something which should have been caught prior to release. And apparently it has [significant variations](https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX) from floating point looseness, somehow. The "1 in 1 trillion" false positive rate is maybe not very likely. It [is claimed](https://www.theverge.com/2021/8/18/22630439/apple-csam-neuralhash-collision-vulnerability-flaw-cryptography) that this is not a significant issue primarily because the hashes are secret (because of course); however, this still creates a possible issues for the system, like editing the hash of an actually-bad image to avoid detection, or (with this and some way to get around the later review stages, like [adverserial image scaling](https://bdtechtalks.com/2020/08/03/machine-learning-adversarial-image-scaling/) or just using legal content likely to trigger a human false-positive) generating otherwise okay-looking images which are flagged. Also, the [Apple announcement](https://www.apple.com/child-safety/) explicitly says "These efforts will evolve and expand over time", which is a worrying thing I did not notice before.