blog external link tracking

blog/ai-accelerator.md

@@ -31,7 +31,7 @@ While "generative AI" now comprises the majority of interest in AI, a large frac
 
 But what, exactly, are the constraints on hardware driving these limits? I'm not an electrical engineer or chip designer, but much of this is public and relatively easy to understand, and some vendors provide helpful information in whitepapers.
 
-Excluding a few specialized and esoteric products like [Lightmatter](https://en.wikipedia.org/wiki/Lightmatter) and [Mythic](https://mythic.ai/)'s, AI accelerators are built on modern digital logic semiconductor processes. They contain at least one logic die - it can be more than one thanks to modern advanced packaging like Intel Foveros and TSMC CoWoS - with some mix of analog circuitry for IO, SRAM (static random access memory) for fast on-chip memory, and logic gates for the control and computation. The main limit on the complexity of the GPU is die area: each transistor in a logic circuit or IO interface or memory array consumes some area, and the cost increases somewhat superlinearly with die area. This is because die are made on a fixed-size wafer which is then cut up ("singulated"), bigger die have a higher total number of random defects ("worse yields") and so need to be discarded more often, and there's a maximum size (the "reticle limit"), above which it's necessary to combine several with expensive advanced packaging.
+Excluding a few specialized and esoteric products like [Lightmatter](https://web.archive.org/web/20240909133702/https://en.wikipedia.org/wiki/Lightmatter) and [Mythic](https://mythic.ai/)'s, AI accelerators are built on modern digital logic semiconductor processes. They contain at least one logic die - it can be more than one thanks to modern advanced packaging like Intel Foveros and TSMC CoWoS - with some mix of analog circuitry for IO, SRAM (static random access memory) for fast on-chip memory, and logic gates for the control and computation. The main limit on the complexity of the GPU is die area: each transistor in a logic circuit or IO interface or memory array consumes some area, and the cost increases somewhat superlinearly with die area. This is because die are made on a fixed-size wafer which is then cut up ("singulated"), bigger die have a higher total number of random defects ("worse yields") and so need to be discarded more often, and there's a maximum size (the "reticle limit"), above which it's necessary to combine several with expensive advanced packaging.
 
 Better manufacturing processes make transistors smaller, faster and lower-power, with the downside that a full wafer costs more. Importantly, though, not everything scales down the same - recently, SRAM has almost entirely stopped getting smaller[^7], and analog has not scaled well for some time. Only logic is still shrinking fast.
 

blog/meme-thresher.md

@@ -26,7 +26,7 @@ Even with a powerful pretrained model providing prior knowledge about the world
 
 It has been said that neural nets really want to learn - even if you make mistakes in the architecture or training code, a lot of the time they will train anyway, if worse than they should. However, they don't necessarily learn what you *want*. With just a thousand data points to train on, and a nontrivial [model](https://github.com/osmarks/meme-search-engine/blob/master/meme-rater/model.py) (an ensemble of <span class="hoverdefn" title="multi-layer perceptrons">MLPs</span>), it would immediately overfit after a single epoch. I tried various regularization schemes - increasing weight decay a lot, dropout, and even cutting the model down as far as linear regression - but this either resulted in the model underfitting or it overfitting with a slightly stranger loss curve.
 
-Fortunately, I had sort of planned for this. The existing pairs of memes I rated were randomly selected from the dataset and as a result often quite low-signal, telling the model things it "already knew" from other examples. None of this is particularly novel[^4] and active learning - inferring what data would be most useful to a model so you can gather and train on it - is widely studied. I used the least bad checkpoint to [find the highest-variance pairs](https://github.com/osmarks/meme-search-engine/blob/master/meme-rater/active_learning.py) out of a large random set, then manually rated those - they were given a separate label so I could place those at the end of training (this ultimately seemed to make the model worse) and have a separate validation set (helpful). I also looked into [finding the highest-gradient pairs](https://github.com/osmarks/meme-search-engine/blob/master/meme-rater/al2.py), but it turns out this isn't what I want: they are in some sense the most "surprising", and the most surprising things were apparently things which were already clear to the model being overturned by a new data point, not marginal cases it could nevertheless learn from.
+Fortunately, I had sort of planned for this. The existing pairs of memes I rated were randomly selected from the dataset and as a result often quite low-signal, telling the model things it "already knew" from other examples. None of this is particularly novel[^4] and active learning - inferring what data would be most useful to a model so you can gather and train on it - is widely studied. I used the least bad checkpoint to [find the highest-variance pairs](https://github.com/osmarks/meme-search-engine/blob/master/meme-rater/active_learning.py) out of a large random set, then manually rated those - they were given a separate label so I could place those at the end of training (this ultimately seemed to make the model worse) and have a separate validation set (helpful). I also looked into [finding the highest-gradient pairs](https://github.com/osmarks/meme-search-engine/blob/fc6d0c94091e5c858e424cd284ff697e415099be/meme-rater/al2.py), but it turns out this isn't what I want: they are in some sense the most "surprising", and the most surprising things were apparently things which were already clear to the model being overturned by a new data point, not marginal cases it could nevertheless learn from.
 
 The new data seemingly made it slightly less prone to overfitting, but, "excitingly", it would achieve the lowest validation loss for the early set and new set at different steps. I was not able to resolve this, so I just eyeballed the curve to find a checkpoint which a reasonable balance between these, ran another round of active learning and manual labelling, and trained on that. It was very hard for the model to learn that at all, which makes sense - many of the pairs were *very* marginal (I didn't include a tie option and didn't want to add one at a late stage, but it would perhaps have been appropriate). At some point I decided that I did not want to try and wrangle the model into working any longer and picked a good-enough checkpoint to use.
 
@@ -70,4 +70,4 @@ Thanks to AI_WAIFU, RossM, Claude-3 Opus and LLaMA-3-70B for their help with dat
 
 [^4]: Someone has, in fact, built a [similar tool](https://github.com/RossM/image-rater/tree/dev) which they used to find the best possible catgirl, but I think they do logistic regression (insufficiently powerful for my requirements) and understanding and rearranging their code to my requirements would have been slower than writing it myself.
 
-[^5]: Actually, most research wouldn't do this because research norms are terrible.
+[^5]: Actually, most research wouldn't do this because research norms are terrible.

blog/other-stuff.md

@@ -285,7 +285,7 @@ The author, Zachary Mason, also wrote [The Lost Books of the Odyssey](https://ww
 * [The Titanomachy](https://kishoto.wordpress.com/2015/09/09/the-titanomachy-rrational-challenge-defied-prophecy/).
 * [Wizard, Cabalist, Ascendant](/stuff/wizard-cabalist-ascendant.xhtml).
 * [Sekhmet Hunts The Dying Gnosis: A Computation](http://www.beneath-ceaseless-skies.com/stories/sekhmet-hunts-the-dying-gnosis-a-computation/).
-* [The Finale of the Ultimate Meta Mega Crossover](https://m.fanfiction.net/s/5389450/1/).
+* [The Finale of the Ultimate Meta Mega Crossover](https://fanfiction.net/s/5389450/1/).
 * [Shannon's Law](https://web.archive.org/web/20140628065855/http://www.tor.com/stories/2011/05/shannons-law).
 * [Dave Scum](https://docs.google.com/document/d/1SddGHeVfcVa5SCDHHTOA4RlKwnef-Q6IMw_Jqw9I0Mw/mobilebasic).
 * [Nate the Snake](https://natethesnake.com/) is a complex setup for a pun.
@@ -351,7 +351,7 @@ Men have gazed at the stars for millennia, and wondered whether there was a deit
 
 ### Freefall
 
-[Freefall](https://freefall.purrsia.com/), a hard-science-fiction webcomic. Long-running enough that it has also gone through a wide range of themes. I find it to be consistently surprisingly thoughtful.
+[Freefall](http://freefall.purrsia.com/), a hard-science-fiction webcomic. Long-running enough that it has also gone through a wide range of themes. I find it to be consistently surprisingly thoughtful.
 
 Special mentions (i.e. "I haven't gotten around to reading these but they are well-reviewed and sound interesting") to:
 * [Children of Time](https://www.goodreads.com/book/show/25499718-children-of-time) by Adrian Tchaikovsky.

blog/problem-solving.md

@@ -16,7 +16,7 @@ The goal of this is for computers to be able to solve them automatically with th
 One common example of a problem-solving puzzle to help develop 'computational skills' is Sudoku.
 While the logical reasoning skills used in Sudoku are admirable, and may even be useful for a few problems in CS, we do not often think about *why we take certain steps* when deducing the value of a tile, instead reasoning retroactively about *how we know* it.
 When we solve Sudoku we do not develop a general algorithm for solving Sudoku in our heads and then execute it; instead, we solve it through a combination of intuition and creative logical reasoning, which are not well-defined or easily applicable to computers.
-[Here](http://educ.jmu.edu/~arnoldea/MathMagHss.pdf) is an interesting article relating to this.
+[Here](https://educ.jmu.edu/~arnoldea/MathMagHss.pdf) is an interesting article relating to this.
 
 'Computational skills' puzzles are inherently unhelpful as tools for improving CS ability, as they do not force players to formalise and be specific in their thought processes while solving problems; instead, they develop the player's intuitive ability.
 In chess, competitive players often talk about 'developing intuition', i.e. being able to play a chess game automatically and without formalising their thought processes.

blog/programming-education.md

@@ -29,7 +29,7 @@ So what can be done? I don't know. Formal education is likely a lost cause: ince
 * Security mindset: as well as being directly useful for ensuring security, always thinking about where your assumptions might be flawed or how something might go wrong is vital for reliability.
 * Good code structuring, e.g. knowing when to disaggregate or aggregate modules. I think that lots of people, particularly when using OOP, are too quick to try and "break apart" interdependent code in a way which makes development much slower without actually providing much flexibility, but thousand-line files with global variables everywhere are hard to work on.
 
-If you have been paying any attention to anything within the past [two years](https://openai.com/blog/openai-codex) or so, you're probably also aware that AI (specifically large language models) will obsolete, augment, change, or do nothing whatsoever to software engineering jobs. My previous list provides some perspective for this: ChatGPT (GPT-3.5 versions; I haven't used the GPT-4 one) can model computers well enough that it can [pretend to be a Linux shell](https://www.engraved.blog/building-a-virtual-machine-inside/) quite accurately, tracking decent amounts of state while it does so; big language models have vague knowledge of basically everything on the internet, even if they don't always connect it well; ChatGPT can [also](https://twitter.com/gf_256/status/1598104835848798208) find some vulnerabilities in code; [tool use](https://til.simonwillison.net/llms/python-react-pattern) [is continually](https://openai.com/blog/function-calling-and-other-api-updates?ref=upstract.com) [being](https://gorilla.cs.berkeley.edu/) [improved](https://twitter.com/emollick/status/1657050639644360706) (probably their quick-script-writing capability already exceeds most humans'). Not every capability is there yet, of course, and I think LLMs are significantly hampered by issues humans don't have, like context window limitations, lack of online learning, and bad planning ability, but these are probably not that fundamental.
+If you have been paying any attention to anything within the past [two years](https://openai.com/blog/openai-codex/) or so, you're probably also aware that AI (specifically large language models) will obsolete, augment, change, or do nothing whatsoever to software engineering jobs. My previous list provides some perspective for this: ChatGPT (GPT-3.5 versions; I haven't used the GPT-4 one) can model computers well enough that it can [pretend to be a Linux shell](https://www.engraved.blog/building-a-virtual-machine-inside/) quite accurately, tracking decent amounts of state while it does so; big language models have vague knowledge of basically everything on the internet, even if they don't always connect it well; ChatGPT can [also](https://twitter.com/gf_256/status/1598104835848798208) find some vulnerabilities in code; [tool use](https://til.simonwillison.net/llms/python-react-pattern) [is continually](https://openai.com/blog/function-calling-and-other-api-updates) [being](https://gorilla.cs.berkeley.edu/) [improved](https://twitter.com/emollick/status/1657050639644360706) (probably their quick-script-writing capability already exceeds most humans'). Not every capability is there yet, of course, and I think LLMs are significantly hampered by issues humans don't have, like context window limitations, lack of online learning, and bad planning ability, but these are probably not that fundamental.
 
 Essentially, your job is probably not safe, as long as development continues (and big organizations actually notice).
 

blog/stack-rsapi.md

@@ -27,7 +27,7 @@ If you know what <span class="hoverdefn" title="Simple Mail Transfer Protocol">S
 
 A fun quirk of the nginx installation on `procyon` is that, since I wanted it to not be able to decrypt requests to `protagonism` (I don't entirely trust it, and duplicating the certificate issuance programs on each would be irritating), I use [ngx_stream_ssl_preread](https://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html) to forward still-encrypted TLS connections either to itself (on another port) or `protagonism`'s reverse proxy. As janky as this sounds, it does seem to work fine, except for one extremely-hard-to-reproduce bug I suspect might be related where users sometimes get shown 404 pages or the status page incorrectly. Traffic is routed over [Tailscale](https://tailscale.com/) using [Headscale](https://github.com/juanfont/headscale)[^5].
 
-Several of `protagonism`'s services are mostly-self-contained personal-use applications, such as [Minoteaur](/minoteaur/) (notes), [ankisyncd](https://github.com/ankicommunity/ankicommunity-sync-server/) (flashcards), [atuin](https://github.com/atuinsh/atuin) (shell history) and [calibre-web](https://github.com/janeczku/calibre-web) (books). The rest are somewhat more interesting, in that they do more and are in some cases publicly accessible. For example, [SPUDNET](https://d.gh0.pw/doku.php?id=gtech:spudnet). It was built to serve the needs of an ["operating system"](https://potatos.madefor.cc/) for ComputerCraft (a Minecraft computer mod) by providing <span class="hoverdefn" title="backdoors">remote debugging services</span>. Originally built about six years ago, it somehow still works with relatively minor changes (new protocol support). It provides bidirectional many-to-one and many-to-many communications over websocket, with an unnecessarily sophisticated authentication system, as well as HTTP long polling fallbacks and incident reports. [Skynet](https://github.com/osmarks/skynet) is a somewhat simpler version.
+Several of `protagonism`'s services are mostly-self-contained personal-use applications, such as [Minoteaur](/minoteaur/) (notes), [ankisyncd](https://github.com/ankicommunity/ankicommunity-sync-server/) (flashcards), [atuin](https://github.com/atuinsh/atuin) (shell history) and [calibre-web](https://github.com/janeczku/calibre-web) (books). The rest are somewhat more interesting, in that they do more and are in some cases publicly accessible. For example, [SPUDNET](https://docs.osmarks.net/hypha/spudnet). It was built to serve the needs of an ["operating system"](https://potatos.madefor.cc/) for ComputerCraft (a Minecraft computer mod) by providing <span class="hoverdefn" title="backdoors">remote debugging services</span>. Originally built about six years ago, it somehow still works with relatively minor changes (new protocol support). It provides bidirectional many-to-one and many-to-many communications over websocket, with an unnecessarily sophisticated authentication system, as well as HTTP long polling fallbacks and incident reports. [Skynet](https://github.com/osmarks/skynet) is a somewhat simpler version.
 
 I also have a monitoring system using [VictoriaMetrics](https://docs.victoriametrics.com/)[^4] and [Grafana](https://grafana.com/). VictoriaMetrics periodically scrapes services for metrics and stores time series, and Grafana can plot them. This is a fairly standard setup, and lots of software exposes Prometheus-compatible metrics itself or has an exporter available (e.g. the [node_exporter](https://github.com/prometheus/node_exporter) for general Linux machine status and the [PostgreSQL exporter](https://github.com/prometheus-community/postgres_exporter)). I went slightly further by exposing metrics in most of my custom applications, so I have, for instance, nice dashboards from my Discord bot.  These used to be public, but apparently that exposing any dashboard in Grafana allows users to read any data out of the backend, which was a bit of a security issue. One might reasonably question how much use I get out of these, as I don't get enough traffic to have to debug performance issues much, but they do look nice and their presence is calming.
 
@@ -102,7 +102,7 @@ I haven't covered *every* osmarks.net service in this post, or even all the ones
 
 [^4]: This used to be Prometheus, but I swapped VictoriaMetrics in to reduce storage requirements.
 
-[^5]: I like Tailscale's ease of use, but it's horrifyingly CPU-intensive for no obvious reason, and `procyon` is not very powerful. This would be a problem if I had traffic.
+[^5]: I like Tailscale's ease of use, but it's unreasonably CPU-intensive for no obvious reason, and `procyon` is not very powerful. This would be a problem if I had traffic.
 
 [^6]: "The structure of any system designed by an organization is isomorphic to the structure of the organization." You could argue that this is more "directly in line with Conway's law" than "freed from it", but ignore that.