new Graphcore post

blog/about.md

@@ -41,7 +41,7 @@ Friend me on [Project Euler](https://projecteuler.net/friends)! My code is `1997
 * (Arch) Linux (btw) systems administration.
 * Mechanical keyboard.
 * Causing inscrutable networking problems.
-* Bench press: 85kg (4RM).
+* Bench press: 90kg (3RM).
 * Nigh-omniscient knowledge of and concern for English grammar.
 
 ::: buttons

blog/graphcore.md

@@ -0,0 +1,78 @@
+---
+title: On Graphcore
+created: 23/02/2026
+description: Trying exotic parallel processing hardware which is dead for fairly good reasons.
+slug: graphcore
+tags: ["hardware", "ai"]
+---
+::: epigraph attribution=@softminus link=https://x.com/softminus/status/1933747679315833356
+The farther you get away from the flops in the physical register file; the longer the wires get, the less bandwidth there is; and with those the microarchitecture's magic fades, its power and abstractions decay, and you are left with the mundane, real, and slow.
+:::
+
+I've briefly talked about Graphcore, a (perhaps the) British AI startup, which designed AI accelerators ("Intelligence Processing Units") with a novel architecture, in my post [comparing accelerators](/accel/#sram-machines), as an example of all-SRAM designs like Groq and Cerebras, but I recently looked again as part of an ongoing program of hardware upgrades. Their commercial failure[^1] - they are in some sense alive, and are hiring people, but were [sold to SoftBank](https://www.graphcore.ai/posts/graphcore-joins-softbank-group-to-build-next-generation-of-ai-compute) [for a pittance](https://www.ft.com/content/e33bab13-2d64-47f7-b0e2-9ba446296234) after running out of money, [lost](https://sifted.eu/articles/graphcore-cofounder-exits-company-one-year-on-from-softbank-acquisition) their CTO in 2025 and have not released any new products since 2022[^3] - was in spite of clever engineering and surprisingly prescient design.
+
+Graphcore IPU chips contain ~1000 independent cores ("tiles"), whereas modern CPUs and GPUs have ~100 and ~500 respectively (the "CUDA cores" in an Nvidia GPU are not true independent cores, and by this standard recent CPUs have about 32 "cores" per core[^18]). They hide latency with simple [barrel threading](https://en.wikipedia.org/wiki/Barrel_processor) - six threads run round-robin. GPUs, instead, have systems to schedule lots of threads onto execution when they're ready to run, and CPUs split up and rearrange instruction streams to achieve the highest single-threaded performance. IPUs can avoid these due to their simpler memory hierarchy: they don't have (hardware-managed) caches or local DRAM, the timings of which can't easily be modelled at compile time. Their only onboard memory is ~600kB of SRAM per core[^2], which is accessible quickly and with constant latency, hence why they [quote](https://hc33.hotchips.org/assets/program/conference/day2/HC2021.Graphcore.SimonKnowles.v04.pdf) bandwidth figures like 60TB/s. This is only achievable if your code is entirely memory accesses, so I think it's more useful to think of it as providing zero-wait local memory access. Inter-core communication requires an all-core sync barrier (~100ns) and cycle-precise scheduling by the compiler, since the on-chip network ("Exchange") doesn't buffer. Inter-chip communication can run over PCIe to the host (where there is one) or dedicated "IPU-Links" (~hundreds of GB/s); within-chip syncs can be done without an all-chip sync because of the lower bandwidth between IPUs than within IPUs. Inter-IPU sync latency[^19] is apparently in the tens of nanoseconds - below normal DRAM access times and faster than some L3 caches!
+
+::: captioned src=/assets/images/ipu.png link
+IPU architecture diagram via [Graphcore docs](https://docs.graphcore.ai/projects/ipu-programmers-guide/en/latest/about_ipu.html).
+:::
+
+Why this architecture? Graphcore has made different claims about it over the years, being quite an old company by AI standards (their founding in 2016 predates transformers and I imagine they had the core ideas beforehand). The most obvious reason for their design is sparsity support and overfitting to contemporary RNNs/CNNs[^9], but there are better reasons. GPT-5.2-high found [a presentation](https://cdn2.hubspot.net/hubfs/729091/assets/ScaledML%20Stanford%2024mar18%20SK.pdf) from 2018 justifying their strategy. They correctly determined that power would be a binding constraint on future AI hardware, that direct-to-GPU interconnects would need to scale beyond a single node and that memory bandwidth would continue to be a bottleneck. Also, they added hardware-accelerated [stochastic rounding](https://shape-of-code.com/2022/11/20/stochastic-rounding-reemerges/) for low-precision training in their first generation, while Nvidia only integrated this in recent Blackwell GPUs. Later, they [talk about](https://hc33.hotchips.org/assets/program/conference/day2/HC2021.Graphcore.SimonKnowles.v04.pdf) the power and cost advantages of avoiding HBM, and how having enough SRAM allows using DRAM with lower bandwidth.
+
+Most of these arguments and decisions are essentially correct, and very early: the overall Graphcore design was locked in a decade ago, but it's only in the past two or three years that datacentre buildouts became heavily power-constrained, Nvidia [started scaling NVLink to racks](https://www.nvidia.com/en-us/data-center/gb200-nvl72/), and HBM became supply-crunched (due to advanced packaging in ~2023 and memory production in ~2025[^6]) rather than merely costly. Some have blamed their lack of adoption on the architecture being difficult to program but this fails to distinguish them from competitors: efficient GPU kernels involve [all kinds of arcana](https://siboehm.com/articles/22/CUDA-MMM) even without newer sometimes-programming-model-breaking innovations such as tensor cores, [TMA](https://docs.nvidia.com/cuda/hopper-tuning-guide/index.html#tensor-memory-accelerator) asynchronous loads, Blackwell's [async matrix multiplications](https://research.colfax-intl.com/cutlass-tutorial-writing-gemm-kernels-using-tensor-memory-for-nvidia-blackwell-gpus/), new low-precision floating point formats, partitioning SMs into compute and communication, and Hopper's [cursed swizzles](https://hazyresearch.stanford.edu/blog/2024-05-12-tk). Google TPUs used to require you to write TensorFlow code and have no public way to write low-level code for cases where the compiler isn't sufficient, and many were willing to put up with this agony because they were reasonably fast and [free](https://sites.research.google/trc/about/) for some hobbyists[^4], and they have a number of external customers these days. Graphcore IPUs lack a performant "eager mode" experience like GPUs, which puts off researchers, but this is also true of TPUs, as are the long compile times[^13]. TPUs and GPUs are (were) more accessible to hobbyists and consumers, but this feels an unreasonably self-serving explanation, IPUs were given to many researchers, and large B2B sales (which they had, or at least tried for) should have been less affected by this.
+
+You could argue that they missed out on the bitter lesson. This appears partly true - an [early talk](https://www.youtube.com/watch?v=dLvkF_TmyAc) has their CTO expect that tensor compute would be less important in the future, that future workloads would be more heterogeneous, and that different specialized architectures would need to be designed/searched for different tasks - but regardless of their opinions, the chips are flexible enough that they can run transformers. The lack of directly-attached DRAM is problematic with big models (which I don't think they anticipated), but, as they describe, the capacious on-chip SRAM makes it tractable in principle to stream weights from cheap high-capacity server DRAM rather than use HBM[^5], as long as your workloads aren't especially latency-sensitive[^7]. Aside from interactive chatbots and now reinforcement learning training, most inference involving models big enough for this to be a problem *isn't* very latency-sensitive. I think the immediate cause of Graphcore's commercial failure was the [end of their deal with Microsoft](https://www.uktech.news/deep-tech/graphcore-microsoft-deal-20221010) in October 2022; unless someone involved was very perceptive[^8] (and saw no value in having IPUs for training), it is unlikely that the deal was shelved over concerns about LLM inference. My sense is that it's something like "nobody ever got fired for buying Nvidia" - people are and were used to Nvidia GPUs despite their bad system-level design (high power draw per accelerator[^10], [enormous failure rates](https://arxiv.org/abs/2503.11901v3), limited integrated networking), they were easy to prototype things on, and because transformers fit GPUs (and TPUs) well, Graphcore could win on cost grounds at best. Also, according to [dubiously sourced slides](https://www.gizchina.com/tech/tsmc-announces-its-first-3nm-ai-chip-customer-neither-apple-nor-huawei), they were planning to skip 5nm manufacturing and go straight to TSMC N3, which was delayed about a year and had yield problems (hence N3B and the relaxed N3E), so in 2022 they were competing against newer and very capable Nvidia H100s with a two-year-old chip. We must wonder whether any prototype Mk3 IPUs were ever built.
+
+However, I have an insatiable yearning for more TFLOP/s and exotic computer equipment, as well as some spare PCIe slots, and it turns out that (despite Graphcore's apparent focus on large customers and datacentres), some hardware made it out to the secondary markets. As well as [C2 IPU](https://www.servethehome.com/hands-on-with-a-graphcore-c2-ipu-pcie-card-at-dell-tech-world/) cards (including engineering samples) which occasionally appear on eBay, I found [C600 IPUs](https://docs.graphcore.ai/projects/C600-datasheet/en/latest/overview.html) selling on the secret Chinese internet[^11]. Graphcore's naming scheme is confusing - they have three publicly disclosed chip generations, which are Mk1 (~300MB SRAM, 1216 cores), Mk2 (~900MB SRAM, 1472 cores) and Bow (Mk2 but with higher clocks through [improved manufacturing](https://www.eetimes.com/graphcore-supercharges-ipu-with-wafer-on-wafer/), and [FP8 support](https://www.graphcore.ai/posts/simple-fp16-and-fp8-training-with-unit-scaling)), C2 (2x Mk1 IPU) and C600 (1x Bow IPU) PCIe cards, and various larger-scale rack/"pod" systems[^12].
+
+They're probably not available in quantity, but at \$500 (plus shipping) per C600, and about the nominal compute of an A100 or 4090, C600s offer the best TFLOP/s/\$ of any available product I know of by a large margin, with the possible exception of old V100 GPUs, with the main downsides being the lack of memory and questionable software support. As someone who often runs compute-intensive but small embedding models, and is willing to tolerate vast quantities of software jank, this fit well, and I ordered one.
+
+::: captioned src=/assets/images/graphcore_c600.jpg
+The C600 on a desk before I installed it. It's slightly grubby from, presumably, prior use. I wonder what it was used for. Strangely, it came in a quantity-1 box with Graphcore branding and a decent amount of empty space - did they not care much about packaging efficiency, or were they being sold in extremely small quantity?
+:::
+
+It failed to turn on when I installed it, but it turns out I had just forgotten to connect one end of the power cable[^17]. Somewhat surprisingly, my [guesswork-based patch](https://github.com/osmarks/gc-kernel-module-patch/) to the kernel module worked fine, and the 2020-vintage [CLI tools](https://docs.graphcore.ai/projects/command-line-tools/en/latest/introduction.html) worked as expected[^14], except the FLOP/s benchmark `gc-flops`, which exited early for some reason and returned an infeasibly high result. I got [IPUpy](https://github.com/osmarks/IPUpy-patch), which runs 1472 Python interpreters concurrently, to work with some minor tweaks, but [IPUDOOM](https://github.com/jndean/IPUDOOM) failed with a mysterious linker error after I spent 30 minutes waiting for GCC 7 to compile. This turned out to be because it shipped an opaque precompiled binary (for the wrong IPU architecture) with code for JITing cross-tile communications (normally this is meant to be statically compiled on the host).
+
+Actual ML workloads were harder. I wanted to run the [SigLIP](https://huggingface.co/google/siglip-so400m-patch14-384) image encoder model previously used for my [meme initiatives](/memescale/). In principle, with [0.7 model TFLOPS](https://github.com/mlfoundations/open_clip/blob/main/docs/model_profile.csv), 280TFLOP/s of FP16 and a reasonable 60% MFU, I should have been able to do 250 images per second. With 400 million parameters and 900MB of SRAM, I needed to use FP8, which the chip supports at double rate, so it should have been possible to go even faster. After spending several hours wrangling ONNX and [PopRT](https://github.com/graphcore/PopRT), since FP8 support was seemingly never added to their [PyTorch fork](https://github.com/graphcore/poptorch), I was able to execute the model, but only at an infeasibly low 100 images per second[^15], because I could only run at batch size 1, because at any higher batch size I got to experience the compiler spinning for ten minutes then producing "insufficient tile memory" errors. The profiling tool, which still worked after unpacking it and running it with a newer Electron version, helpfully broke down cycle count by kernel, showing that enormous amounts of time were spent in some kind of on-tile copy operation and presumably-low-utilization matrix multiplies. With all the layers of abstraction between the model and hardware, I did not know why, however.
+
+I suspected that it might have been due to inefficient attention computation. There is a [Flash Attention](https://github.com/graphcore-research/flash-attention-ipu) for IPU, but it's very unready and only works with Torch, which, as we established, does not work with FP8 in the outdated SDK. Poking at the open-source code further revealed nothing to me but enormous amounts of unpleasant C++ slop. In my hubris, I thought that with modern LLM technology it should be possible to simply replace all the inconvenient parts - the ML compiler and planner logic, but not the compiler and LLVM backend for individual tiles, which seems fine, and the low-level driver - with a cut-down pipeline for transformer inference only. Graphcore was going for training support also, and cared about complex mostly-convolutional models (in fact, matrix multiplies are handled as 1x1 convolutions). However, to generate useful code, you need to be able to operate across multiple tiles, and that requires exchange code generation[^16], and for some reason this is both closed-source and much more complicated than the "compute some timings, set four registers and trigger sync" I had anticipated. Reverse-engineering efforts are ongoing.
+
+Even without this, there are some possible applications which do work quite well. Small [GPT-2 training](https://github.com/graphcore-research/flash-attention-ipu/blob/main/demo/train.py) was perfectly operable when I tested it, and the stack seems good enough for my other very-small-model work. If anyone knows where the "Pod" hardware went (there used to be cloud offerings, but no more), I would like to try some out too.
+
+[^1]: Unlike many other startups, they had a coherent idea and succeeded in shipping two generations of basically-functional hardware and software, which is commendable. They also still retain an ML research team which produces [decent work](https://graphcore-research.github.io/publications/).
+
+[^2]: Less on the first Mk1 IPUs.
+
+[^3]: There were vague rumours of another generation of IPUs being designed (publicly, [this presentation](https://www.csm.ornl.gov/srt/conferences/Scala/2022/brown.pdf#page=22)), but no public concrete information currently.
+
+[^4]: There's some history here with TPU Podcast and later EleutherAI and GPT-Neo which is as far as I know almost totally undocumented.
+
+[^5]: This is [also](https://arxiv.org/abs/2303.06865) possible with GPUs (with their VRAM and main system memory). This is loosely related to how [FSDP](https://docs.pytorch.org/tutorials/intermediate/FSDP_tutorial.html) works.
+
+[^6]: It is possible that we'll see shortages in advanced logic soon. I don't know enough about the market to say why we don't seem to have.
+
+[^7]: If your model does fit in SRAM then inference could be extremely fast: as I mentioned before, this is how Cerebras and Groq operate. However, you need lots of chips, and software which works.
+
+[^8]: This is possible! Recall that GPT-4 was trained around August 2022, months before ChatGPT (the release was later, due to extended internal testing).
+
+[^9]: The underlying assumption here may have been that future AI would be human-brain-like (highly bandwidth-bound, small batch size, weight-sharing mostly temporally), but I have no way to tell. The company's enjoyment of their circular, loosely brain-looking, unhelpful-for-parsing-architectures neural net diagrams slightly supports this.
+
+[^10]: Graphcore explicitly [calls this out](https://www.youtube.com/watch?v=dLvkF_TmyAc&t=527) and says that power simply couldn't go higher than contemporary V100 GPUs (300W). Ultimately, it could, with Nvidia's new B200s pushing 1000W (though this has been challenging and expensive, and they can't be deployed everywhere). The HBM situation is similar, with incremental changes pushing up capacity and bandwidth (admittedly at great cost).
+
+[^11]: Xianyu, via Superbuy.
+
+[^12]: They wanted to focus on complete systems after their first generation, but apparently the Chinese market wanted more flexible PCIe cards, so they had to release C600. There might have been an export-controls reason, but I don't know of any which affected the pods and not the PCIe cards.
+
+[^13]: Possibly it's that Google's software is/was *less* annoying, or they were more willing to "eat bitterness" and make their engineers and researchers do more work to save money at scale, because TPUs avoided most external margins.
+
+[^14]: Unlike much of the rest of the stack, these do not have available source code.
+
+[^15]: On lower power than my RTX 3090, admittedly, which is something.
+
+[^16]: In principle the existing library could do this, but I don't want to interact with lots of C++ code and it seems quite opinionated.
+
+[^17]: EPS, not 8-pin PCIe like on consumer GPUs. Adapters are readily available.
+
+[^18]: SIMD lanes in AVX-512 units are close to "CUDA cores". The GPU "core" number I used is SMSPs.
+
+[^19]: Not data latency, which is [250ns](https://www.graphcore.ai/posts/accelerating-resnet50-training-on-the-ipu-behind-our-mlperf-benchmark). There are separate cables for sync.

links_cache.json

@@ -4794,5 +4794,453 @@
         "date": "2025-07-11T01:01:26.883Z",
         "website": null,
         "auto": true
+    },
+    "https://en.wikipedia.org/wiki/Denver": {
+        "excerpt": "This article is about the city and county in Colorado. For other uses, see Denver (disambiguation).",
+        "title": "Denver",
+        "author": "Contributors to Wikimedia projects",
+        "date": "2001-09-26T14:34:32Z",
+        "website": "Wikimedia Foundation, Inc.",
+        "auto": true
+    },
+    "https://en.wikipedia.org/wiki/Mexico_City": {
+        "excerpt": "Mexico City",
+        "title": "Mexico City",
+        "author": "Contributors to Wikimedia projects",
+        "date": "2001-10-30T04:58:27Z",
+        "website": "Wikimedia Foundation, Inc.",
+        "auto": true
+    },
+    "https://worksinprogress.co/issue/the-magic-of-through-running/": {
+        "excerpt": "Commuter trains often stop at the edge of cities. Building a short tunnel to join them up is often by far the most efficient way to improve a city's transport.",
+        "title": "The magic of through running",
+        "author": "wip-admin",
+        "date": "2025-06-12T08:15:50+00:00",
+        "website": null,
+        "auto": true
+    },
+    "https://www.businessinsider.com/the-8-fastest-elevators-in-the-world-2013-1": {
+        "excerpt": "Check out the eight fastest elevators worldwide, showcasing engineering marvels that redefine vertical transportation.",
+        "title": "Asian Skyscrapers Dominate A New List Of The World's Fastest Elevators",
+        "author": "Megan Willett-Wei",
+        "date": "2013-01-23T17:07:46Z",
+        "website": "Business Insider",
+        "auto": true
+    },
+    "https://westhunt.wordpress.com/2012/03/23/a-sobering-thought/": {
+        "excerpt": "In scuba diving, pressure increases by one atmosphere with each ten meters of depth.   As pressure increases, people breathing standard air mix gradually get silly.  This is mainly caused by the in…",
+        "title": "A sobering thought",
+        "author": "Posted on",
+        "date": "2012-03-23T19:19:23+00:00",
+        "website": "West Hunter",
+        "auto": true
+    },
+    "https://pubmed.ncbi.nlm.nih.gov/1130736/": {
+        "excerpt": "Nitrogen has recognized narcotic potential at hyperbaric pressures. No narcotic effect of helium has been demonstrated at any pressure. We evaluated the effect of nitrogen in air at one atmosphere on human performance by comparing it with helium-oxygen using a four-alternative divided-attention task …",
+        "title": "The anesthetic effect of air at atmospheric pressure - PubMed",
+        "author": "P M Winter, D L Bruce, M J Bach, G W Jay, E I Eger 2nd",
+        "date": null,
+        "website": "PubMed",
+        "auto": true
+    },
+    "https://web.archive.org/web/20080914174633/http://www.techdiver.ws/exotic_gases.shtml": {
+        "excerpt": "Exotic diving gases",
+        "title": "Exotic diving gases",
+        "author": null,
+        "date": null,
+        "website": null,
+        "auto": true
+    },
+    "https://en.wikipedia.org/wiki/Oxygen_toxicity": {
+        "excerpt": "Oxygen toxicity is a condition resulting from the harmful effects of breathing molecular oxygen (O2) at increased partial pressures. Severe cases can result in cell damage and death, with effects most often seen in the central nervous system, lungs, and eyes. Historically, the central nervous system condition was called the Paul Bert effect, and the pulmonary condition the Lorrain Smith effect, after the researchers who pioneered the discoveries and descriptions in the late 19th century. Oxygen toxicity is a concern for underwater divers, those on high concentrations of supplemental oxygen, and those undergoing hyperbaric oxygen therapy.",
+        "title": "Oxygen toxicity",
+        "author": "Contributors to Wikimedia projects",
+        "date": "2004-02-09T20:09:06Z",
+        "website": "Wikimedia Foundation, Inc.",
+        "auto": true
+    },
+    "https://www.ifixit.com/News/11986/iphones-are-allergic-to-helium": {
+        "excerpt": "This is the kind of tale that you don’t hear every day. During the installation of a new MRI machine, a technician started getting calls that iPhones weren’t…",
+        "title": "iPhones Are Allergic to Helium",
+        "author": "Kyle Wiens",
+        "date": "2025-09-26T16:18:07-07:00",
+        "website": "iFixit",
+        "auto": true
+    },
+    "https://data.london.gov.uk/dataset/london-underground-average-monthly-temperatures/": {
+        "excerpt": "Created 7 years ago, updated 5 months ago",
+        "title": "London Underground Average Monthly Temperatures – London Datastore",
+        "author": null,
+        "date": null,
+        "website": null,
+        "auto": true
+    },
+    "https://www.goodreads.com/book/show/59593576-venomous-lumpsucker": {
+        "excerpt": "A dark and witty story of environmental collapse and ru…",
+        "title": "Venomous Lumpsucker",
+        "author": "Ned Beauman14 books388 followersFollowFollow",
+        "date": null,
+        "website": "Goodreads",
+        "auto": true
+    },
+    "https://en.wikipedia.org/wiki/Hundreds_of_Beavers": {
+        "excerpt": "",
+        "title": "Hundreds of Beavers",
+        "author": "Contributors to Wikimedia projects",
+        "date": "2024-02-28T21:58:49Z",
+        "website": "Wikimedia Foundation, Inc.",
+        "auto": true
+    },
+    "https://en.wikipedia.org/wiki/List_of_tallest_structures": {
+        "excerpt": "From Wikipedia, the free encyclopedia",
+        "title": "List of tallest structures",
+        "author": "Contributors to Wikimedia projects",
+        "date": "2003-01-07T22:54:29Z",
+        "website": "Wikimedia Foundation, Inc.",
+        "auto": true
+    },
+    "https://asteriskmag.com/issues/05/lies-damned-lies-and-manometer-readings": {
+        "excerpt": "America’s HVAC labor force is plagued by dishonesty and frequently incapable of meeting industry standards. Interventions in indoor air quality are the next frontier in pandemic prevention — but are they up to the task?",
+        "title": "Lies, Damned Lies, and  Manometer Readings—Asterisk",
+        "author": null,
+        "date": null,
+        "website": null,
+        "auto": true
+    },
+    "https://www.worksinprogress.news/p/flipping-the-switch-on-far-uvc": {
+        "excerpt": "We’ve known about far-UVC’s promise for a decade. Why isn't it everywhere?",
+        "title": "Flipping the switch on far-UVC",
+        "author": "Richard Williamson",
+        "date": "2025-03-10T13:55:11+00:00",
+        "website": "The Works in Progress Newsletter",
+        "auto": true
+    },
+    "https://www.directdebit.co.uk/": {
+        "excerpt": "Welcome to the home of Direct Debit, the UK's favourite way to make payments automatically.",
+        "title": "Direct Debit",
+        "author": null,
+        "date": null,
+        "website": null,
+        "auto": true
+    },
+    "https://www.legislation.gov.uk/uksi/2017/752/regulation/77": {
+        "excerpt": "These Regulations transpose in part Directive 2015/2366/EU of the European Parliament and of the Council of 25th November 2015 on payment services in the internal market, amending Directives 2002/65/EC, 2009/110/EC and 2013/36/EU and Regulation (EU) No. 1093/2010, and repealing Directive 2007/64/EC (OJ L 337 23.12.2015, p.35) also known as the Revised Payment Services Directive or  “PSD2”. The Financial Conduct Authority (“FCA”) is responsible for transposing other parts of PSD2. A transposition note setting out how PSD2 will be transposed into UK law is published with the Explanatory Memorandum to these Regulations on legislation.gov.uk.",
+        "title": "The Payment Services Regulations 2017",
+        "author": null,
+        "date": null,
+        "website": null,
+        "auto": true
+    },
+    "https://www.bitsaboutmoney.com/": {
+        "excerpt": "Biweekly newsletter about intersection of tech and finance. Deep dives into the plumbing that moves money and the businesses that build those pipes.",
+        "title": "Bits about Money (@patio11)",
+        "author": null,
+        "date": null,
+        "website": "Bits about Money",
+        "auto": true
+    },
+    "https://stripe.com/gb/legal/acquirer-disclosure": {
+        "excerpt": "Acquirer Disclosure",
+        "title": "Acquirer Disclosure",
+        "author": null,
+        "date": null,
+        "website": null,
+        "auto": true
+    },
+    "https://en.wikipedia.org/wiki/EMV": {
+        "excerpt": "EMV is a payment method based on a technical standard for smart payment cards and for payment terminals and automated teller machines which can accept them. EMV stands for \"Europay, Mastercard, and Visa\", the three companies that created the standard.[1]",
+        "title": "EMV",
+        "author": "Contributors to Wikimedia projects",
+        "date": "2004-02-07T11:40:40Z",
+        "website": "Wikimedia Foundation, Inc.",
+        "auto": true
+    },
+    "https://www.bitsaboutmoney.com/archive/how-credit-cards-make-money/": {
+        "excerpt": "Credit cards make money through net interest, interchange, fees, and marketing contributions.",
+        "title": "How credit cards make money",
+        "author": "Patrick McKenzie (patio11)",
+        "date": "2021-11-05T16:05:00.000Z",
+        "website": "Bits about Money",
+        "auto": true
+    },
+    "https://en.wikipedia.org/wiki/Durbin_amendment": {
+        "excerpt": "From Wikipedia, the free encyclopedia",
+        "title": "Durbin amendment",
+        "author": "Contributors to Wikimedia projects",
+        "date": "2013-08-22T10:10:48Z",
+        "website": "Wikimedia Foundation, Inc.",
+        "auto": true
+    },
+    "https://www.mastercard.com/us/en/business/support/merchant-surcharge-rules.html": {
+        "excerpt": "Mastercard credit card surcharge rules and fees for merchants are based on product type and merchant category. Learn about disclosure requirements, surcharge caps and more.",
+        "title": "Mastercard Credit Card Surcharge Rules and Fees for Merchants",
+        "author": null,
+        "date": null,
+        "website": "Mastercard",
+        "auto": true
+    },
+    "https://www.psr.org.uk/": {
+        "excerpt": "PSR works to make payment systems accessible, reliable, and fair. We promote competition, innovation, and protection for payment system users in the UK.",
+        "title": "Home",
+        "author": null,
+        "date": null,
+        "website": null,
+        "auto": true
+    },
+    "https://www.graphcore.ai/posts/graphcore-joins-softbank-group-to-build-next-generation-of-ai-compute": {
+        "excerpt": "Graphcore today announced that the company has been acquired by SoftBank Group.",
+        "title": "Graphcore joins SoftBank Group to build next generation of AI compute",
+        "author": "Nigel Toon",
+        "date": "2024-07-11T22:59:06.000Z",
+        "website": "Graphcore",
+        "auto": true
+    },
+    "https://www.ft.com/content/e33bab13-2d64-47f7-b0e2-9ba446296234": {
+        "excerpt": "Deal will give Bristol-based company a resource boost while advancing the ‘next big bet’ of Masayoshi Son’s group",
+        "title": "SoftBank buys UK chipmaker Graphcore in AI push",
+        "author": "Tim Bradshaw, Ivan Levingston, David Keohane",
+        "date": "2024-07-11T23:01:29.403Z",
+        "website": "Financial Times",
+        "auto": true
+    },
+    "https://sifted.eu/articles/graphcore-cofounder-exits-company-one-year-on-from-softbank-acquisition": {
+        "excerpt": "The departure follows the Japanese multinational buying the struggling Bristol-based chip maker in July last year",
+        "title": "Graphcore cofounder exits company one year on from SoftBank acquisition | Sifted",
+        "author": "Kai Nicol-Schwarz",
+        "date": null,
+        "website": "Sifted",
+        "auto": true
+    },
+    "https://en.wikipedia.org/wiki/Barrel_processor": {
+        "excerpt": "From Wikipedia, the free encyclopedia",
+        "title": "Barrel processor",
+        "author": "Contributors to Wikimedia projects",
+        "date": "2004-12-14T01:32:04Z",
+        "website": "Wikimedia Foundation, Inc.",
+        "auto": true
+    },
+    "https://shape-of-code.com/2022/11/20/stochastic-rounding-reemerges/": {
+        "excerpt": "Just like integer types, floating-point types are capable of representing a finite number of numeric values.  An important difference between integer and floating types is that the result of arithmetic and relational operations using integer types is exactly representable in an integer type (provided they don’t overflow), while the result of arithmetic operations using floating types may not be exactly representable in the corresponding floating type.",
+        "title": "The Shape of Code » Stochastic rounding reemerges",
+        "author": "Derek Jones",
+        "date": null,
+        "website": null,
+        "auto": true
+    },
+    "https://www.nvidia.com/en-us/data-center/gb200-nvl72/": {
+        "excerpt": "Discover the powerful GB200 NVL72 GPU, engineered for AI workloads and next-gen data centers.",
+        "title": "NVIDIA GB200 NVL72",
+        "author": null,
+        "date": null,
+        "website": "NVIDIA",
+        "auto": true
+    },
+    "https://research.colfax-intl.com/cutlass-tutorial-writing-gemm-kernels-using-tensor-memory-for-nvidia-blackwell-gpus/": {
+        "excerpt": "The NVIDIA Blackwell architecture introduces some new features that significantly change the shape of a GEMM kernel. In this series of posts, we explore the new features available on Blackwell and …",
+        "title": "CUTLASS Tutorial: Writing GEMM Kernels Using Tensor Memory For NVIDIA® Blackwell GPUs",
+        "author": null,
+        "date": "2025-04-19T16:00:33+00:00",
+        "website": "Colfax Research",
+        "auto": true
+    },
+    "https://hazyresearch.stanford.edu/blog/2024-05-12-tk": {
+        "excerpt": "how make gpu fast?",
+        "title": "GPUs Go Brrr",
+        "author": null,
+        "date": null,
+        "website": null,
+        "auto": true
+    },
+    "https://graphcore-research.github.io/publications/": {
+        "excerpt": "The official Graphcore Research blog.",
+        "title": "Our Papers",
+        "author": "Graphcore Research",
+        "date": null,
+        "website": "Graphcore Research Blog",
+        "auto": true
+    },
+    "https://sites.research.google/trc/about/": {
+        "excerpt": "TPU Research Cloud by Google",
+        "title": "TPU Research Cloud - About",
+        "author": null,
+        "date": null,
+        "website": null,
+        "auto": true
+    },
+    "https://www.uktech.news/deep-tech/graphcore-microsoft-deal-20221010": {
+        "excerpt": "The landmark deal between Graphcore and Microsoft has fallen through, causing key investors in the UK firm to devalue their stake.",
+        "title": "Graphcore loses Microsoft deal as key investors write down stake value",
+        "author": "Oscar Hornstein",
+        "date": "2022-10-10T10:33:56+00:00",
+        "website": "UKTN",
+        "auto": true
+    },
+    "https://www.gizchina.com/tech/tsmc-announces-its-first-3nm-ai-chip-customer-neither-apple-nor-huawei": {
+        "excerpt": "TSMC announces its first 3nm AI chip customer - neither Apple nor Huawei. Graphcore Colossus will skip the 5nm process to launch a 3nm chip",
+        "title": "TSMC announces its first 3nm AI chip customer - neither Apple nor Huawei",
+        "author": "Efe Udin",
+        "date": "2020-08-28T05:55:17.000Z",
+        "website": "Gizchina",
+        "auto": true
+    },
+    "https://arxiv.org/abs/2303.06865": {
+        "excerpt": "The high computational and memory requirements of large language model (LLM) inference make it feasible only with multiple high-end accelerators. Motivated by the emerging demand for latency-insensitive tasks with batched processing, this paper initiates the study of high-throughput LLM inference using limited resources, such as a single commodity GPU. We present FlexGen, a high-throughput generation engine for running LLMs with limited GPU memory. FlexGen can be flexibly configured under various hardware resource constraints by aggregating memory and computation from the GPU, CPU, and disk. By solving a linear programming problem, it searches for efficient patterns to store and access tensors. FlexGen further compresses the weights and the attention cache to 4 bits with negligible accuracy loss. These techniques enable FlexGen to have a larger space of batch size choices and thus significantly increase maximum throughput. As a result, when running OPT-175B on a single 16GB GPU, FlexGen achieves significantly higher throughput compared to state-of-the-art offloading systems, reaching a generation throughput of 1 token/s for the first time with an effective batch size of 144. On the HELM benchmark, FlexGen can benchmark a 30B model with a 16GB GPU on 7 representative sub-scenarios in 21 hours. The code is available at https://github.com/FMInference/FlexGen",
+        "title": "FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU",
+        "author": "[Submitted on 13 Mar 2023 (v1), last revised 12 Jun 2023 (this version, v2)]",
+        "date": null,
+        "website": "arXiv.org",
+        "auto": true
+    },
+    "https://docs.pytorch.org/tutorials/intermediate/FSDP_tutorial.html": {
+        "excerpt": "PyTorch Documentation. Explore PyTorch, an open-source machine learning library that accelerates the path from research prototyping to production deployment. Discover tutorials, API references, and guides to help you build and deploy deep learning models efficiently.",
+        "title": "Getting Started with Fully Sharded Data Parallel (FSDP2)",
+        "author": "PyTorch Contributors",
+        "date": "2023-01-01T00:00:00Z",
+        "website": null,
+        "auto": true
+    },
+    "https://www.servethehome.com/hands-on-with-a-graphcore-c2-ipu-pcie-card-at-dell-tech-world/": {
+        "excerpt": "We were able to get our hands on a Graphcore C2 IPU PCIe card at Dell Technologies World 2019 to show one of the hottest pieces of AI and deep learning tech",
+        "title": "Hands-on With a Graphcore C2 IPU PCIe Card at Dell Tech World",
+        "author": "Patrick Kennedy",
+        "date": "2019-06-07T15:45:33+00:00",
+        "website": "ServeTheHome",
+        "auto": true
+    },
+    "https://docs.graphcore.ai/projects/C600-datasheet/en/latest/overview.html": {
+        "excerpt": "The Graphcore® C600 IPU-Processor card is a dual-slot, full-height PCI Express Gen4 card containing Graphcore’s Mk2 IPU with FP8 support, designed to accelerate machine intelligence applications for both training and inference. All other components are supplied by industry-standard vendors. GC-C600 is the regulatory model for the C600 PCIe card.",
+        "title": "1. Overview — C600 Datasheet",
+        "author": null,
+        "date": null,
+        "website": null,
+        "auto": true
+    },
+    "https://www.eetimes.com/graphcore-supercharges-ipu-with-wafer-on-wafer/": {
+        "excerpt": "Graphcore Bow IPU AI processor, the first commercial WoW chip, boosts performance up to 40% using same design",
+        "title": "Graphcore Supercharges IPU with Wafer-on-Wafer",
+        "author": "Sally Ward-Foxton",
+        "date": "2022-03-03T10:00:04+00:00",
+        "website": "EE Times",
+        "auto": true
+    },
+    "https://www.graphcore.ai/posts/simple-fp16-and-fp8-training-with-unit-scaling": {
+        "excerpt": "Unit Scaling is a new low-precision machine learning method able to train language models in FP16 and FP8 without loss scaling.",
+        "title": "Simple FP16 and FP8 training with unit scaling",
+        "author": "Charlie Blake",
+        "date": "2023-03-29T10:38:56.000Z",
+        "website": "Graphcore",
+        "auto": true
+    },
+    "https://en.wikipedia.org/wiki/Second_law_of_thermodynamics": {
+        "excerpt": "The second law of thermodynamics is a physical law based on universal empirical observation concerning heat and energy interconversions. A simple statement of the law is that heat always flows spontaneously from hotter to colder regions of matter (or 'downhill' in terms of the temperature gradient). Another statement is: \"Not all heat can be converted into work in a cyclic process.\"[1][2][3] These are informal definitions, however; more formal definitions appear below.",
+        "title": "Second law of thermodynamics",
+        "author": "Contributors to Wikimedia projects",
+        "date": "2002-10-24T19:45:24Z",
+        "website": "Wikimedia Foundation, Inc.",
+        "auto": true
+    },
+    "https://github.com/osmarks/gc-kernel-module-patch/": {
+        "excerpt": "Graphcore kernel driver patched for modern kernels - osmarks/gc-kernel-module-patch",
+        "title": "GitHub - osmarks/gc-kernel-module-patch: Graphcore kernel driver patched for modern kernels",
+        "author": "osmarks",
+        "date": null,
+        "website": "GitHub",
+        "auto": true
+    },
+    "https://github.com/osmarks/IPUpy-patch": {
+        "excerpt": "jndean IPUpy patched for C600. Contribute to osmarks/IPUpy-patch development by creating an account on GitHub.",
+        "title": "GitHub - osmarks/IPUpy-patch: jndean IPUpy patched for C600",
+        "author": "osmarks",
+        "date": null,
+        "website": "GitHub",
+        "auto": true
+    },
+    "https://github.com/jndean/IPUDOOM": {
+        "excerpt": "DOOM (1993) ported to run on the IPU 👿. Contribute to jndean/IPUDOOM development by creating an account on GitHub.",
+        "title": "GitHub - jndean/IPUDOOM: DOOM (1993) ported to run on the IPU 👿",
+        "author": "jndean",
+        "date": null,
+        "website": "GitHub",
+        "auto": true
+    },
+    "https://docs.graphcore.ai/projects/command-line-tools/en/latest/introduction.html": {
+        "excerpt": "Note: Searching from the top-level index page will search all documents. Searching from a specific document will search only that document.",
+        "title": "1. Introduction — Graphcore Command Line Tools",
+        "author": null,
+        "date": null,
+        "website": null,
+        "auto": true
+    },
+    "https://huggingface.co/google/siglip-so400m-patch14-384": {
+        "excerpt": "We’re on a journey to advance and democratize artificial intelligence through open source and open science.",
+        "title": "google/siglip-so400m-patch14-384 · Hugging Face",
+        "author": null,
+        "date": null,
+        "website": null,
+        "auto": true
+    },
+    "https://github.com/mlfoundations/open_clip/blob/main/docs/model_profile.csv": {
+        "excerpt": "An open source implementation of CLIP. Contribute to mlfoundations/open_clip development by creating an account on GitHub.",
+        "title": "open_clip/docs/model_profile.csv at main · mlfoundations/open_clip",
+        "author": "mlfoundations",
+        "date": null,
+        "website": "GitHub",
+        "auto": true
+    },
+    "https://docs.graphcore.ai/projects/ipu-programmers-guide/en/latest/about_ipu.html": {
+        "excerpt": "The IPU is based on a highly parallel architecture designed to accelerate\nmachine learning applications. It provides very high floating-point performance\non mixed-precision floating-point data. The floating-point calculations are performed in\neither IEEE 754 single-precision floating-point or half-precision floating-point. The white paper contains details about all aspects of IPU floating-point number representation and arithmetic.",
+        "title": "2. IPU hardware overview — IPU Programmer's Guide",
+        "author": null,
+        "date": null,
+        "website": null,
+        "auto": true
+    },
+    "https://github.com/graphcore/PopRT": {
+        "excerpt": "Contribute to graphcore/PopRT development by creating an account on GitHub.",
+        "title": "GitHub - graphcore/PopRT",
+        "author": "graphcore",
+        "date": null,
+        "website": "GitHub",
+        "auto": true
+    },
+    "https://github.com/graphcore/poptorch": {
+        "excerpt": "PyTorch interface for the IPU. Contribute to graphcore/poptorch development by creating an account on GitHub.",
+        "title": "GitHub - graphcore/poptorch: PyTorch interface for the IPU",
+        "author": "graphcore",
+        "date": null,
+        "website": "GitHub",
+        "auto": true
+    },
+    "https://github.com/graphcore-research/flash-attention-ipu": {
+        "excerpt": "Poplar implementation of FlashAttention for IPU. Contribute to graphcore-research/flash-attention-ipu development by creating an account on GitHub.",
+        "title": "GitHub - graphcore-research/flash-attention-ipu: Poplar implementation of FlashAttention for IPU",
+        "author": "graphcore-research",
+        "date": null,
+        "website": "GitHub",
+        "auto": true
+    },
+    "https://github.com/graphcore-research/flash-attention-ipu/blob/main/demo/train.py": {
+        "excerpt": "Poplar implementation of FlashAttention for IPU. Contribute to graphcore-research/flash-attention-ipu development by creating an account on GitHub.",
+        "title": "flash-attention-ipu/demo/train.py at main · graphcore-research/flash-attention-ipu",
+        "author": "graphcore-research",
+        "date": null,
+        "website": "GitHub",
+        "auto": true
+    },
+    "https://arxiv.org/abs/2503.11901v3": {
+        "excerpt": "This study characterizes GPU resilience in Delta HPC, a large-scale AI system that consists of 1,056 A100 and H100 GPUs, with over 1,300 petaflops of peak throughput. Delta HPC is operated by the National Center for Supercomputing Applications (NCSA) at the University of Illinois Urbana-Champaign. We used 2.5 years of operational data (11.7 million GPU hours) on GPU errors. Our major findings include: (i) H100 GPU memory resilience is worse than A100 GPU memory, with 3.2x lower per-GPU MTBE for memory errors, (ii) The GPU memory error-recovery mechanisms on H100 GPUs are insufficient to handle the increased memory capacity, (iii) H100 GPUs demonstrate significantly improved GPU hardware resilience over A100 GPUs with respect to critical hardware components, (iv) GPU errors on both A100 and H100 GPUs frequently result in job failures due to the lack of robust recovery mechanisms at the application level, and (v) We project the impact of GPU node availability on larger-scales and find that significant overprovisioning of 5% is necessary to handle GPU failures.",
+        "title": "Characterizing GPU Resilience and Impact on AI/HPC Systems",
+        "author": null,
+        "date": null,
+        "website": "arXiv.org",
+        "auto": true
+    },
+    "https://www.graphcore.ai/posts/accelerating-resnet50-training-on-the-ipu-behind-our-mlperf-benchmark": {
+        "excerpt": "Graphcore researchers and engineers explain how they achieved dramatic performance results for the popular computer vision model.",
+        "title": "Accelerating ResNet-50 Training on the IPU for MLPerf",
+        "author": "Dr. Mario Michael Krell, Zhenying Liu, Emmanuel Menage, and Bartosz Bogdanski",
+        "date": "2022-01-17T09:07:39.000Z",
+        "website": "Graphcore",
+        "auto": true
     }
 }

src/global.json

@@ -29,7 +29,9 @@
         "Humans aren't even AGI.",
         "The goal is whatever the outcome is. It'll be decided retroactively.",
         "The business end of a tool call.",
-        "Relentlessly cursed, intermittently."
+        "Relentlessly cursed, intermittently.",
+        "By the posthuman technocapital singularity, for the posthuman technocapital singularity.",
+        "Disclaim. Marinade. Overshadow."
     ],
     "feeds": {
         "In the Pipeline": "https://www.science.org/blogs/pipeline/feed",
@@ -56,7 +58,7 @@
         "Dominic Cummings": "https://dominiccummings.substack.com/feed",
         "citrons": "https://citrons.xyz/a/journal/rss.xml",
         "Marginal Revolution": "https://feeds.feedblitz.com/marginalrevolution",
-        "SemiAnalysis": "https://www.semianalysis.com/feed",
+        "SemiAnalysis": "https://newsletter.semianalysis.com/feed",
         "Mythic Beasts": "https://www.mythic-beasts.com/blog/feed/",
         "Tales from the Void": "https://randomsprint.substack.com/feed",
         "Max Barry": "https://maxbarry.com/index.rss",
@@ -97,7 +99,8 @@
         ["perceptron.png", "https://en.wikipedia.org/wiki/Perceptron"],
         ["rhombic_dodecahedron.gif", "https://en.wikipedia.org/wiki/Rhombic_dodecahedron"],
         ["zeroptr.gif", "https://zptr.cc/88x31/"],
-        ["haskell.gif", "https://haskell.org"]
+        ["haskell.gif", "https://haskell.org"],
+        ["merging.gif", "https://merge.moe/"]
     ],
     "mycorrhiza": "https://docs.osmarks.net",
     "tagColors": {

src/links.json

@@ -742,5 +742,75 @@
         title: "One thing that I think is a little bit frustrating about modern life that is about to get way, way, way worse, unless we legislate it away, is that you can create models of human behavior that are better at predicting future behavior than intuition in ways that we feel are sort of unfair.",
         author: "@wayneburkett",
         date: "2025-05-27"
-    }
+    },
+    "/assets/misc/copenhagen_ethics.html": {
+        title: "The Copenhagen Interpretation of Ethics",
+        date: "2015-06-24",
+        author: "jai"
+    },
+    "https://www.sciencedirect.com/science/article/pii/S0360132324003706": {
+        title: "Ventilation provision and use in homes in Great Britain: A national survey",
+        date: "2024-06-01",
+        author: ["Cairan Van Rooyen", "Tim Sharpe"]
+    },
+    "https://assets.publishing.service.gov.uk/media/62a761edd3bf7f03667c667e/ADF2_revised.pdf": {
+        title: "The Building Regulations 2010 Ventilation Approved Document F",
+        date: "2021"
+    },
+    "https://www.frbservices.org/binaries/content/assets/crsocms/news/research/2024-risk-officer-survey-results.pdf": {
+        title: "Key Findings From the Annual Federal Reserve Financial Services (FRFS) Financial Institution Risk Officer Survey (2024)",
+        author: "Federal Reserve Financial Services",
+        date: "2024"
+    },
+    "https://usa.visa.com/dam/VCOM/global/support-legal/documents/merchant-surcharging-considerations-and-requirements.pdf": {
+        title: "Merchant Surcharging Considerations and Requirements",
+        date: "2024-08-14",
+        author: "Visa"
+    },
+    "https://icm.aexp-static.com/content/dam/gms/en_us/optblue/us-mog.pdf": {
+        title: "American Express Merchant Operating Guide",
+        date: "2025-10-01",
+        author: "American Express"
+    },
+    "https://eur-lex.europa.eu/eli/reg/2015/751/oj/eng": {
+        title: "Regulation (EU) 2015/751 of the European Parliament and of the Council of 29 April 2015 on interchange fees for card-based payment transactions (Text with EEA relevance)",
+        date: "2015-05-19"
+    },
+    "https://archive.ph/og9Hh": {
+        title: "Visa and Mastercard Near Deal With Merchants That Would Change Rewards Landscape",
+        date: "2025-11-08",
+        author: "AnnaMaria Andriotis"
+    },
+    "https://hc33.hotchips.org/assets/program/conference/day2/HC2021.Graphcore.SimonKnowles.v04.pdf": {
+        title: "Graphcore Colossus Mk2 IPU",
+        date: "2021-08-24",
+        author: "Simon Knowles"
+    },
+    "https://gwern.net/complexity": {
+        title: "Complexity no Bar to AI",
+        date: "2019-06-09",
+        author: "gwern"
+    },
+    "https://www.youtube.com/watch?v=dLvkF_TmyAc": {
+        title: "Simon Knowles - More complex models and more powerful machines",
+        date: "2018-05-22",
+        website: "YouTube",
+        author: "Man AHL"
+    },
+    "https://www.csm.ornl.gov/srt/conferences/Scala/2022/brown.pdf": {
+        title: "Hybrid AI/HPC Approaches for Next Generation Multi-Trillion-Parameter Models",
+        date: "2022-11-18",
+        author: "Phil Brown"
+    },
+    "https://cdn2.hubspot.net/hubfs/729091/assets/ScaledML%20Stanford%2024mar18%20SK.pdf": {
+        title: "Scaling Throughput Processors for Machine Intelligence",
+        date: "2018-03-24",
+        author: "Simon Knowles"
+    },
+    "https://www.youtube.com/watch?v=dLvkF_TmyAc&t=527": {
+        title: "Simon Knowles - More complex models and more powerful machines",
+        date: "2018-05-22",
+        website: "YouTube",
+        author: "Man AHL"
+    },
 }