osmarks/website
1
0
Fork 0
mirror of https://github.com/osmarks/website synced 2025-04-27 13:13:13 +00:00
Code Issues Releases Wiki Activity

updates and typo fixes

Browse Source
This commit is contained in:
osmarks 2024-11-21 12:02:59 +00:00
parent 32a9276cb3
commit 1c1648e0ff
4 changed files with 17 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
blog/ai-accelerator.md
View File

@ -65,7 +65,7 @@ As a brief sanity check, we can check against the values for the Nvidia A100[^10
## Classes of accelerator ## Classes of accelerator
I am in some sense trivializing years of hard work by thousands of hardware engineers by doing this coarse categorization, but I think there are primary three classes of AI accelerator being built. I am in some sense trivializing years of hard work by thousands of hardware engineers by doing this coarse categorization, but I think there are three primary classes of AI accelerator being built.
### GPU-likes ### GPU-likes
@ -83,7 +83,7 @@ A danger of these architectures is that they're easy to overfit to specific mode
### SRAM machines ### SRAM machines
DRAM is costly, slow and power-hungry, so why don't we get rid of it? Many startups have tried this or similar things, by making big die with many small cores with attached SRAM and arithmetic units. This eliminates the power cost and bottlenecks of DRAM, meaning code can run much faster... as long as its data fits into the <1GB of SRAM available on each accelerator. In the past, this would often have been sufficient; now, scaling has eaten the world, and LLMs run into the terabytes at the highest end and ~10GB at the lower. DRAM is costly, slow and power-hungry, so why don't we get rid of it? Many startups have tried this or similar things, by making big die with many small cores with attached SRAM and arithmetic units. This eliminates the power cost and bottlenecks of DRAM[^13], meaning code can run much faster... as long as its data fits into the <1GB of SRAM available on each accelerator. In the past, this would often have been sufficient; now, scaling has eaten the world, and LLMs run into the terabytes at the highest end and ~10GB at the lower.
A good example of this is Graphcore "IPUs" (intelligence processing units). They're very good at convolutions[^12] but achieve low utilization on large matrix multiplications, though the high memory bandwidth makes them better at small batches than a GPU would be. It's not clear to me exactly what their design intention was, since their architecture's main advantage is fine-grained local control, which standard neural nets did not need at the time and require even less now. It may be intended for "graph neural networks", which are used in some fields where inputs have more structure than text, or sparse training, where speed benefits are derived from skipping zeroed weights or activations. Like GPUs, this flexibility does also make them useful for non-AI tasks. A good example of this is Graphcore "IPUs" (intelligence processing units). They're very good at convolutions[^12] but achieve low utilization on large matrix multiplications, though the high memory bandwidth makes them better at small batches than a GPU would be. It's not clear to me exactly what their design intention was, since their architecture's main advantage is fine-grained local control, which standard neural nets did not need at the time and require even less now. It may be intended for "graph neural networks", which are used in some fields where inputs have more structure than text, or sparse training, where speed benefits are derived from skipping zeroed weights or activations. Like GPUs, this flexibility does also make them useful for non-AI tasks.
@ -118,3 +118,5 @@ The design space of AI accelerators built on digital logic - for mainstream work
[^11]: [Power consumption of a test chip](https://www.youtube.com/watch?v=rsxCZAE8QNA&t=1067) and [instruction overhead](https://www.youtube.com/watch?v=rsxCZAE8QNA&t=646). [^11]: [Power consumption of a test chip](https://www.youtube.com/watch?v=rsxCZAE8QNA&t=1067) and [instruction overhead](https://www.youtube.com/watch?v=rsxCZAE8QNA&t=646).
[^12]: [https://arxiv.org/abs/1912.03413](https://arxiv.org/abs/1912.03413) page 76. [^12]: [https://arxiv.org/abs/1912.03413](https://arxiv.org/abs/1912.03413) page 76.
[^13]: Also possibly supply shortages. HBM is [pretty scarce](https://www.reuters.com/technology/nvidia-supplier-sk-hynix-says-hbm-chips-almost-sold-out-2025-2024-05-02/) right now, and it may prove impossible to ramp fast enough if AI hardware demand consumes an increasing proportion of logic fab output.

2
blog/magic.md
View File

@ -25,7 +25,7 @@ The way more or less every magic system, including "hard" ones, leaves its opera
1. Spellcasters *do* have to specify almost everything (well, down to moderately complex physics), possibly making magic a way to internalize what would otherwise take lots of machinery. This is roughly the approach in [Ra](https://qntm.org/ra) and [The Machineries of Empire](https://www.goodreads.com/book/show/26118426-ninefox-gambit). Characters are legally required to offhandedly mention arbitrarily complex mathematics. 1. Spellcasters *do* have to specify almost everything (well, down to moderately complex physics), possibly making magic a way to internalize what would otherwise take lots of machinery. This is roughly the approach in [Ra](https://qntm.org/ra) and [The Machineries of Empire](https://www.goodreads.com/book/show/26118426-ninefox-gambit). Characters are legally required to offhandedly mention arbitrarily complex mathematics.
2. The less you specify the more power is required, presumably to derive the remainder some other way. I haven't seen anything but [Break Them All](https://forums.sufficientvelocity.com/threads/break-them-all-original-precross.12960/) (an obscure unfinished prequel (?) to a fanfiction to a webserial) do this. This provides a fairly natural way to make magic users more powerful from study, and to make advance preparation useful. 2. The less you specify the more power is required, presumably to derive the remainder some other way. I haven't seen anything but [Break Them All](https://forums.sufficientvelocity.com/threads/break-them-all-original-precross.12960/) (an obscure unfinished prequel (?) to a fanfiction to a webserial) do this. This provides a fairly natural way to make magic users more powerful from study, and to make advance preparation useful.
3. Magic works as either individual magic users expect (or want) it to or as everyone (in aggregate) expects it to. This is quite common, though my favourite implementation is the [Discworld](https://www.goodreads.com/series/40650-discworld) gods. 3. Magic works as either individual magic users expect (or want) it to or as everyone (in aggregate) expects it to. This is quite common, though my favourite implementation is the [Discworld](https://www.goodreads.com/series/40650-discworld) gods.
4. At some level lots of specification is necessary but, [as in software](/assets/images/magic-system.png), this is abstracted in common use. The implications of this are quite close to 2, with the addition that things aren't possible at all until someone works out and shares the low-level techniques, making it more valuable, and the possibility that the underlying systems are lost knowledge. 4. At some level lots of specification is necessary but, [as in software](/assets/images/magic-system.png), this is abstracted in common use. The implications of this are quite close to 2, with the addition that things aren't possible at all until someone works out and shares the low-level techniques, making those more valuable, and the possibility that the underlying systems are lost knowledge.
5. A controlling intelligence resolves the details as required. This implies that someone built it, and possibly that it can be negotiated with or tricked. 5. A controlling intelligence resolves the details as required. This implies that someone built it, and possibly that it can be negotiated with or tricked.
6. There are many, many effects which are possible, assigned through something random like a hash function, and people only know of/care about/use a small fraction of them. This does still preclude complex effects like healing unless there is also something biasing them toward being useful. [HPMoR](https://www.facebook.com/509414227/posts/10157671264984228/) does something like this and this is (kind of) the [Unsong](https://unsongbook.com/) magic system. 6. There are many, many effects which are possible, assigned through something random like a hash function, and people only know of/care about/use a small fraction of them. This does still preclude complex effects like healing unless there is also something biasing them toward being useful. [HPMoR](https://www.facebook.com/509414227/posts/10157671264984228/) does something like this and this is (kind of) the [Unsong](https://unsongbook.com/) magic system.
7. Simulationism - the universe is simulated using high-level approximations, and there's some unintended behaviour in edge cases because it was poorly programmed. Admittedly, this is kind of boring as a solution to anything. 7. Simulationism - the universe is simulated using high-level approximations, and there's some unintended behaviour in edge cases because it was poorly programmed. Admittedly, this is kind of boring as a solution to anything.

6
blog/minecraft-power-creep.md
View File

@ -20,7 +20,7 @@ Organizing items: previously a hard enough problem that [blog posts](https://gam
Somewhat later, [Feed The Beast](https://www.feed-the-beast.com/modpacks/76-ftb-ultimate?tab=mods) (1.4.7) brought, among *many* other things, Applied Energistics. [Applied Energistics](https://web.archive.org/web/20130301012622/http://ae-mod.info/ME-Storage-and-Automation/) replaced the complexity of pipe-based sorting and storage systems with a trivially extensible cabled network which also handled storage and on-demand autocrafting. This means that rather than requiring buildout of dedicated automation systems for every item and making everything else by hand, a much smaller set of machines could be shared between many recipes and only be invoked when needed[^1]. Somewhat later, [Feed The Beast](https://www.feed-the-beast.com/modpacks/76-ftb-ultimate?tab=mods) (1.4.7) brought, among *many* other things, Applied Energistics. [Applied Energistics](https://web.archive.org/web/20130301012622/http://ae-mod.info/ME-Storage-and-Automation/) replaced the complexity of pipe-based sorting and storage systems with a trivially extensible cabled network which also handled storage and on-demand autocrafting. This means that rather than requiring buildout of dedicated automation systems for every item and making everything else by hand, a much smaller set of machines could be shared between many recipes and only be invoked when needed[^1].
Applied Energistics' developer seemingly considered this a mistake, because around the time of [1.7.10](https://www.feed-the-beast.com/modpacks/23-ftb-infinity-evolved-17?tab=mods), the mod was overhauled with "channels", which encourage more thoroughly planned and complex network designs, as well as moving away from from standard "have ores in the ground" world generation to include rare meteors containing "alien processor presses" necessary to start. The channel system also encouraged creative subnetwork design for certain problems and incentivized organizing storage in certain ways which nobody ever did[^2] to optimize space use. Slightly earlier, Thermal Expansion introduced its "itemducts", essentially a simplified version of the RedPower pneumatic tubes which didn't need external support equipment, and [Ender IO](https://www.curseforge.com/minecraft/mc-mods/ender-io) has roughly contemporaneous conduits which simplify routing even further by fitting multiple pipes into a single block. Applied Energistics' developer seemingly considered this a mistake, because around the time of [1.7.10](https://www.feed-the-beast.com/modpacks/23-ftb-infinity-evolved-17?tab=mods), the mod was overhauled with "channels", which encourage more thoroughly planned and complex network designs, as well as moving away from from standard "have ores in the ground" world generation to include rare meteors containing "alien processor presses" necessary to start. The channel system also encouraged creative subnetwork design for certain problems and the mod incentivized organizing storage in certain ways which nobody ever did[^2] to optimize space use. Slightly earlier, Thermal Expansion introduced its "itemducts", essentially a simplified version of the RedPower pneumatic tubes which didn't need external support equipment, and [Ender IO](https://www.curseforge.com/minecraft/mc-mods/ender-io) has roughly contemporaneous conduits which simplify routing even further by fitting multiple pipes into a single block.
::: captioned src="/assets/images/ae2_example_cn.png" ::: captioned src="/assets/images/ae2_example_cn.png"
This mid-game system can store several thousand items and automatically craft a few types from its stocks. This mid-game system can store several thousand items and automatically craft a few types from its stocks.
@ -36,7 +36,7 @@ A few other less-widely-used logistics solutions have existed:
* [Logistics Pipes](https://ftb.fandom.com/wiki/Getting_Started_(Logistics_Pipes)) extended Buildcraft pipes with AE2-like capabilities while remaining tied to the physical virtual world more than AE2. It seems to have been introduced around 1.4.7 (its history is shrouded in dead links). Modern versions have [Pretty Pipes](https://www.curseforge.com/minecraft/mc-mods/pretty-pipes). * [Logistics Pipes](https://ftb.fandom.com/wiki/Getting_Started_(Logistics_Pipes)) extended Buildcraft pipes with AE2-like capabilities while remaining tied to the physical virtual world more than AE2. It seems to have been introduced around 1.4.7 (its history is shrouded in dead links). Modern versions have [Pretty Pipes](https://www.curseforge.com/minecraft/mc-mods/pretty-pipes).
* [ComputerCraft](https://tweaked.cc/) can move items between inventories (with several addons, in older versions), so people built storage systems such as [Milo](https://forums.computercraft.cc/index.php?topic=87.0), [Artist](https://github.com/squidDev-CC/artist), ItemSystem (which appears to be lost now) and about [four](https://github.com/osmarks/dragon) [janky](https://github.com/osmarks/wyvern) [projects](https://docs.osmarks.net/hypha/spectral) I've made. * [ComputerCraft](https://tweaked.cc/) can move items between inventories (with several addons, in older versions), so people built storage systems such as [Milo](https://forums.computercraft.cc/index.php?topic=87.0), [Artist](https://github.com/squidDev-CC/artist), ItemSystem (which appears to be lost now) and about [four](https://github.com/osmarks/dragon) [janky](https://github.com/osmarks/wyvern) [projects](https://docs.osmarks.net/hypha/spectral) I've made.
* [XNet](https://www.mcjty.eu/docs/mods/xnet) is essentailly a confusing domain-specific programming language for moving items, energy and fluids. * [XNet](https://www.mcjty.eu/docs/mods/xnet) is essentially a confusing domain-specific programming language for moving items, energy and fluids.
* [Botania's Corporea system](https://www.reddit.com/r/feedthebeast/comments/8hdl35/a_guide_to_botania_corporea_networks_what_are/) augments vanilla storage systems with some extra automation and IO capabilities. * [Botania's Corporea system](https://www.reddit.com/r/feedthebeast/comments/8hdl35/a_guide_to_botania_corporea_networks_what_are/) augments vanilla storage systems with some extra automation and IO capabilities.
* [Immersive Engineering](https://www.curseforge.com/minecraft/mc-mods/immersive-engineering) has conveyor belts with more or less the same restrictions as BuildCraft pipes. * [Immersive Engineering](https://www.curseforge.com/minecraft/mc-mods/immersive-engineering) has conveyor belts with more or less the same restrictions as BuildCraft pipes.
* [PneumaticCraft](https://www.curseforge.com/minecraft/mc-mods/pneumaticcraft-repressurized) has its own logistics network based on drones, like Factorio's. * [PneumaticCraft](https://www.curseforge.com/minecraft/mc-mods/pneumaticcraft-repressurized) has its own logistics network based on drones, like Factorio's.
@ -85,7 +85,7 @@ On the other hand, aside from weird outliers like Mekanism's [5x ore processing]
## Magic ## Magic
Magic mods[^5] are much more heterogeneous, and don't borrow as much from each others' designs as tech mods, plausibly because they don't feel the need to interoperate as much or to match reality at all. Tekkit, for all that it generally has less powerful mods than newer packs, contains [Equivalent Exchange](https://www.minecraftforum.net/forums/mapping-and-modding-java-edition/minecraft-mods/1282288-1-2-5-equivalent-exchange-2-v1-4-6-7), which breaks everything by allowing anything to be converted into anything else for free (once you have one item as a template). Most magic mods do not interact with so much of the game: they have their own internal progression like Thaumcraft's [research](https://ftb.fandom.com/wiki/Research_(Thaumcraft_4)), to some extent Botania's runes, and [Astral Sorcery](https://www.curseforge.com/minecraft/mc-mods/astral-sorcery)'s attunement perk tree and only sometimes provide capabilities you can "bring out" of the mod. These are sometimes quite powerful - [Psi](https://www.curseforge.com/minecraft/mc-mods/psi) arguably obsoletes most early-to-mid-game tools - but they generally don't compound across mods or appear directly commensurable enough that developers react by increasing the capabilities of their own mods. Magic mods[^5] are much more heterogeneous, and don't borrow as much from each others' designs as tech mods, plausibly because they don't feel the need to interoperate as much or to match reality at all. Tekkit, for all that it generally has less powerful mods than newer packs, contains [Equivalent Exchange](https://www.minecraftforum.net/forums/mapping-and-modding-java-edition/minecraft-mods/1282288-1-2-5-equivalent-exchange-2-v1-4-6-7), which breaks everything by allowing anything to be converted into anything else for free (once you have one item as a template). Most magic mods do not interact with so much of the game: they have their own internal progression like Thaumcraft's [research](https://ftb.fandom.com/wiki/Research_(Thaumcraft_4)), to some extent Botania's runes, and [Astral Sorcery](https://www.curseforge.com/minecraft/mc-mods/astral-sorcery)'s attunement perk tree, and only sometimes provide capabilities you can "bring out" of the mod. These are sometimes quite powerful - [Psi](https://www.curseforge.com/minecraft/mc-mods/psi) arguably obsoletes most early-to-mid-game tools - but they generally don't compound across mods or appear directly commensurable enough that developers react by increasing the capabilities of their own mods.
## Equipment ## Equipment

13
blog/other-stuff.md
View File

@ -71,7 +71,7 @@ But if this is so, why did the gods make us at all? And if we were meant only to
The gods are cruel not because they make us work. They are cruel because they allow us to hope. The gods are cruel not because they make us work. They are cruel because they allow us to hope.
::: :::
[The Divine Cities](https://www.goodreads.com/series/159695-the-divine-cities) - I like any[^1] fiction about killing God, and this has great characterization and a creatively exotic setting too. I haven't actually finished City of Miracles (the third book) yet, though. [The Divine Cities](https://www.goodreads.com/series/159695-the-divine-cities) - I like any[^1] fiction about killing God, and this has great characterization and a creatively exotic setting too. The ending of the last book did sort of annoy me, though.
### Doing God's Work ### Doing God's Work
@ -103,7 +103,7 @@ I don't know exactly what about [Ender's Game](https://en.wikipedia.org/wiki/End
### The Machineries of Empire ### The Machineries of Empire
[The Machineries of Empire](https://www.goodreads.com/series/160439-the-machineries-of-empire) has a unique setting, although I do feel like it has some issues with pacing, the vaguely mathematical nature of setting annoys me because the few pieces of specific maths are wrong[^2] (and, related to this, the setting is also not really congruent: there's not enough information there to model what might happen next yourself, or why anything might happen), and the fact that somehow nobody in several hundred years has thought of a particular externally-obvious thing until the protagonist did. [The Machineries of Empire](https://www.goodreads.com/series/160439-the-machineries-of-empire) has a unique setting, although I do feel like it has some issues with pacing, the vaguely mathematical nature of the setting annoys me because the few pieces of specific maths are wrong[^2] (and, related to this, the setting is also not really congruent: there's not enough information there to model what might happen next yourself, or why anything might happen), and it bothers me that somehow nobody in several hundred years has thought of a particular externally-obvious thing until the protagonist did.
### Mage Errant ### Mage Errant
@ -115,6 +115,10 @@ Dragons keep their money in banks like everyone else. Hoards don’t accrue inte
### Firefall ### Firefall
::: epigraph attribution=Sarasti
Not talking about case studies. Brains are survival engines, not truth detectors. If self-deception promotes fitness, the brain lies. Stops noticing- irrelevant things. Truth never matters. Only fitness. By now you don't experience the world as it exists at all. You experience a simulation built from assumptions. Shortcuts. Lies. Whole species is agnosiac by default. Rorschach does nothing to you that you don't already do to yourselves.
:::
[Firefall](https://www.goodreads.com/book/show/22838183-firefall) (technically Blindsight and Echopraxia in an omnibus edition) is one of those rare books which is actually decent at portraying very alien intelligences. I preferred Blindsight to Echopraxia but both are worth reading. Some people seem to have thought that it is "cosmic horror" (particularly Blindsight) and/or had their psyche shattered by the implications of what they read within, but this didn't happen to me for whatever reason. It is certainly bleak. Also, Peter Watts somehow makes vampires work well scientifically. [Firefall](https://www.goodreads.com/book/show/22838183-firefall) (technically Blindsight and Echopraxia in an omnibus edition) is one of those rare books which is actually decent at portraying very alien intelligences. I preferred Blindsight to Echopraxia but both are worth reading. Some people seem to have thought that it is "cosmic horror" (particularly Blindsight) and/or had their psyche shattered by the implications of what they read within, but this didn't happen to me for whatever reason. It is certainly bleak. Also, Peter Watts somehow makes vampires work well scientifically.
I also read and enjoyed [The Freeze-Frame Revolution](https://www.goodreads.com/book/show/36510759-the-freeze-frame-revolution) by the same author; it's annoyingly short, being a novella, but it's dense, weirdly cognizant of AI safety concerns for its time, and packed with some clever ideas. I also read and enjoyed [The Freeze-Frame Revolution](https://www.goodreads.com/book/show/36510759-the-freeze-frame-revolution) by the same author; it's annoyingly short, being a novella, but it's dense, weirdly cognizant of AI safety concerns for its time, and packed with some clever ideas.
@ -268,12 +272,13 @@ The author, Zachary Mason, also wrote [The Lost Books of the Odyssey](https://ww
* [The Study of Anglophysics](https://slatestarcodex.com/2014/04/03/the-study-of-anglophysics/). * [The Study of Anglophysics](https://slatestarcodex.com/2014/04/03/the-study-of-anglophysics/).
* [an important lesson in cryptography](https://archiveofourown.org/works/50491414). * [an important lesson in cryptography](https://archiveofourown.org/works/50491414).
* [Stargate Physics 101](https://archiveofourown.org/works/3673335). * [Stargate Physics 101](https://archiveofourown.org/works/3673335).
* [Programmer at Large](https://archiveofourown.org/works/9233966/chapters/21827111) does futuristic and strange software and society worldbuilding. * [Programmer at Large](https://archiveofourown.org/works/9233966/chapters/20941043) does futuristic and strange software and society worldbuilding.
* [Economies of Force](https://apex-magazine.com/short-fiction/economies-of-force/). * [Economies of Force](https://apex-magazine.com/short-fiction/economies-of-force/).
* [The Titanomachy](https://kishoto.wordpress.com/2015/09/09/the-titanomachy-rrational-challenge-defied-prophecy/). * [The Titanomachy](https://kishoto.wordpress.com/2015/09/09/the-titanomachy-rrational-challenge-defied-prophecy/).
* [Wizard, Cabalist, Ascendant](/stuff/wizard-cabalist-ascendant.xhtml). * [Wizard, Cabalist, Ascendant](/stuff/wizard-cabalist-ascendant.xhtml).
* [Sekhmet Hunts The Dying: A Computation]9http://www.beneath-ceaseless-skies.com/stories/sekhmet-hunts-the-dying-gnosis-a-computation/. * [Sekhmet Hunts The Dying Gnosis: A Computation](http://www.beneath-ceaseless-skies.com/stories/sekhmet-hunts-the-dying-gnosis-a-computation/).
* [The Finale of the Ultimate Meta Mega Crossover](https://m.fanfiction.net/s/5389450/1/). * [The Finale of the Ultimate Meta Mega Crossover](https://m.fanfiction.net/s/5389450/1/).
* [Shannon's Law](https://web.archive.org/web/20140628065855/http://www.tor.com/stories/2011/05/shannons-law).
* More to be added when I feel like it. * More to be added when I feel like it.
## Games ## Games