mirror of
https://github.com/osmarks/website
synced 2024-10-30 00:56:15 +00:00
Minor updates, prototype internal search mechanism
This commit is contained in:
parent
5436237317
commit
8d81924804
@ -39,9 +39,9 @@ You can contact me through [email](mailto:me@osmarks.net), <span class="hoverdef
|
||||
* (Arch) Linux (btw) systems administration.
|
||||
* Mechanical keyboard.
|
||||
* Causing inscrutable networking problems.
|
||||
* Bench press: 77.5kg (5RM).
|
||||
* Bench press: 85kg (4RM).
|
||||
* Nigh-omniscient knowledge of and concern for English grammar.
|
||||
|
||||
::: buttons
|
||||
|
||||
:::
|
||||
:::
|
||||
|
||||
@ -16,7 +16,7 @@ I did allude to the fact that I intended to write this at some point, and now I
|
||||
* This doesn't work either, as the actual explanation for everything is just "the author says so" (or, in games, "the code says so"). In-universe, unless an author really likes infodumps, you may not find out either way[^2].
|
||||
* Magic is power centred in people (or individuals/life in general) rather than infrastructure - fictional casters can typically do a lot from their knowledge/skills/levels/etc and easily available "mana" (maybe with a few material components), while in real life I require lots of tools (my computers, 3D printer, knife, ominously specific computer-adjacent equipment, etc) and external runtime dependencies (electricity and internet connectivity).
|
||||
* This is explicitly discussed in [Mage Errant](https://www.goodreads.com/series/252085-mage-errant) as causing weird sociopolitical situations if you actually run through the consequences.
|
||||
* It seems to consistently hold - I can think of some partial counterexamples, but they still pretty heavily confer power on individuals. Even in [Minecraft](/assets/images/botania-tech-mod.png), "magic mods" tend to contain more powers and equipment for the player than "tech mods".
|
||||
* It seems to consistently hold - I can think of some partial counterexamples, but they still pretty heavily confer power on individuals. Even in [Minecraft](/assets/images/botania-tech-mod.png), "magic mods" tend to contain more powers and equipment for the player than "tech mods"[^6].
|
||||
|
||||
Now that I've forced you to read those paragraphs, my real position is that "magic" is a fuzzy aesthetic concept related to all of these things to varying degrees: for any "technology" I can think of, I can adjust some surface details and/or backstory to make it "magic", and vice versa. At some level, people generally don't like the way science, technology and the real world work, since our default ontologies aren't built for them, and we have some standard ways to imply "I'm ignoring these in favour something I like more".
|
||||
|
||||
@ -42,4 +42,6 @@ This is more of a set of somewhat linked ramblings than a cohesive post like I u
|
||||
|
||||
[^4]: This is actually not a great example: it's not as complicated as some other "simple" things in other fiction, and the spell-complexity issue is explicitly called out later in reference to a cleaning spell.
|
||||
|
||||
[^5]: I think this is because authors frequently study English and not ~~a real subject~~ sciences.
|
||||
[^5]: I think this is because authors frequently study English and not ~~a real subject~~ sciences.
|
||||
|
||||
[^6]: See also [this essay](https://genetyx8.github.io/flat-square-torus/2021/07/09/magic-mods.html).
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
---
|
||||
title: Other things you may like
|
||||
description: A nonexhaustive list of... content/media... which I like and which you may also be interested in as a visitor of my site.
|
||||
description: A nonexhaustive list of media which I like and which you may also be interested in as a visitor of my site.
|
||||
created: 11/06/2020
|
||||
updated: 15/07/2024
|
||||
slug: otherstuff
|
||||
@ -53,7 +53,7 @@ Obviously this is just stuff *I* like; you might not like it, which isn't really
|
||||
* [A Hero's War](https://fictionpress.com/s/3238329/1/A-Hero-s-War) - bootstrapping industrialization in a setting with magic. Unfortunately, unfinished and seems likely to remain that way.
|
||||
* [Snow Crash](https://www.goodreads.com/book/show/40651883-snow-crash) - a fun action story even though I don't take the tangents into Sumerian mythology (?) very seriously.
|
||||
* Since this list was written, I think it became notorious for introducing the "metaverse" as pushed by Facebook now. This is very silly. Everyone who is paying attention knows that the real metaverse is Roblox.
|
||||
* [Limitless](https://en.wikipedia.org/wiki/Limitless_(TV_series)) (the movie is also decent) - actually among the least bad depictions of superhuman intelligence I've seen in media, and generally funny.
|
||||
* [Limitless](https://en.wikipedia.org/wiki/Limitless_(TV_series)) (the movie is also decent) - among the less bad depictions of superhuman intelligence I've seen in media, and generally funny.
|
||||
* [Pantheon](https://en.wikipedia.org/wiki/Pantheon_(TV_series)) - ~~unfortunately cancelled and pulled from streaming (for tax purposes somehow?) and thus hard to watch,~~ apparently uncancelled and hosted by Amazon now?! Still hard to watch. One of about three TV series I've seen on the subject of brain uploads, and I think the smartest, not that this is a very high bar since it's frequently quite silly (they repeatedly talk about how uploads are just data which can be copied, and then forget this every time it would be useful). Some day I want my own ominous giant cube of servers in Norway.
|
||||
* [Mark of the Fool](https://www.goodreads.com/series/346305-mark-of-the-fool) - somewhat standardly D&D-like world, but the characters are well-written and take reasonable decisions.
|
||||
* [Nice Dragons Finish Last](https://www.goodreads.com/series/128485-heartstrikers) - enjoyable urban fantasy.
|
||||
@ -102,4 +102,4 @@ You can suggest other possibly-good stuff in the comments and I may add it to an
|
||||
|
||||
[^2]: <div><blockquote>Think of normal spacetime, said the author/illustrator, as a hypersurface. Each point on that surface had a tangent space associated with it. The tangent space could be considered a linearization of the area around the point, with extraneous information knifed away. Anyone stuck in the region of a threshold winnower’s effect was painfully affected by the linearization.</blockquote>That is not at all how that works. Also some parts on cryptography. I can only assume reviewers generally ignored this because they studied English.</div>
|
||||
|
||||
[^3]: I think [DeepMind's](https://deepmind.google/discover/blog/capture-the-flag-the-emergence-of-complex-cooperative-agents/) [agents](https://danijar.com/project/dreamerv3/) [work](https://deepmind.google/discover/blog/alphastar-mastering-the-real-time-strategy-game-starcraft-ii/) is more than sufficient to build massively superhuman space pilot/combat automation without further fundamental advances or faster computers.
|
||||
[^3]: I think [DeepMind's](https://deepmind.google/discover/blog/capture-the-flag-the-emergence-of-complex-cooperative-agents/) [agents](https://danijar.com/project/dreamerv3/) [work](https://deepmind.google/discover/blog/alphastar-mastering-the-real-time-strategy-game-starcraft-ii/) is more than sufficient to build massively superhuman space pilot/combat automation without further fundamental advances or faster computers.
|
||||
|
||||
622
package-lock.json
generated
622
package-lock.json
generated
@ -10,22 +10,31 @@
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@msgpack/msgpack": "^3.0.0-beta2",
|
||||
"@vscode/markdown-it-katex": "^1.1.0",
|
||||
"axios": "^1.5.0",
|
||||
"better-sqlite3": "^11.0.0",
|
||||
"binary-fuse-filter": "^1.0.0",
|
||||
"chalk": "^4.1.0",
|
||||
"css-select": "^5.1.0",
|
||||
"dayjs": "^1.8.28",
|
||||
"dom-serializer": "^2.0.0",
|
||||
"domutils": "^3.1.0",
|
||||
"esbuild": "^0.19.6",
|
||||
"fs-extra": "^8.1.0",
|
||||
"gray-matter": "^4.0.2",
|
||||
"handlebars": "^4.7.6",
|
||||
"html-minifier": "^4.0.0",
|
||||
"html-to-text": "^9.0.5",
|
||||
"idb": "^7.1.1",
|
||||
"js-xxhash": "^4.0.0",
|
||||
"markdown-it": "^14.1.0",
|
||||
"markdown-it-anchor": "^8.6.7",
|
||||
"markdown-it-container": "^4.0.0",
|
||||
"markdown-it-footnote": "^3.0.3",
|
||||
"msgpackr": "^1.11.0",
|
||||
"mustache": "^4.0.1",
|
||||
"nanoid": "^2.1.11",
|
||||
"porter2": "^1.0.1",
|
||||
"pug": "^3.0.2",
|
||||
"ramda": "^0.26.1",
|
||||
"sass": "^1.26.8",
|
||||
@ -403,6 +412,97 @@
|
||||
"node": ">= 14"
|
||||
}
|
||||
},
|
||||
"node_modules/@msgpackr-extract/msgpackr-extract-darwin-arm64": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-darwin-arm64/-/msgpackr-extract-darwin-arm64-3.0.3.tgz",
|
||||
"integrity": "sha512-QZHtlVgbAdy2zAqNA9Gu1UpIuI8Xvsd1v8ic6B2pZmeFnFcMWiPLfWXh7TVw4eGEZ/C9TH281KwhVoeQUKbyjw==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
]
|
||||
},
|
||||
"node_modules/@msgpackr-extract/msgpackr-extract-darwin-x64": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-darwin-x64/-/msgpackr-extract-darwin-x64-3.0.3.tgz",
|
||||
"integrity": "sha512-mdzd3AVzYKuUmiWOQ8GNhl64/IoFGol569zNRdkLReh6LRLHOXxU4U8eq0JwaD8iFHdVGqSy4IjFL4reoWCDFw==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
]
|
||||
},
|
||||
"node_modules/@msgpackr-extract/msgpackr-extract-linux-arm": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-linux-arm/-/msgpackr-extract-linux-arm-3.0.3.tgz",
|
||||
"integrity": "sha512-fg0uy/dG/nZEXfYilKoRe7yALaNmHoYeIoJuJ7KJ+YyU2bvY8vPv27f7UKhGRpY6euFYqEVhxCFZgAUNQBM3nw==",
|
||||
"cpu": [
|
||||
"arm"
|
||||
],
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
]
|
||||
},
|
||||
"node_modules/@msgpackr-extract/msgpackr-extract-linux-arm64": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-linux-arm64/-/msgpackr-extract-linux-arm64-3.0.3.tgz",
|
||||
"integrity": "sha512-YxQL+ax0XqBJDZiKimS2XQaf+2wDGVa1enVRGzEvLLVFeqa5kx2bWbtcSXgsxjQB7nRqqIGFIcLteF/sHeVtQg==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
]
|
||||
},
|
||||
"node_modules/@msgpackr-extract/msgpackr-extract-linux-x64": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-linux-x64/-/msgpackr-extract-linux-x64-3.0.3.tgz",
|
||||
"integrity": "sha512-cvwNfbP07pKUfq1uH+S6KJ7dT9K8WOE4ZiAcsrSes+UY55E/0jLYc+vq+DO7jlmqRb5zAggExKm0H7O/CBaesg==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
]
|
||||
},
|
||||
"node_modules/@msgpackr-extract/msgpackr-extract-win32-x64": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-win32-x64/-/msgpackr-extract-win32-x64-3.0.3.tgz",
|
||||
"integrity": "sha512-x0fWaQtYp4E6sktbsdAqnehxDgEc/VwM7uLsRCYWaiGu0ykYdZPiS8zCWdnjHwyiumousxfBm4SO31eXqwEZhQ==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"win32"
|
||||
]
|
||||
},
|
||||
"node_modules/@selderee/plugin-htmlparser2": {
|
||||
"version": "0.11.0",
|
||||
"resolved": "https://registry.npmjs.org/@selderee/plugin-htmlparser2/-/plugin-htmlparser2-0.11.0.tgz",
|
||||
"integrity": "sha512-P33hHGdldxGabLFjPPpaTxVolMrzrcegejx+0GxjrIb9Zv48D8yAIA/QTDR2dFl7Uz7urX8aX6+5bCZslr+gWQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"domhandler": "^5.0.3",
|
||||
"selderee": "^0.11.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://ko-fi.com/killymxi"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/linkify-it": {
|
||||
"version": "3.0.2",
|
||||
"resolved": "https://registry.npmjs.org/@types/linkify-it/-/linkify-it-3.0.2.tgz",
|
||||
@ -425,6 +525,15 @@
|
||||
"integrity": "sha512-eC4U9MlIcu2q0KQmXszyn5Akca/0jrQmwDRgpAMJai7qBWq4amIQhZyNau4VYGtCeALvW1/NtjzJJ567aZxfKA==",
|
||||
"peer": true
|
||||
},
|
||||
"node_modules/@vscode/markdown-it-katex": {
|
||||
"version": "1.1.0",
|
||||
"resolved": "https://registry.npmjs.org/@vscode/markdown-it-katex/-/markdown-it-katex-1.1.0.tgz",
|
||||
"integrity": "sha512-9cF2eJpsJOEs2V1cCAoJW/boKz9GQQLvZhNvI030K90z6ZE9lRGc9hDVvKut8zdFO2ObjwylPXXXVYvTdP2O2Q==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"katex": "^0.16.4"
|
||||
}
|
||||
},
|
||||
"node_modules/acorn": {
|
||||
"version": "7.4.1",
|
||||
"resolved": "https://registry.npmjs.org/acorn/-/acorn-7.4.1.tgz",
|
||||
@ -544,6 +653,12 @@
|
||||
"node": ">=8"
|
||||
}
|
||||
},
|
||||
"node_modules/binary-fuse-filter": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/binary-fuse-filter/-/binary-fuse-filter-1.0.0.tgz",
|
||||
"integrity": "sha512-fKEvB5bWxoh0eCFmAT0IYsB4e8lpIqIfNiVIFeyjzTVQFepek6EYV3iUYuDfw6T+1nIx/7rvwTtEiod8tqc8oA==",
|
||||
"license": "Apache-2.0"
|
||||
},
|
||||
"node_modules/bindings": {
|
||||
"version": "1.5.0",
|
||||
"resolved": "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz",
|
||||
@ -562,6 +677,12 @@
|
||||
"readable-stream": "^3.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/boolbase": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/boolbase/-/boolbase-1.0.0.tgz",
|
||||
"integrity": "sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww==",
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/braces": {
|
||||
"version": "3.0.2",
|
||||
"resolved": "https://registry.npmjs.org/braces/-/braces-3.0.2.tgz",
|
||||
@ -722,6 +843,34 @@
|
||||
"@babel/types": "^7.6.1"
|
||||
}
|
||||
},
|
||||
"node_modules/css-select": {
|
||||
"version": "5.1.0",
|
||||
"resolved": "https://registry.npmjs.org/css-select/-/css-select-5.1.0.tgz",
|
||||
"integrity": "sha512-nwoRF1rvRRnnCqqY7updORDsuqKzqYJ28+oSMaJMMgOauh3fvwHqMS7EZpIPqK8GL+g9mKxF1vP/ZjSeNjEVHg==",
|
||||
"license": "BSD-2-Clause",
|
||||
"dependencies": {
|
||||
"boolbase": "^1.0.0",
|
||||
"css-what": "^6.1.0",
|
||||
"domhandler": "^5.0.2",
|
||||
"domutils": "^3.0.1",
|
||||
"nth-check": "^2.0.1"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/fb55"
|
||||
}
|
||||
},
|
||||
"node_modules/css-what": {
|
||||
"version": "6.1.0",
|
||||
"resolved": "https://registry.npmjs.org/css-what/-/css-what-6.1.0.tgz",
|
||||
"integrity": "sha512-HTUrgRJ7r4dsZKU6GjmpfRK1O76h97Z8MfS1G0FozR+oF2kG6Vfe8JE6zwrkbxigziPHinCJ+gCPjA9EaBDtRw==",
|
||||
"license": "BSD-2-Clause",
|
||||
"engines": {
|
||||
"node": ">= 6"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/fb55"
|
||||
}
|
||||
},
|
||||
"node_modules/dayjs": {
|
||||
"version": "1.10.6",
|
||||
"resolved": "https://registry.npmjs.org/dayjs/-/dayjs-1.10.6.tgz",
|
||||
@ -749,6 +898,15 @@
|
||||
"node": ">=4.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/deepmerge": {
|
||||
"version": "4.3.1",
|
||||
"resolved": "https://registry.npmjs.org/deepmerge/-/deepmerge-4.3.1.tgz",
|
||||
"integrity": "sha512-3sUqbMEc77XqpdNO7FRyRog+eW3ph+GYCbj+rK+uYyRMuwsVy0rMiVtPn+QJlKFvWP/1PYpapqYn0Me2knFn+A==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/delayed-stream": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
|
||||
@ -770,6 +928,61 @@
|
||||
"resolved": "https://registry.npmjs.org/doctypes/-/doctypes-1.1.0.tgz",
|
||||
"integrity": "sha1-6oCxBqh1OHdOijpKWv4pPeSJ4Kk="
|
||||
},
|
||||
"node_modules/dom-serializer": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-2.0.0.tgz",
|
||||
"integrity": "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"domelementtype": "^2.3.0",
|
||||
"domhandler": "^5.0.2",
|
||||
"entities": "^4.2.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/cheeriojs/dom-serializer?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/domelementtype": {
|
||||
"version": "2.3.0",
|
||||
"resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-2.3.0.tgz",
|
||||
"integrity": "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw==",
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/fb55"
|
||||
}
|
||||
],
|
||||
"license": "BSD-2-Clause"
|
||||
},
|
||||
"node_modules/domhandler": {
|
||||
"version": "5.0.3",
|
||||
"resolved": "https://registry.npmjs.org/domhandler/-/domhandler-5.0.3.tgz",
|
||||
"integrity": "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w==",
|
||||
"license": "BSD-2-Clause",
|
||||
"dependencies": {
|
||||
"domelementtype": "^2.3.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/fb55/domhandler?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/domutils": {
|
||||
"version": "3.1.0",
|
||||
"resolved": "https://registry.npmjs.org/domutils/-/domutils-3.1.0.tgz",
|
||||
"integrity": "sha512-H78uMmQtI2AhgDJjWeQmHwJJ2bLPD3GMmO7Zja/ZZh84wkm+4ut+IUnUdRa8uCGX88DiVx1j6FRe1XfxEgjEZA==",
|
||||
"license": "BSD-2-Clause",
|
||||
"dependencies": {
|
||||
"dom-serializer": "^2.0.0",
|
||||
"domelementtype": "^2.3.0",
|
||||
"domhandler": "^5.0.3"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/fb55/domutils?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/end-of-stream": {
|
||||
"version": "1.4.4",
|
||||
"resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.4.tgz",
|
||||
@ -1071,6 +1284,41 @@
|
||||
"node": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/html-to-text": {
|
||||
"version": "9.0.5",
|
||||
"resolved": "https://registry.npmjs.org/html-to-text/-/html-to-text-9.0.5.tgz",
|
||||
"integrity": "sha512-qY60FjREgVZL03vJU6IfMV4GDjGBIoOyvuFdpBDIX9yTlDw0TjxVBQp+P8NvpdIXNJvfWBTNul7fsAQJq2FNpg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@selderee/plugin-htmlparser2": "^0.11.0",
|
||||
"deepmerge": "^4.3.1",
|
||||
"dom-serializer": "^2.0.0",
|
||||
"htmlparser2": "^8.0.2",
|
||||
"selderee": "^0.11.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=14"
|
||||
}
|
||||
},
|
||||
"node_modules/htmlparser2": {
|
||||
"version": "8.0.2",
|
||||
"resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-8.0.2.tgz",
|
||||
"integrity": "sha512-GYdjWKDkbRLkZ5geuHs5NY1puJ+PXwP7+fHPRz06Eirsb9ugf6d8kkXav6ADhcODhFFPMIXyxkxSuMf3D6NCFA==",
|
||||
"funding": [
|
||||
"https://github.com/fb55/htmlparser2?sponsor=1",
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/fb55"
|
||||
}
|
||||
],
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"domelementtype": "^2.3.0",
|
||||
"domhandler": "^5.0.3",
|
||||
"domutils": "^3.0.1",
|
||||
"entities": "^4.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/idb": {
|
||||
"version": "7.1.1",
|
||||
"resolved": "https://registry.npmjs.org/idb/-/idb-7.1.1.tgz",
|
||||
@ -1201,6 +1449,15 @@
|
||||
"resolved": "https://registry.npmjs.org/js-stringify/-/js-stringify-1.0.2.tgz",
|
||||
"integrity": "sha1-Fzb939lyTyijaCrcYjCufk6Weds="
|
||||
},
|
||||
"node_modules/js-xxhash": {
|
||||
"version": "4.0.0",
|
||||
"resolved": "https://registry.npmjs.org/js-xxhash/-/js-xxhash-4.0.0.tgz",
|
||||
"integrity": "sha512-3Q2eIqG6s1KEBBmkj9tGM9lef8LJbuRyTVBdI3GpTnrvtytunjLPO0wqABp5qhtMzfA32jYn1FlnIV7GH1RAHQ==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=18.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/js-yaml": {
|
||||
"version": "3.14.1",
|
||||
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-3.14.1.tgz",
|
||||
@ -1230,6 +1487,31 @@
|
||||
"promise": "^7.0.1"
|
||||
}
|
||||
},
|
||||
"node_modules/katex": {
|
||||
"version": "0.16.11",
|
||||
"resolved": "https://registry.npmjs.org/katex/-/katex-0.16.11.tgz",
|
||||
"integrity": "sha512-RQrI8rlHY92OLf3rho/Ts8i/XvjgguEjOkO1BEXcU3N8BqPpSzBNwV/G0Ukr+P/l3ivvJUE/Fa/CwbS6HesGNQ==",
|
||||
"funding": [
|
||||
"https://opencollective.com/katex",
|
||||
"https://github.com/sponsors/katex"
|
||||
],
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"commander": "^8.3.0"
|
||||
},
|
||||
"bin": {
|
||||
"katex": "cli.js"
|
||||
}
|
||||
},
|
||||
"node_modules/katex/node_modules/commander": {
|
||||
"version": "8.3.0",
|
||||
"resolved": "https://registry.npmjs.org/commander/-/commander-8.3.0.tgz",
|
||||
"integrity": "sha512-OkTL9umf+He2DZkUq8f8J9of7yL6RJKI24dVITBmNfZBmri9zYZQrKkuXiKhyfPSu8tUhnVBB1iKXevvnlR4Ww==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">= 12"
|
||||
}
|
||||
},
|
||||
"node_modules/kind-of": {
|
||||
"version": "6.0.3",
|
||||
"resolved": "https://registry.npmjs.org/kind-of/-/kind-of-6.0.3.tgz",
|
||||
@ -1238,6 +1520,15 @@
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/leac": {
|
||||
"version": "0.6.0",
|
||||
"resolved": "https://registry.npmjs.org/leac/-/leac-0.6.0.tgz",
|
||||
"integrity": "sha512-y+SqErxb8h7nE/fiEX07jsbuhrpO9lL8eca7/Y1nuWV2moNlXhyd59iDGcRf6moVyDMbmTNzL40SUyrFU/yDpg==",
|
||||
"license": "MIT",
|
||||
"funding": {
|
||||
"url": "https://ko-fi.com/killymxi"
|
||||
}
|
||||
},
|
||||
"node_modules/linkify-it": {
|
||||
"version": "5.0.0",
|
||||
"resolved": "https://registry.npmjs.org/linkify-it/-/linkify-it-5.0.0.tgz",
|
||||
@ -1350,6 +1641,37 @@
|
||||
"resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",
|
||||
"integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A=="
|
||||
},
|
||||
"node_modules/msgpackr": {
|
||||
"version": "1.11.0",
|
||||
"resolved": "https://registry.npmjs.org/msgpackr/-/msgpackr-1.11.0.tgz",
|
||||
"integrity": "sha512-I8qXuuALqJe5laEBYoFykChhSXLikZmUhccjGsPuSJ/7uPip2TJ7lwdIQwWSAi0jGZDXv4WOP8Qg65QZRuXxXw==",
|
||||
"license": "MIT",
|
||||
"optionalDependencies": {
|
||||
"msgpackr-extract": "^3.0.2"
|
||||
}
|
||||
},
|
||||
"node_modules/msgpackr-extract": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/msgpackr-extract/-/msgpackr-extract-3.0.3.tgz",
|
||||
"integrity": "sha512-P0efT1C9jIdVRefqjzOQ9Xml57zpOXnIuS+csaB4MdZbTdmGDLo8XhzBG1N7aO11gKDDkJvBLULeFTo46wwreA==",
|
||||
"hasInstallScript": true,
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"dependencies": {
|
||||
"node-gyp-build-optional-packages": "5.2.2"
|
||||
},
|
||||
"bin": {
|
||||
"download-msgpackr-prebuilds": "bin/download-prebuilds.js"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@msgpackr-extract/msgpackr-extract-darwin-arm64": "3.0.3",
|
||||
"@msgpackr-extract/msgpackr-extract-darwin-x64": "3.0.3",
|
||||
"@msgpackr-extract/msgpackr-extract-linux-arm": "3.0.3",
|
||||
"@msgpackr-extract/msgpackr-extract-linux-arm64": "3.0.3",
|
||||
"@msgpackr-extract/msgpackr-extract-linux-x64": "3.0.3",
|
||||
"@msgpackr-extract/msgpackr-extract-win32-x64": "3.0.3"
|
||||
}
|
||||
},
|
||||
"node_modules/mustache": {
|
||||
"version": "4.2.0",
|
||||
"resolved": "https://registry.npmjs.org/mustache/-/mustache-4.2.0.tgz",
|
||||
@ -1392,6 +1714,21 @@
|
||||
"node": ">=10"
|
||||
}
|
||||
},
|
||||
"node_modules/node-gyp-build-optional-packages": {
|
||||
"version": "5.2.2",
|
||||
"resolved": "https://registry.npmjs.org/node-gyp-build-optional-packages/-/node-gyp-build-optional-packages-5.2.2.tgz",
|
||||
"integrity": "sha512-s+w+rBWnpTMwSFbaE0UXsRlg7hU4FjekKU4eyAih5T8nJuNZT1nNsskXpxmeqSK9UzkBl6UgRlnKc8hz8IEqOw==",
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"dependencies": {
|
||||
"detect-libc": "^2.0.1"
|
||||
},
|
||||
"bin": {
|
||||
"node-gyp-build-optional-packages": "bin.js",
|
||||
"node-gyp-build-optional-packages-optional": "optional.js",
|
||||
"node-gyp-build-optional-packages-test": "build-test.js"
|
||||
}
|
||||
},
|
||||
"node_modules/normalize-path": {
|
||||
"version": "3.0.0",
|
||||
"resolved": "https://registry.npmjs.org/normalize-path/-/normalize-path-3.0.0.tgz",
|
||||
@ -1400,6 +1737,18 @@
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/nth-check": {
|
||||
"version": "2.1.1",
|
||||
"resolved": "https://registry.npmjs.org/nth-check/-/nth-check-2.1.1.tgz",
|
||||
"integrity": "sha512-lqjrjmaOoAnWfMmBPL+XNnynZh2+swxiX3WUE0s4yEHI6m+AwrK2UZOimIRl3X/4QctVqS8AiZjFqyOGrMXb/w==",
|
||||
"license": "BSD-2-Clause",
|
||||
"dependencies": {
|
||||
"boolbase": "^1.0.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/fb55/nth-check?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/object-assign": {
|
||||
"version": "4.1.1",
|
||||
"resolved": "https://registry.npmjs.org/object-assign/-/object-assign-4.1.1.tgz",
|
||||
@ -1424,11 +1773,33 @@
|
||||
"no-case": "^2.2.0"
|
||||
}
|
||||
},
|
||||
"node_modules/parseley": {
|
||||
"version": "0.12.1",
|
||||
"resolved": "https://registry.npmjs.org/parseley/-/parseley-0.12.1.tgz",
|
||||
"integrity": "sha512-e6qHKe3a9HWr0oMRVDTRhKce+bRO8VGQR3NyVwcjwrbhMmFCX9KszEV35+rn4AdilFAq9VPxP/Fe1wC9Qjd2lw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"leac": "^0.6.0",
|
||||
"peberminta": "^0.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://ko-fi.com/killymxi"
|
||||
}
|
||||
},
|
||||
"node_modules/path-parse": {
|
||||
"version": "1.0.7",
|
||||
"resolved": "https://registry.npmjs.org/path-parse/-/path-parse-1.0.7.tgz",
|
||||
"integrity": "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw=="
|
||||
},
|
||||
"node_modules/peberminta": {
|
||||
"version": "0.9.0",
|
||||
"resolved": "https://registry.npmjs.org/peberminta/-/peberminta-0.9.0.tgz",
|
||||
"integrity": "sha512-XIxfHpEuSJbITd1H3EeQwpcZbTLHc+VVr8ANI9t5sit565tsI4/xK3KWTUFE2e6QiangUkh3B0jihzmGnNrRsQ==",
|
||||
"license": "MIT",
|
||||
"funding": {
|
||||
"url": "https://ko-fi.com/killymxi"
|
||||
}
|
||||
},
|
||||
"node_modules/picomatch": {
|
||||
"version": "2.3.0",
|
||||
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.0.tgz",
|
||||
@ -1440,6 +1811,12 @@
|
||||
"url": "https://github.com/sponsors/jonschlinkert"
|
||||
}
|
||||
},
|
||||
"node_modules/porter2": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/porter2/-/porter2-1.0.1.tgz",
|
||||
"integrity": "sha512-sK+5ZnjxjFrlJ2CYF1xGFv/CSBAWK0QSfrjzh4hdgjlmR90wTKW9O7a+LLk676ZinianDnMl3Jh2Vrg3OIHWBQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/prebuild-install": {
|
||||
"version": "7.1.1",
|
||||
"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.1.tgz",
|
||||
@ -1717,6 +2094,18 @@
|
||||
"node": ">=4"
|
||||
}
|
||||
},
|
||||
"node_modules/selderee": {
|
||||
"version": "0.11.0",
|
||||
"resolved": "https://registry.npmjs.org/selderee/-/selderee-0.11.0.tgz",
|
||||
"integrity": "sha512-5TF+l7p4+OsnP8BCCvSyZiSPc4x4//p5uPwK8TCnVPJYRmU2aYKMpOXvw8zM5a5JvuuCGN1jmsMwuU2W02ukfA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"parseley": "^0.12.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://ko-fi.com/killymxi"
|
||||
}
|
||||
},
|
||||
"node_modules/semver": {
|
||||
"version": "7.5.4",
|
||||
"resolved": "https://registry.npmjs.org/semver/-/semver-7.5.4.tgz",
|
||||
@ -2161,6 +2550,51 @@
|
||||
"resolved": "https://registry.npmjs.org/@msgpack/msgpack/-/msgpack-3.0.0-beta2.tgz",
|
||||
"integrity": "sha512-y+l1PNV0XDyY8sM3YtuMLK5vE3/hkfId+Do8pLo/OPxfxuFAUwcGz3oiiUuV46/aBpwTzZ+mRWVMtlSKbradhw=="
|
||||
},
|
||||
"@msgpackr-extract/msgpackr-extract-darwin-arm64": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-darwin-arm64/-/msgpackr-extract-darwin-arm64-3.0.3.tgz",
|
||||
"integrity": "sha512-QZHtlVgbAdy2zAqNA9Gu1UpIuI8Xvsd1v8ic6B2pZmeFnFcMWiPLfWXh7TVw4eGEZ/C9TH281KwhVoeQUKbyjw==",
|
||||
"optional": true
|
||||
},
|
||||
"@msgpackr-extract/msgpackr-extract-darwin-x64": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-darwin-x64/-/msgpackr-extract-darwin-x64-3.0.3.tgz",
|
||||
"integrity": "sha512-mdzd3AVzYKuUmiWOQ8GNhl64/IoFGol569zNRdkLReh6LRLHOXxU4U8eq0JwaD8iFHdVGqSy4IjFL4reoWCDFw==",
|
||||
"optional": true
|
||||
},
|
||||
"@msgpackr-extract/msgpackr-extract-linux-arm": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-linux-arm/-/msgpackr-extract-linux-arm-3.0.3.tgz",
|
||||
"integrity": "sha512-fg0uy/dG/nZEXfYilKoRe7yALaNmHoYeIoJuJ7KJ+YyU2bvY8vPv27f7UKhGRpY6euFYqEVhxCFZgAUNQBM3nw==",
|
||||
"optional": true
|
||||
},
|
||||
"@msgpackr-extract/msgpackr-extract-linux-arm64": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-linux-arm64/-/msgpackr-extract-linux-arm64-3.0.3.tgz",
|
||||
"integrity": "sha512-YxQL+ax0XqBJDZiKimS2XQaf+2wDGVa1enVRGzEvLLVFeqa5kx2bWbtcSXgsxjQB7nRqqIGFIcLteF/sHeVtQg==",
|
||||
"optional": true
|
||||
},
|
||||
"@msgpackr-extract/msgpackr-extract-linux-x64": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-linux-x64/-/msgpackr-extract-linux-x64-3.0.3.tgz",
|
||||
"integrity": "sha512-cvwNfbP07pKUfq1uH+S6KJ7dT9K8WOE4ZiAcsrSes+UY55E/0jLYc+vq+DO7jlmqRb5zAggExKm0H7O/CBaesg==",
|
||||
"optional": true
|
||||
},
|
||||
"@msgpackr-extract/msgpackr-extract-win32-x64": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-win32-x64/-/msgpackr-extract-win32-x64-3.0.3.tgz",
|
||||
"integrity": "sha512-x0fWaQtYp4E6sktbsdAqnehxDgEc/VwM7uLsRCYWaiGu0ykYdZPiS8zCWdnjHwyiumousxfBm4SO31eXqwEZhQ==",
|
||||
"optional": true
|
||||
},
|
||||
"@selderee/plugin-htmlparser2": {
|
||||
"version": "0.11.0",
|
||||
"resolved": "https://registry.npmjs.org/@selderee/plugin-htmlparser2/-/plugin-htmlparser2-0.11.0.tgz",
|
||||
"integrity": "sha512-P33hHGdldxGabLFjPPpaTxVolMrzrcegejx+0GxjrIb9Zv48D8yAIA/QTDR2dFl7Uz7urX8aX6+5bCZslr+gWQ==",
|
||||
"requires": {
|
||||
"domhandler": "^5.0.3",
|
||||
"selderee": "^0.11.0"
|
||||
}
|
||||
},
|
||||
"@types/linkify-it": {
|
||||
"version": "3.0.2",
|
||||
"resolved": "https://registry.npmjs.org/@types/linkify-it/-/linkify-it-3.0.2.tgz",
|
||||
@ -2183,6 +2617,14 @@
|
||||
"integrity": "sha512-eC4U9MlIcu2q0KQmXszyn5Akca/0jrQmwDRgpAMJai7qBWq4amIQhZyNau4VYGtCeALvW1/NtjzJJ567aZxfKA==",
|
||||
"peer": true
|
||||
},
|
||||
"@vscode/markdown-it-katex": {
|
||||
"version": "1.1.0",
|
||||
"resolved": "https://registry.npmjs.org/@vscode/markdown-it-katex/-/markdown-it-katex-1.1.0.tgz",
|
||||
"integrity": "sha512-9cF2eJpsJOEs2V1cCAoJW/boKz9GQQLvZhNvI030K90z6ZE9lRGc9hDVvKut8zdFO2ObjwylPXXXVYvTdP2O2Q==",
|
||||
"requires": {
|
||||
"katex": "^0.16.4"
|
||||
}
|
||||
},
|
||||
"acorn": {
|
||||
"version": "7.4.1",
|
||||
"resolved": "https://registry.npmjs.org/acorn/-/acorn-7.4.1.tgz",
|
||||
@ -2265,6 +2707,11 @@
|
||||
"resolved": "https://registry.npmjs.org/binary-extensions/-/binary-extensions-2.2.0.tgz",
|
||||
"integrity": "sha512-jDctJ/IVQbZoJykoeHbhXpOlNBqGNcwXJKJog42E5HDPUwQTSdjCHdihjj0DlnheQ7blbT6dHOafNAiS8ooQKA=="
|
||||
},
|
||||
"binary-fuse-filter": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/binary-fuse-filter/-/binary-fuse-filter-1.0.0.tgz",
|
||||
"integrity": "sha512-fKEvB5bWxoh0eCFmAT0IYsB4e8lpIqIfNiVIFeyjzTVQFepek6EYV3iUYuDfw6T+1nIx/7rvwTtEiod8tqc8oA=="
|
||||
},
|
||||
"bindings": {
|
||||
"version": "1.5.0",
|
||||
"resolved": "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz",
|
||||
@ -2283,6 +2730,11 @@
|
||||
"readable-stream": "^3.4.0"
|
||||
}
|
||||
},
|
||||
"boolbase": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/boolbase/-/boolbase-1.0.0.tgz",
|
||||
"integrity": "sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww=="
|
||||
},
|
||||
"braces": {
|
||||
"version": "3.0.2",
|
||||
"resolved": "https://registry.npmjs.org/braces/-/braces-3.0.2.tgz",
|
||||
@ -2403,6 +2855,23 @@
|
||||
"@babel/types": "^7.6.1"
|
||||
}
|
||||
},
|
||||
"css-select": {
|
||||
"version": "5.1.0",
|
||||
"resolved": "https://registry.npmjs.org/css-select/-/css-select-5.1.0.tgz",
|
||||
"integrity": "sha512-nwoRF1rvRRnnCqqY7updORDsuqKzqYJ28+oSMaJMMgOauh3fvwHqMS7EZpIPqK8GL+g9mKxF1vP/ZjSeNjEVHg==",
|
||||
"requires": {
|
||||
"boolbase": "^1.0.0",
|
||||
"css-what": "^6.1.0",
|
||||
"domhandler": "^5.0.2",
|
||||
"domutils": "^3.0.1",
|
||||
"nth-check": "^2.0.1"
|
||||
}
|
||||
},
|
||||
"css-what": {
|
||||
"version": "6.1.0",
|
||||
"resolved": "https://registry.npmjs.org/css-what/-/css-what-6.1.0.tgz",
|
||||
"integrity": "sha512-HTUrgRJ7r4dsZKU6GjmpfRK1O76h97Z8MfS1G0FozR+oF2kG6Vfe8JE6zwrkbxigziPHinCJ+gCPjA9EaBDtRw=="
|
||||
},
|
||||
"dayjs": {
|
||||
"version": "1.10.6",
|
||||
"resolved": "https://registry.npmjs.org/dayjs/-/dayjs-1.10.6.tgz",
|
||||
@ -2421,6 +2890,11 @@
|
||||
"resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz",
|
||||
"integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA=="
|
||||
},
|
||||
"deepmerge": {
|
||||
"version": "4.3.1",
|
||||
"resolved": "https://registry.npmjs.org/deepmerge/-/deepmerge-4.3.1.tgz",
|
||||
"integrity": "sha512-3sUqbMEc77XqpdNO7FRyRog+eW3ph+GYCbj+rK+uYyRMuwsVy0rMiVtPn+QJlKFvWP/1PYpapqYn0Me2knFn+A=="
|
||||
},
|
||||
"delayed-stream": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
|
||||
@ -2436,6 +2910,39 @@
|
||||
"resolved": "https://registry.npmjs.org/doctypes/-/doctypes-1.1.0.tgz",
|
||||
"integrity": "sha1-6oCxBqh1OHdOijpKWv4pPeSJ4Kk="
|
||||
},
|
||||
"dom-serializer": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-2.0.0.tgz",
|
||||
"integrity": "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg==",
|
||||
"requires": {
|
||||
"domelementtype": "^2.3.0",
|
||||
"domhandler": "^5.0.2",
|
||||
"entities": "^4.2.0"
|
||||
}
|
||||
},
|
||||
"domelementtype": {
|
||||
"version": "2.3.0",
|
||||
"resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-2.3.0.tgz",
|
||||
"integrity": "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw=="
|
||||
},
|
||||
"domhandler": {
|
||||
"version": "5.0.3",
|
||||
"resolved": "https://registry.npmjs.org/domhandler/-/domhandler-5.0.3.tgz",
|
||||
"integrity": "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w==",
|
||||
"requires": {
|
||||
"domelementtype": "^2.3.0"
|
||||
}
|
||||
},
|
||||
"domutils": {
|
||||
"version": "3.1.0",
|
||||
"resolved": "https://registry.npmjs.org/domutils/-/domutils-3.1.0.tgz",
|
||||
"integrity": "sha512-H78uMmQtI2AhgDJjWeQmHwJJ2bLPD3GMmO7Zja/ZZh84wkm+4ut+IUnUdRa8uCGX88DiVx1j6FRe1XfxEgjEZA==",
|
||||
"requires": {
|
||||
"dom-serializer": "^2.0.0",
|
||||
"domelementtype": "^2.3.0",
|
||||
"domhandler": "^5.0.3"
|
||||
}
|
||||
},
|
||||
"end-of-stream": {
|
||||
"version": "1.4.4",
|
||||
"resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.4.tgz",
|
||||
@ -2643,6 +3150,29 @@
|
||||
"uglify-js": "^3.5.1"
|
||||
}
|
||||
},
|
||||
"html-to-text": {
|
||||
"version": "9.0.5",
|
||||
"resolved": "https://registry.npmjs.org/html-to-text/-/html-to-text-9.0.5.tgz",
|
||||
"integrity": "sha512-qY60FjREgVZL03vJU6IfMV4GDjGBIoOyvuFdpBDIX9yTlDw0TjxVBQp+P8NvpdIXNJvfWBTNul7fsAQJq2FNpg==",
|
||||
"requires": {
|
||||
"@selderee/plugin-htmlparser2": "^0.11.0",
|
||||
"deepmerge": "^4.3.1",
|
||||
"dom-serializer": "^2.0.0",
|
||||
"htmlparser2": "^8.0.2",
|
||||
"selderee": "^0.11.0"
|
||||
}
|
||||
},
|
||||
"htmlparser2": {
|
||||
"version": "8.0.2",
|
||||
"resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-8.0.2.tgz",
|
||||
"integrity": "sha512-GYdjWKDkbRLkZ5geuHs5NY1puJ+PXwP7+fHPRz06Eirsb9ugf6d8kkXav6ADhcODhFFPMIXyxkxSuMf3D6NCFA==",
|
||||
"requires": {
|
||||
"domelementtype": "^2.3.0",
|
||||
"domhandler": "^5.0.3",
|
||||
"domutils": "^3.0.1",
|
||||
"entities": "^4.4.0"
|
||||
}
|
||||
},
|
||||
"idb": {
|
||||
"version": "7.1.1",
|
||||
"resolved": "https://registry.npmjs.org/idb/-/idb-7.1.1.tgz",
|
||||
@ -2735,6 +3265,11 @@
|
||||
"resolved": "https://registry.npmjs.org/js-stringify/-/js-stringify-1.0.2.tgz",
|
||||
"integrity": "sha1-Fzb939lyTyijaCrcYjCufk6Weds="
|
||||
},
|
||||
"js-xxhash": {
|
||||
"version": "4.0.0",
|
||||
"resolved": "https://registry.npmjs.org/js-xxhash/-/js-xxhash-4.0.0.tgz",
|
||||
"integrity": "sha512-3Q2eIqG6s1KEBBmkj9tGM9lef8LJbuRyTVBdI3GpTnrvtytunjLPO0wqABp5qhtMzfA32jYn1FlnIV7GH1RAHQ=="
|
||||
},
|
||||
"js-yaml": {
|
||||
"version": "3.14.1",
|
||||
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-3.14.1.tgz",
|
||||
@ -2761,11 +3296,31 @@
|
||||
"promise": "^7.0.1"
|
||||
}
|
||||
},
|
||||
"katex": {
|
||||
"version": "0.16.11",
|
||||
"resolved": "https://registry.npmjs.org/katex/-/katex-0.16.11.tgz",
|
||||
"integrity": "sha512-RQrI8rlHY92OLf3rho/Ts8i/XvjgguEjOkO1BEXcU3N8BqPpSzBNwV/G0Ukr+P/l3ivvJUE/Fa/CwbS6HesGNQ==",
|
||||
"requires": {
|
||||
"commander": "^8.3.0"
|
||||
},
|
||||
"dependencies": {
|
||||
"commander": {
|
||||
"version": "8.3.0",
|
||||
"resolved": "https://registry.npmjs.org/commander/-/commander-8.3.0.tgz",
|
||||
"integrity": "sha512-OkTL9umf+He2DZkUq8f8J9of7yL6RJKI24dVITBmNfZBmri9zYZQrKkuXiKhyfPSu8tUhnVBB1iKXevvnlR4Ww=="
|
||||
}
|
||||
}
|
||||
},
|
||||
"kind-of": {
|
||||
"version": "6.0.3",
|
||||
"resolved": "https://registry.npmjs.org/kind-of/-/kind-of-6.0.3.tgz",
|
||||
"integrity": "sha512-dcS1ul+9tmeD95T+x28/ehLgd9mENa3LsvDTtzm3vyBEO7RPptvAD+t44WVXaUjTBRcrpFeFlC8WCruUR456hw=="
|
||||
},
|
||||
"leac": {
|
||||
"version": "0.6.0",
|
||||
"resolved": "https://registry.npmjs.org/leac/-/leac-0.6.0.tgz",
|
||||
"integrity": "sha512-y+SqErxb8h7nE/fiEX07jsbuhrpO9lL8eca7/Y1nuWV2moNlXhyd59iDGcRf6moVyDMbmTNzL40SUyrFU/yDpg=="
|
||||
},
|
||||
"linkify-it": {
|
||||
"version": "5.0.0",
|
||||
"resolved": "https://registry.npmjs.org/linkify-it/-/linkify-it-5.0.0.tgz",
|
||||
@ -2856,6 +3411,29 @@
|
||||
"resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",
|
||||
"integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A=="
|
||||
},
|
||||
"msgpackr": {
|
||||
"version": "1.11.0",
|
||||
"resolved": "https://registry.npmjs.org/msgpackr/-/msgpackr-1.11.0.tgz",
|
||||
"integrity": "sha512-I8qXuuALqJe5laEBYoFykChhSXLikZmUhccjGsPuSJ/7uPip2TJ7lwdIQwWSAi0jGZDXv4WOP8Qg65QZRuXxXw==",
|
||||
"requires": {
|
||||
"msgpackr-extract": "^3.0.2"
|
||||
}
|
||||
},
|
||||
"msgpackr-extract": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/msgpackr-extract/-/msgpackr-extract-3.0.3.tgz",
|
||||
"integrity": "sha512-P0efT1C9jIdVRefqjzOQ9Xml57zpOXnIuS+csaB4MdZbTdmGDLo8XhzBG1N7aO11gKDDkJvBLULeFTo46wwreA==",
|
||||
"optional": true,
|
||||
"requires": {
|
||||
"@msgpackr-extract/msgpackr-extract-darwin-arm64": "3.0.3",
|
||||
"@msgpackr-extract/msgpackr-extract-darwin-x64": "3.0.3",
|
||||
"@msgpackr-extract/msgpackr-extract-linux-arm": "3.0.3",
|
||||
"@msgpackr-extract/msgpackr-extract-linux-arm64": "3.0.3",
|
||||
"@msgpackr-extract/msgpackr-extract-linux-x64": "3.0.3",
|
||||
"@msgpackr-extract/msgpackr-extract-win32-x64": "3.0.3",
|
||||
"node-gyp-build-optional-packages": "5.2.2"
|
||||
}
|
||||
},
|
||||
"mustache": {
|
||||
"version": "4.2.0",
|
||||
"resolved": "https://registry.npmjs.org/mustache/-/mustache-4.2.0.tgz",
|
||||
@ -2892,11 +3470,28 @@
|
||||
"semver": "^7.3.5"
|
||||
}
|
||||
},
|
||||
"node-gyp-build-optional-packages": {
|
||||
"version": "5.2.2",
|
||||
"resolved": "https://registry.npmjs.org/node-gyp-build-optional-packages/-/node-gyp-build-optional-packages-5.2.2.tgz",
|
||||
"integrity": "sha512-s+w+rBWnpTMwSFbaE0UXsRlg7hU4FjekKU4eyAih5T8nJuNZT1nNsskXpxmeqSK9UzkBl6UgRlnKc8hz8IEqOw==",
|
||||
"optional": true,
|
||||
"requires": {
|
||||
"detect-libc": "^2.0.1"
|
||||
}
|
||||
},
|
||||
"normalize-path": {
|
||||
"version": "3.0.0",
|
||||
"resolved": "https://registry.npmjs.org/normalize-path/-/normalize-path-3.0.0.tgz",
|
||||
"integrity": "sha512-6eZs5Ls3WtCisHWp9S2GUy8dqkpGi4BVSz3GaqiE6ezub0512ESztXUwUB6C6IKbQkY2Pnb/mD4WYojCRwcwLA=="
|
||||
},
|
||||
"nth-check": {
|
||||
"version": "2.1.1",
|
||||
"resolved": "https://registry.npmjs.org/nth-check/-/nth-check-2.1.1.tgz",
|
||||
"integrity": "sha512-lqjrjmaOoAnWfMmBPL+XNnynZh2+swxiX3WUE0s4yEHI6m+AwrK2UZOimIRl3X/4QctVqS8AiZjFqyOGrMXb/w==",
|
||||
"requires": {
|
||||
"boolbase": "^1.0.0"
|
||||
}
|
||||
},
|
||||
"object-assign": {
|
||||
"version": "4.1.1",
|
||||
"resolved": "https://registry.npmjs.org/object-assign/-/object-assign-4.1.1.tgz",
|
||||
@ -2918,16 +3513,35 @@
|
||||
"no-case": "^2.2.0"
|
||||
}
|
||||
},
|
||||
"parseley": {
|
||||
"version": "0.12.1",
|
||||
"resolved": "https://registry.npmjs.org/parseley/-/parseley-0.12.1.tgz",
|
||||
"integrity": "sha512-e6qHKe3a9HWr0oMRVDTRhKce+bRO8VGQR3NyVwcjwrbhMmFCX9KszEV35+rn4AdilFAq9VPxP/Fe1wC9Qjd2lw==",
|
||||
"requires": {
|
||||
"leac": "^0.6.0",
|
||||
"peberminta": "^0.9.0"
|
||||
}
|
||||
},
|
||||
"path-parse": {
|
||||
"version": "1.0.7",
|
||||
"resolved": "https://registry.npmjs.org/path-parse/-/path-parse-1.0.7.tgz",
|
||||
"integrity": "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw=="
|
||||
},
|
||||
"peberminta": {
|
||||
"version": "0.9.0",
|
||||
"resolved": "https://registry.npmjs.org/peberminta/-/peberminta-0.9.0.tgz",
|
||||
"integrity": "sha512-XIxfHpEuSJbITd1H3EeQwpcZbTLHc+VVr8ANI9t5sit565tsI4/xK3KWTUFE2e6QiangUkh3B0jihzmGnNrRsQ=="
|
||||
},
|
||||
"picomatch": {
|
||||
"version": "2.3.0",
|
||||
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.0.tgz",
|
||||
"integrity": "sha512-lY1Q/PiJGC2zOv/z391WOTD+Z02bCgsFfvxoXXf6h7kv9o+WmsmzYqrAwY63sNgOxE4xEdq0WyUnXfKeBrSvYw=="
|
||||
},
|
||||
"porter2": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/porter2/-/porter2-1.0.1.tgz",
|
||||
"integrity": "sha512-sK+5ZnjxjFrlJ2CYF1xGFv/CSBAWK0QSfrjzh4hdgjlmR90wTKW9O7a+LLk676ZinianDnMl3Jh2Vrg3OIHWBQ=="
|
||||
},
|
||||
"prebuild-install": {
|
||||
"version": "7.1.1",
|
||||
"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.1.tgz",
|
||||
@ -3158,6 +3772,14 @@
|
||||
"kind-of": "^6.0.0"
|
||||
}
|
||||
},
|
||||
"selderee": {
|
||||
"version": "0.11.0",
|
||||
"resolved": "https://registry.npmjs.org/selderee/-/selderee-0.11.0.tgz",
|
||||
"integrity": "sha512-5TF+l7p4+OsnP8BCCvSyZiSPc4x4//p5uPwK8TCnVPJYRmU2aYKMpOXvw8zM5a5JvuuCGN1jmsMwuU2W02ukfA==",
|
||||
"requires": {
|
||||
"parseley": "^0.12.0"
|
||||
}
|
||||
},
|
||||
"semver": {
|
||||
"version": "7.5.4",
|
||||
"resolved": "https://registry.npmjs.org/semver/-/semver-7.5.4.tgz",
|
||||
|
||||
@ -5,22 +5,31 @@
|
||||
"main": "index.js",
|
||||
"dependencies": {
|
||||
"@msgpack/msgpack": "^3.0.0-beta2",
|
||||
"@vscode/markdown-it-katex": "^1.1.0",
|
||||
"axios": "^1.5.0",
|
||||
"better-sqlite3": "^11.0.0",
|
||||
"binary-fuse-filter": "^1.0.0",
|
||||
"chalk": "^4.1.0",
|
||||
"css-select": "^5.1.0",
|
||||
"dayjs": "^1.8.28",
|
||||
"dom-serializer": "^2.0.0",
|
||||
"domutils": "^3.1.0",
|
||||
"esbuild": "^0.19.6",
|
||||
"fs-extra": "^8.1.0",
|
||||
"gray-matter": "^4.0.2",
|
||||
"handlebars": "^4.7.6",
|
||||
"html-minifier": "^4.0.0",
|
||||
"html-to-text": "^9.0.5",
|
||||
"idb": "^7.1.1",
|
||||
"js-xxhash": "^4.0.0",
|
||||
"markdown-it": "^14.1.0",
|
||||
"markdown-it-anchor": "^8.6.7",
|
||||
"markdown-it-container": "^4.0.0",
|
||||
"markdown-it-footnote": "^3.0.3",
|
||||
"msgpackr": "^1.11.0",
|
||||
"mustache": "^4.0.1",
|
||||
"nanoid": "^2.1.11",
|
||||
"porter2": "^1.0.1",
|
||||
"pug": "^3.0.2",
|
||||
"ramda": "^0.26.1",
|
||||
"sass": "^1.26.8",
|
||||
|
||||
99
src/fts.mjs
Normal file
99
src/fts.mjs
Normal file
@ -0,0 +1,99 @@
|
||||
import * as R from "ramda"
|
||||
import * as htmlToText from "html-to-text"
|
||||
import * as binaryFuseFilter from "binary-fuse-filter"
|
||||
import { xxHash32 as hash } from "js-xxhash"
|
||||
import * as msgpack from "msgpackr"
|
||||
import { BIGRAM_SEPARATOR, FREQUENCY_SEPARATOR, FREQUENCY_THRESHOLDS, tokenize } from "./fts_common.mjs"
|
||||
|
||||
const index = []
|
||||
|
||||
const BIGRAM_INCLUSION = 0.3
|
||||
const URL = /https?:\/\/(www\.)?[-a-zA-Z0-9@:%._\+~#=]{1,256}\.[a-zA-Z0-9()]{1,6}\b([-a-zA-Z0-9()@:%_\+.~#?&//=]*)/g
|
||||
|
||||
export const stripHTML = html => htmlToText.convert(html.replace(URL, " "), {
|
||||
wordwrap: false,
|
||||
selectors: [
|
||||
{ selector: "a", options: { ignoreHref: true } },
|
||||
{ selector: "img", format: "skip" }
|
||||
]
|
||||
})
|
||||
|
||||
export const pushEntry = (sourceType, entry) => {
|
||||
const { html, url, timestamp, title, description, ignoreDescription } = entry
|
||||
// TODO: this puts URLs inline, maybe do something with that
|
||||
const text = (title ?? "") + " " + (!ignoreDescription ? (description && stripHTML(description)) ?? "" : "") + " " + stripHTML(html)
|
||||
const words = tokenize(text)
|
||||
const counts = {}
|
||||
for (const word of words) {
|
||||
counts[word] = (counts[word] ?? 0) + 1
|
||||
}
|
||||
const bigrams = {}
|
||||
for (const [a, b] of R.zip(words, R.drop(1, words))) {
|
||||
bigrams[a + BIGRAM_SEPARATOR + b] = (bigrams[a + BIGRAM_SEPARATOR + b] ?? 0) + 1
|
||||
}
|
||||
index.push({
|
||||
url,
|
||||
timestamp,
|
||||
counts,
|
||||
bigrams,
|
||||
title,
|
||||
description,
|
||||
sourceType
|
||||
})
|
||||
}
|
||||
|
||||
export const build = () => {
|
||||
let totalTerms = 0
|
||||
let totalBigrams = 0
|
||||
const totalBigramCounts = {}
|
||||
const totalTermCounts = {}
|
||||
for (const entry of index) {
|
||||
for (const [bigram, count] of Object.entries(entry.bigrams)) {
|
||||
totalBigramCounts[bigram] = (totalBigramCounts[bigram] ?? 0) + count
|
||||
totalBigrams += count
|
||||
}
|
||||
for (const [word, count] of Object.entries(entry.counts)) {
|
||||
totalTermCounts[word] = (totalTermCounts[word] ?? 0) + count
|
||||
totalTerms += count
|
||||
}
|
||||
}
|
||||
const pmi = (bigram, count) => {
|
||||
const [a, b] = bigram.split(BIGRAM_SEPARATOR, 2)
|
||||
// bigram provides no useful information if term is unique anyway
|
||||
// want ascending order (lower is better)
|
||||
if (totalTermCounts[a] === 1 || !totalTermCounts[b] === 1) { return 0 }
|
||||
return -(count / totalBigrams) / ((totalTermCounts[a] / totalTerms) * (totalTermCounts[b] / totalTerms))
|
||||
}
|
||||
const pmis = new Map(Object.entries(totalBigramCounts).map(([k, v]) => [k, pmi(k, v)]))
|
||||
const records = []
|
||||
for (const entry of index) {
|
||||
const keys = []
|
||||
for (const [word, count] of Object.entries(entry.counts)) {
|
||||
for (const threshold of FREQUENCY_THRESHOLDS) {
|
||||
if (count >= threshold) {
|
||||
keys.push(hash(word + FREQUENCY_SEPARATOR + threshold))
|
||||
}
|
||||
}
|
||||
}
|
||||
const sorted = R.sortBy(x => pmis.get(x), Object.entries(entry.bigrams))
|
||||
for (const [bigram, count] of sorted.slice(0, Math.ceil(keys.length * BIGRAM_INCLUSION))) {
|
||||
keys.push(hash(bigram))
|
||||
}
|
||||
const [filter, err] = binaryFuseFilter.populateBinaryFuse8(keys)
|
||||
if (err) {
|
||||
throw err // Golang...
|
||||
}
|
||||
filter.Fingerprints = new Uint8Array(filter.Fingerprints)
|
||||
records.push({
|
||||
filter,
|
||||
url: entry.url,
|
||||
timestamp: entry.timestamp ? entry.timestamp.format("YYYY-MM-DD") : null,
|
||||
title: entry.title,
|
||||
description: entry.description,
|
||||
sourceType: entry.sourceType
|
||||
})
|
||||
}
|
||||
console.log(`Total terms: ${totalTerms}`)
|
||||
console.log(`Total bigrams: ${totalBigrams}`)
|
||||
return msgpack.pack(records)
|
||||
}
|
||||
81
src/fts_client.mjs
Normal file
81
src/fts_client.mjs
Normal file
@ -0,0 +1,81 @@
|
||||
import { BIGRAM_SEPARATOR, FREQUENCY_SEPARATOR, FREQUENCY_THRESHOLDS, tokenize } from "./fts_common.mjs"
|
||||
import { populateBinaryFuse8 } from "binary-fuse-filter"
|
||||
import { xxHash32 as hash } from "js-xxhash"
|
||||
import { unpack } from "msgpackr"
|
||||
import { drop, zip } from "ramda"
|
||||
|
||||
const SCORE_EXP = 1.9
|
||||
const BIGRAM_FACTOR = 3
|
||||
|
||||
let index = null
|
||||
|
||||
const query = input => {
|
||||
const tokens = tokenize(input)
|
||||
const bigrams = new Set()
|
||||
for (const [a, b] of zip(tokens, drop(1, tokens))) {
|
||||
bigrams.add(a + BIGRAM_SEPARATOR + b)
|
||||
}
|
||||
const cache = {}
|
||||
const hashCached = x => {
|
||||
if (cache[x]) { return cache[x] }
|
||||
const ret = hash(x)
|
||||
cache[x] = ret
|
||||
return ret
|
||||
}
|
||||
const results = []
|
||||
for (const doc of index) {
|
||||
let score = 0
|
||||
for (const token of tokens) {
|
||||
let count = 0
|
||||
for (const frequency of FREQUENCY_THRESHOLDS) {
|
||||
const query = hashCached(token + FREQUENCY_SEPARATOR + frequency)
|
||||
if (doc.filter.contains(query)) {
|
||||
if (count > 0 || frequency == FREQUENCY_THRESHOLDS[0]) {
|
||||
count = frequency
|
||||
}
|
||||
}
|
||||
}
|
||||
if (count > 0) {
|
||||
score += SCORE_EXP ** count
|
||||
}
|
||||
}
|
||||
for (const bigram of bigrams) {
|
||||
if (doc.filter.contains(hashCached(bigram))) {
|
||||
score *= BIGRAM_FACTOR
|
||||
}
|
||||
}
|
||||
if (score > 0) {
|
||||
results.push({
|
||||
score,
|
||||
url: doc.url,
|
||||
title: doc.title,
|
||||
description: doc.description,
|
||||
timestamp: doc.timestamp,
|
||||
sourceType: doc.sourceType
|
||||
})
|
||||
}
|
||||
}
|
||||
results.sort((a, b) => b.score - a.score)
|
||||
return results.slice(0, 10)
|
||||
}
|
||||
|
||||
const loadIndex = async () => {
|
||||
if (index) { return }
|
||||
index = await fetch("/fts.bin").then(x => x.arrayBuffer()).then(x => new Uint8Array(x)).then(x => unpack(x))
|
||||
index.forEach(entry => {
|
||||
const x = entry.filter
|
||||
const newFilter = populateBinaryFuse8([])[0]
|
||||
newFilter.Seed = BigInt(x.Seed)
|
||||
newFilter.SegmentLength = x.SegmentLength
|
||||
newFilter.SegmentLengthMask = x.SegmentLengthMask
|
||||
newFilter.SegmentCount = x.SegmentCount
|
||||
newFilter.SegmentCountLength = x.SegmentCountLength
|
||||
newFilter.len = x.len
|
||||
newFilter.Fingerprints = x.Fingerprints
|
||||
entry.filter = newFilter
|
||||
})
|
||||
}
|
||||
|
||||
await loadIndex()
|
||||
|
||||
export default query
|
||||
18
src/fts_common.mjs
Normal file
18
src/fts_common.mjs
Normal file
@ -0,0 +1,18 @@
|
||||
import * as stemmer from "porter2"
|
||||
|
||||
export const BIGRAM_SEPARATOR = "\x00"
|
||||
export const FREQUENCY_SEPARATOR = "\x01"
|
||||
export const FREQUENCY_THRESHOLDS = [1, 4, 9]
|
||||
const NUMERIC = /^[0-9\.e]+$/
|
||||
|
||||
function segmentWords(text) {
|
||||
if (Intl && Intl.Segmenter) {
|
||||
const segmenter = new Intl.Segmenter("en", { granularity: "word" })
|
||||
return Array.from(segmenter.segment(text)).filter(x => x.isWordLike).filter(x => !NUMERIC.test(x.segment)).map(x => x.segment)
|
||||
} else {
|
||||
// Fallback path
|
||||
return text.split(/[\s\p{P}]+/u).filter(Boolean)
|
||||
}
|
||||
}
|
||||
|
||||
export const tokenize = x => segmentWords(x).map(x => x.toLowerCase()).map(stemmer.stem)
|
||||
@ -67,5 +67,6 @@
|
||||
["bee.png", "https://citrons.xyz/a/memetic-apioform-page.html"],
|
||||
["perceptron.png", "https://en.wikipedia.org/wiki/Perceptron"],
|
||||
["rhombic_dodecahedron.gif", "https://en.wikipedia.org/wiki/Rhombic_dodecahedron"]
|
||||
]
|
||||
],
|
||||
"mycorrhiza": "https://docs.osmarks.net"
|
||||
}
|
||||
|
||||
117
src/index.js
117
src/index.js
@ -22,6 +22,12 @@ const sqlite = require("better-sqlite3")
|
||||
const axios = require("axios")
|
||||
const msgpack = require("@msgpack/msgpack")
|
||||
const esbuild = require("esbuild")
|
||||
const htmlparser2 = require("htmlparser2")
|
||||
const cssSelect = require("css-select")
|
||||
const domSerializer = require("dom-serializer")
|
||||
const domutils = require("domutils")
|
||||
|
||||
const fts = require("./fts.mjs")
|
||||
|
||||
dayjs.extend(customParseFormat)
|
||||
|
||||
@ -33,6 +39,7 @@ const errorPagesDir = path.join(root, "error")
|
||||
const assetsDir = path.join(root, "assets")
|
||||
const outDir = path.join(root, "out")
|
||||
const srcDir = path.join(root, "src")
|
||||
const nodeModules = path.join(root, "node_modules")
|
||||
|
||||
const buildID = nanoid()
|
||||
globalData.buildID = buildID
|
||||
@ -76,7 +83,6 @@ globalData.hashBG = hashBG
|
||||
|
||||
const removeExtension = x => x.replace(/\.[^/.]+$/, "")
|
||||
|
||||
const mdutils = MarkdownIt().utils
|
||||
const renderContainer = (tokens, idx) => {
|
||||
let opening = true
|
||||
if (tokens[idx].type === "container__close") {
|
||||
@ -144,6 +150,7 @@ const renderContainer = (tokens, idx) => {
|
||||
|
||||
const readFile = path => fsp.readFile(path, { encoding: "utf8" })
|
||||
const anchor = require("markdown-it-anchor")
|
||||
const { htmlToText } = require("html-to-text")
|
||||
const md = new MarkdownIt({ html: true })
|
||||
.use(require("markdown-it-container"), "", { render: renderContainer, validate: params => true })
|
||||
.use(require("markdown-it-footnote"))
|
||||
@ -152,6 +159,7 @@ const md = new MarkdownIt({ html: true })
|
||||
symbol: "§"
|
||||
})
|
||||
})
|
||||
.use(require("@vscode/markdown-it-katex").default)
|
||||
const minifyHTML = x => htmlMinifier(x, {
|
||||
collapseWhitespace: true,
|
||||
sortAttributes: true,
|
||||
@ -236,7 +244,15 @@ const processExperiments = async () => {
|
||||
return fse.copy(path.join(subdirectory, file), path.join(out, file))
|
||||
}
|
||||
}))
|
||||
return path.join(out, "index.html")
|
||||
const indexPath = path.join(out, "index.html")
|
||||
fts.pushEntry("experiment", {
|
||||
url: "/" + page.data.slug,
|
||||
title: page.data.title,
|
||||
description: page.data.description,
|
||||
html: page.content,
|
||||
timestamp: dayjs(await fsp.stat(indexPath).then(x => x.mtimeMs))
|
||||
})
|
||||
return indexPath
|
||||
},
|
||||
{ processMeta: meta => {
|
||||
meta.slug = meta.slug || basename
|
||||
@ -254,6 +270,13 @@ const processBlog = async () => {
|
||||
meta.wordCount = page.content.split(/\s+/).map(x => x.trim()).filter(x => x).length
|
||||
meta.haveSidenotes = true
|
||||
meta.content = renderMarkdown(page.content)
|
||||
fts.pushEntry("blog", {
|
||||
html: meta.content,
|
||||
url: "/" + meta.slug,
|
||||
timestamp: meta.updated ?? meta.created,
|
||||
title: meta.title,
|
||||
description: meta.description
|
||||
})
|
||||
return meta
|
||||
})
|
||||
|
||||
@ -332,16 +355,32 @@ const writeCache = (k, v, ts=Date.now()) => {
|
||||
writeCacheStmt.run(k, Buffer.from(enc.buffer, enc.byteOffset, enc.byteLength), ts)
|
||||
}
|
||||
|
||||
const DESC_CUT_LEN = 256
|
||||
const fetchMicroblog = async () => {
|
||||
const cached = readCache("microblog", 60*60*1000)
|
||||
if (cached) {
|
||||
globalData.microblog = cached
|
||||
} else {
|
||||
// We have a server patch which removes the 20-post hardcoded limit.
|
||||
// For some exciting reason microblog.pub does not expose pagination in the *API* components.
|
||||
// This is a workaround.
|
||||
const posts = (await axios({ url: globalData.microblogSource, headers: { "Accept": 'application/ld+json; profile="https://www.w3.org/ns/activitystreams"' } })).data.orderedItems
|
||||
writeCache("microblog", posts)
|
||||
globalData.microblog = posts
|
||||
}
|
||||
|
||||
for (const post of globalData.microblog) {
|
||||
if (!post.object.content) { continue }
|
||||
const desc = fts.stripHTML(post.object.content)
|
||||
fts.pushEntry("microblog", {
|
||||
url: post.object.id,
|
||||
timestamp: dayjs(post.object.published),
|
||||
html: post.object.content,
|
||||
description: desc.length > DESC_CUT_LEN ? desc.slice(0, DESC_CUT_LEN) + "..." : desc,
|
||||
ignoreDescription: true
|
||||
})
|
||||
}
|
||||
|
||||
globalData.microblog = globalData.microblog.slice(0, 6).map((post, i) => minifyHTML(globalData.templates.activitypub({
|
||||
...globalData,
|
||||
permalink: post.object.id,
|
||||
@ -399,13 +438,24 @@ const minifyJSTask = async () => {
|
||||
}
|
||||
|
||||
const compilePageJSTask = async () => {
|
||||
await esbuild.build({
|
||||
entryPoints: [ path.join(srcDir, "page.js") ],
|
||||
bundle: true,
|
||||
outfile: path.join(outAssets, "js/page.js"),
|
||||
minify: true,
|
||||
sourcemap: true
|
||||
})
|
||||
await Promise.all([
|
||||
esbuild.build({
|
||||
entryPoints: [ path.join(srcDir, "page.js") ],
|
||||
bundle: true,
|
||||
outfile: path.join(outAssets, "js/page.js"),
|
||||
minify: true,
|
||||
sourcemap: true,
|
||||
external: ["/assets/js/fts_client.js"]
|
||||
}),
|
||||
esbuild.build({
|
||||
entryPoints: [ path.join(srcDir, "fts_client.mjs") ],
|
||||
bundle: true,
|
||||
outfile: path.join(outAssets, "js/fts_client.js"),
|
||||
minify: true,
|
||||
sourcemap: true,
|
||||
format: "esm"
|
||||
})
|
||||
])
|
||||
}
|
||||
|
||||
const compileServiceWorkerJSTask = async () => {
|
||||
@ -421,18 +471,14 @@ const compileServiceWorkerJSTask = async () => {
|
||||
})
|
||||
}
|
||||
|
||||
const genServiceWorker = async () => {
|
||||
const serviceWorker = mustache.render(await readFile(path.join(assetsDir, "sw.js")), globalData)
|
||||
await minifyJSFile(serviceWorker, "sw.js", path.join(outDir, "sw.js"))
|
||||
}
|
||||
|
||||
const copyAsset = subpath => fse.copy(path.join(assetsDir, subpath), path.join(outAssets, subpath))
|
||||
|
||||
const doImages = async () => {
|
||||
copyAsset("images")
|
||||
copyAsset("titillium-web.woff2")
|
||||
copyAsset("titillium-web-semibold.woff2")
|
||||
copyAsset("miracode.woff2")
|
||||
await Promise.all(["images", "titillium-web.woff2", "titillium-web-semibold.woff2", "miracode.woff2", "misc"].map(subpath => fse.copy(path.join(assetsDir, subpath), path.join(outAssets, subpath))))
|
||||
|
||||
await fse.copy(path.join(nodeModules, "katex", "dist", "fonts"), path.join(outAssets, "fonts"))
|
||||
await fse.copy(path.join(nodeModules, "katex", "dist", "katex.min.css"), path.join(outAssets, "katex.min.css"))
|
||||
|
||||
globalData.images = {}
|
||||
await Promise.all(
|
||||
(await fse.readdir(path.join(assetsDir, "images"), { encoding: "utf-8" })).map(async image => {
|
||||
@ -471,6 +517,37 @@ const doImages = async () => {
|
||||
)
|
||||
}
|
||||
|
||||
const fetchMycorrhiza = async () => {
|
||||
const allPages = await axios({ url: globalData.mycorrhiza + "/list" })
|
||||
const dom = htmlparser2.parseDocument(allPages.data)
|
||||
const urls = cssSelect.selectAll("main > ol a", dom).map(x => x.attribs.href)
|
||||
for (const url of urls) {
|
||||
// TODO: this can run in parallel
|
||||
const page = await axios({ url: globalData.mycorrhiza + url })
|
||||
const dom = htmlparser2.parseDocument(page.data)
|
||||
const title = domutils.innerText(cssSelect.selectAll(".navi-title a, .navi-title span", dom).slice(2))
|
||||
const article = cssSelect.selectOne("main #hypha article", dom)
|
||||
const content = article ? domSerializer.render(article) : ""
|
||||
let description = null
|
||||
if (description = cssSelect.selectOne("meta[property=og:description]", dom)) {
|
||||
description = description.attribs.content
|
||||
}
|
||||
fts.pushEntry("mycorrhiza", {
|
||||
url: globalData.mycorrhiza + url,
|
||||
title,
|
||||
description,
|
||||
html: content,
|
||||
timestamp: null
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
const buildFTS = async () => {
|
||||
console.log(chalk.yellow("Building full-text search index"))
|
||||
const blob = fts.build()
|
||||
await fsp.writeFile(path.join(outDir, "fts.bin"), blob)
|
||||
}
|
||||
|
||||
const tasks = {
|
||||
errorPages: { deps: ["pagedeps"], fn: processErrorPages },
|
||||
templates: { deps: [], fn: loadTemplates },
|
||||
@ -491,7 +568,9 @@ const tasks = {
|
||||
images: { deps: ["assetsDir"], fn: doImages },
|
||||
offlinePage: { deps: ["assetsDir", "pagedeps"], fn: () => applyTemplate(globalData.templates.experiment, path.join(assetsDir, "offline.html"), () => path.join(outAssets, "offline.html"), {}) },
|
||||
assets: { deps: ["manifest", "minifyJS", "serviceWorker", "images", "compilePageJS"] },
|
||||
main: { deps: ["writeBuildID", "index", "errorPages", "assets", "experiments", "blog", "rss"] }
|
||||
main: { deps: ["writeBuildID", "index", "errorPages", "assets", "experiments", "blog", "rss"] },
|
||||
searchIndex: { deps: ["blog", "fetchMicroblog", "fetchMycorrhiza", "experiments"], fn: buildFTS },
|
||||
fetchMycorrhiza: { deps: [], fn: fetchMycorrhiza }
|
||||
}
|
||||
|
||||
const compile = async () => {
|
||||
|
||||
73
src/page.js
73
src/page.js
@ -63,6 +63,14 @@ const hashString = function(str, seed = 0) {
|
||||
const colHash = (str, saturation = 100, lightness = 70) => `hsl(${hashString(str) % 360}, ${saturation}%, ${lightness}%)`
|
||||
window.colHash = colHash
|
||||
|
||||
const e = (cls, parent, content, type="div") => {
|
||||
const element = document.createElement(type)
|
||||
element.classList.add(cls)
|
||||
if (content) { element.appendChild(document.createTextNode(content)) }
|
||||
if (parent) { parent.appendChild(element) }
|
||||
return element
|
||||
}
|
||||
|
||||
// Arbitrary Points code, wrapped in an IIFE to not pollute the global environment much more than it already is
|
||||
window.points = (async () => {
|
||||
const achievementInfo = {
|
||||
@ -242,14 +250,6 @@ window.points = (async () => {
|
||||
},
|
||||
});
|
||||
|
||||
const e = (cls, parent, content) => {
|
||||
const element = document.createElement("div")
|
||||
element.classList.add(cls)
|
||||
if (content) { element.appendChild(document.createTextNode(content)) }
|
||||
if (parent) { parent.appendChild(element) }
|
||||
return element
|
||||
}
|
||||
|
||||
const achievementsContainer = e("achievements", document.body)
|
||||
const displayAchievement = (title, description, conditions, points) => {
|
||||
const elem = e("achievement", achievementsContainer)
|
||||
@ -589,4 +589,59 @@ if (customStyle) {
|
||||
document.head.appendChild(customStyleEl)
|
||||
}
|
||||
window.customStyleEl = customStyleEl
|
||||
window.customStyle = customStyle
|
||||
window.customStyle = customStyle
|
||||
|
||||
const nameMappings = {
|
||||
"blog": "Blog",
|
||||
"microblog": "Microblog",
|
||||
"experiment": "Experiments",
|
||||
"mycorrhiza": "Documentation"
|
||||
}
|
||||
|
||||
// replace login navbar option with search because whatever
|
||||
const loginButton = document.querySelector("nav a:last-of-type")
|
||||
loginButton.href = "#"
|
||||
loginButton.innerText = "Search"
|
||||
loginButton.onclick = async ev => {
|
||||
ev.preventDefault()
|
||||
const query = (await import("/assets/js/fts_client.js")).default
|
||||
const overlay = document.createElement("div")
|
||||
overlay.classList.add("search-overlay")
|
||||
document.body.appendChild(overlay)
|
||||
const input = document.createElement("input")
|
||||
input.type = "text"
|
||||
input.placeholder = "Search"
|
||||
let resultsEl
|
||||
input.oninput = () => {
|
||||
if (resultsEl) {
|
||||
resultsEl.remove()
|
||||
}
|
||||
resultsEl = document.createElement("div")
|
||||
resultsEl.classList.add("search-results")
|
||||
for (const result of query(input.value)) {
|
||||
const item = e("search-result", resultsEl)
|
||||
const titleLine = nameMappings[result.sourceType] + " / " + (result.title ?? result.timestamp)
|
||||
const link = e("search-result-link", item, titleLine, "a")
|
||||
link.setAttribute("href", result.url)
|
||||
if (result.title && result.timestamp) {
|
||||
e("deemph", item, result.timestamp)
|
||||
}
|
||||
if (result.description) {
|
||||
e("description", item, result.description)
|
||||
}
|
||||
item.style.border = `${colHash(result.sourceType)} solid 4px`
|
||||
item.style.background = `${colHash(result.sourceType, 50, 10)}`
|
||||
}
|
||||
overlay.appendChild(resultsEl)
|
||||
}
|
||||
input.onkeydown = ev => {
|
||||
if (ev.key === "Enter" || ev.key === "Backspace") {
|
||||
if (input.value === "") {
|
||||
// quit search mode
|
||||
overlay.remove()
|
||||
}
|
||||
}
|
||||
}
|
||||
overlay.appendChild(input)
|
||||
input.focus()
|
||||
}
|
||||
@ -361,4 +361,29 @@ table
|
||||
text-align: right
|
||||
|
||||
.buttons .button
|
||||
margin: 0.2em
|
||||
margin: 0.2em
|
||||
|
||||
.search-overlay
|
||||
position: fixed
|
||||
overflow: scroll
|
||||
top: 0
|
||||
left: 0
|
||||
bottom: 0
|
||||
right: 0
|
||||
background: rgba(0, 0, 0, 0.9)
|
||||
padding: 2em
|
||||
color: white
|
||||
z-index: 1000
|
||||
input
|
||||
margin: 0.5em
|
||||
font-size: 1.5em
|
||||
border: 4px solid green
|
||||
a
|
||||
color: white
|
||||
.search-results
|
||||
width: 100%
|
||||
.search-result
|
||||
padding: 0.5em
|
||||
margin: 0.5em
|
||||
.description
|
||||
font-style: italic
|
||||
@ -27,6 +27,8 @@ html(lang="en")
|
||||
link(rel="manifest", href="/assets/manifest.webmanifest")
|
||||
link(rel="shortcut icon", href="/assets/images/logo256.png", type="image/png")
|
||||
meta(content=`https://${domain}/assets/images/logo256.png`, property="og:image")
|
||||
if katex
|
||||
link(rel="stylesheet", href="/assets/katex.min.css")
|
||||
style!= css
|
||||
if comments !== "off"
|
||||
script(src=`https://${domain}/rsapi/static/comments.js`, async=true)
|
||||
@ -38,7 +40,7 @@ html(lang="en")
|
||||
.logotext osmarks
|
||||
+nav-item(`/me/`, "About Me")
|
||||
+nav-item(`https://mse.${domain}/`, "Meme Search")
|
||||
+nav-item("https://github.com/osmarks/website", "Contribute")
|
||||
+nav-item(`https://docs.${domain}/random`, "Documentation")
|
||||
+nav-item(`https://b.${domain}`, "Microblog")
|
||||
+nav-item(`https://status.${domain}`, "Status")
|
||||
+nav-item(`https://r.${domain}/login`, "Login")
|
||||
@ -80,4 +82,4 @@ html(lang="en")
|
||||
if comments !== "off"
|
||||
main.isso
|
||||
h2 Comments
|
||||
section(id="comments-wrapper")
|
||||
section(id="comments-wrapper")
|
||||
|
||||
Loading…
Reference in New Issue
Block a user