Historically we've used copy-cat to provide a web-based emulator for
running example code on our documentation site. However, copy-cat is
often out-of-date with CC:T, which means example snippets fail when you
try to run them!
This commit vendors in copy-cat (or rather an updated version of it)
into CC:T itself, allowing us to ensure the emulator is always in sync
with the mod.
While the ARCHITECTURE.md documentation goes into a little bit more
detail here, the general implementation is as follows
- In project/src/main we implement the core of the emulator. This
includes a basic reimplementation of some of CC's classes to work on
the web (mostly the HTTP API and ComputerThread), and some additional
code to expose the computers to Javascript.
- This is all then compiled to Javascript using [TeaVM][1] (we actually
use a [personal fork of it][2] as there's a couple of changes I've
not upstreamed yet).
- The Javascript side then pulls in the these compiled classes (and
the CC ROM) and hooks them up to [cc-web-term][3] to display the
actual computer.
- As we're no longer pulling in copy-cat, we can simplify our bundling
system a little - we now just compile to ESM modules directly.
[1]: https://github.com/konsoletyper/teavm
[2]: https://github.com/SquidDev/teavm/tree/squid-patches
[3]: https://github.com/squiddev-cc/cc-web-term
This moves MemoryMount to the main core module, and converts it to be a
"proper" WritableMount. It's still naively implemented - definitely
would be good to flesh out our tests in the future - but enough for what
we need it for.
We also do the following:
- Remove the FileEntry.path variable, and instead pass the path around
as a variable.
- Clean up BinaryReadableHandle to use ByteBuffers in a more idiomatic
way.
- Add a couple more tests to our FS tests. These are in a bit of an odd
place, where we want both Lua tests (for emulator compliance) and
Java tests (for testing different implementations) - something to
think about in the future.
- Add AbstractInMemoryMount, which contains all of ArchiveMount's file
tree logic, but not the caching functionality.
- Convert MemoryMount to inherit from AbstractInMemoryMount.
- Add a helper method to add a file to an AbstractInMemoryMount, and
use that within {Resource,Jar}Mount.
There's definitely more work to be done here - it might be nice to split
FileEntry into separate Directory and File interfaces, or at least make
them slightly more immutable, but that's definitely a future job.
- Add a new WebsocketClient interface, which WebsocketHandle uses for
sending messages and closing. This reduces coupling between Websocket
and WebsocketHandle, which is nice, though admitedly only use for
copy-cat :).
- WebsocketHandle now uses Websocket(Client).isClosed(), rather than
tracking the closed state itself - this makes the class mostly a thin
Lua wrapper over the client, which is nice.
- Convert Options into a record.
- Clarify the behaviour of ws.close() and the websocket_closed event.
Our previous test was incorrect as it called WebsocketHandle.close
(rather than WebsocketHandle.doClose), which had slightly different
semantics in whether the event is queued.
We already use preact for the copy-cat integration, so it makes sense to
use it during the static pass too. This allows us to drop a dependency
on react.
As this is responsible for interrupting computers, we should make sure
its priority is higher than the background threads. It spends most of
its time sleeping, so should be fine.
When the target method is in a different class loader to CC, our
generated method fails, as it cannot find the target class. To get
around that, we create a MethodHandle to the target method, and then
inject that into the generated class (with Java's new dynamic constant
system). We can then invoke the MethodHandle in our generated code,
avoiding any references to the target class/method.
I want to write some tests to check that various packets round-trip
corretly. However, these packets don't (and shouldn't) implement
.equals, and so we need a more reflective(/hacky) way of comparing them.
- Overhaul model loading to work with the new API. This allows for
using the emissive texture system in a more generic way, which is
nice!
- Convert some of our custom models to use Fabric's model hooks (i.e.
emitItemQuads). We don't make use of this right now, but might be
useful for rendering tools with enchantment glints.
Note this does /not/ change any of the turtle block entity rendering
code to use Fabric/Forge's model code. This will be a change we want
to make in the future.
- Some cleanup of our config API. This fixes us printing lots of
warnings when creating a new config file on Fabric (same bug also
occurs on Forge, but that's a loader problem).
- Fix a few warnings
We were generating methods with the original object, rather than the
extra one.
Updated our tests to actually catch this. Unfortunately the only places
we use this interface is in HTTP responses and transferred files,
neither of which show up in the Lua-side tests.
- Document that settings.set doesn't persist values. I think this
closes#1512 - haven't heard back from them.
- Add missing close reasons to the websocket_closed event. Closes#1493.
- Mention what values are preserved by os.queueEvent. This is just the
same as modem.transmit. Closes#1490.
It turns out we don't document the "port" option anywhere, so probably
worth doing a bit of an overhaul here.
- Expand the top-level HTTP rules comment, clarifying how things are
matched and describing each field.
- Improve the comments on the default HTTP rule. We now also describe
the $private rule and its motivation.
- Don't drop/ignore invalid rules. This gets written back to the
original config file, so is very annoying! Instead we now log an
error and convert the rule into a "deny all" rule, which should make
it obvious something is wrong.
- Remove the "force_print" code. This is a relic of before we used
table.pack, and so didn't know how many expressions had been
returned.
- Check the input string is a valid expression separately before
wrapping it in an _echo(...). Fixes#1506.
- Fix mainThread=true methods calling IArguments.escapes too late. This
should be done before scheduling on the main thread, not on the main
thread itself!
- Fix VarargsArguments.escapes not checking that the argument haven't
been closed. This is slightly prone to race conditions, but I don't
think it's worth the overhead of tracking the owning thread.
Maybe when panama and its resource scopes are released.
Thanks Sara for pointing this out!
Slightly irked that none of our tests caught this. Alas.
Also fix a typo in AddressPredicate. Yes, no commit discipline.
- Move the class cache out of Generator into MethodSupplierImpl. This
means we cache class generation globally (that's really expensive!),
but the class -> method list lookup is local.
- Move the global GenericSource/GenericMethod registry out of core,
passing in the list of generic methods to the ComputerContext.
I'm not entirely thrilled by the slight overlap of MethodSupplierImpl and
Generator here, something to clean up in the future.
- Move several interfaces out of `d00.computercraft.core.asm` into a
new `aethods` package. It may make sense to expose this to the
public API in a future commit (possibly part of #1462).
- Add a new MethodSupplier<T> interface, which provides methods to
iterate over all methods exported by an object (either directly, or
including those from ObjectSources).
This interface's concrete implementation (asm.MethodSupplierImpl),
uses Generators and IntCaches as before - we can now make that all
package-private though, which is nice!
- Make the LuaMethod and PeripheralMethod MethodSupplier local to the
ComputerContext. This currently has no effect (the underlying
Generator is still global), but eventually we'll make GenericMethods
non-global, which unlocks the door for #1382.
- Update everything to use this new interface. This is mostly pretty
sensible, but is a little uglier on the MC side (especially in
generic peripherals), as we need to access the global ServerContext.
- Use integer indexes instead of strings (i.e. text, textColour). This
is a tiny bit faster.
- Avoid re-creating tables when clearing.
We're still mostly limited by the VM (slow) and string concatenation
(slow!). Short of having some low-level mutable buffer type, I don't
think we can improve this much :(.
Instead of reporting an error with `.report(f(...))`, we now do
`.report(f, ...)`. This allows consumers to ignore error messages when
not needed, such as when just doing syntax highlighting.
- Provide a helper method for creating threads with a lower priority.
- Use that in our network code (which already used this priority) and
for the computer worker threads (which used the default priority
before). I genuinely thought I did this years ago.
This means the config is no longer stored as static fields, which is a
little cleaner. Would like to move everything else in the future, but
this is a good first step.
We could do this in a more concise manner by wrapping Throwable rather
than reimplementing printStackTrace. However, doing this way allows us
to handle nested exceptions too.
Modrinth proxies images hosted on non-trusted domains through wsrv.nl,
for understandable reasons. However, wsrv.nl blocks tweaked.cc - I'm not
sure why. Instead we reference the image on GH directly, which works!
Also:
- Fix the modrinthSyncBody task pointing to a missing file.
- Update the licenses of a few files, post getting permission from
people. <3 all.
- Add a `timeout` parameter to http request and websocket methods.
- For requests, this sets the connection and read timeout.
- For websockets, this sets the connection and handshake timeout.
- Remove the timeout config option, as this is now specified by user
code.
- Use netty for handling websocket handshakes, meaning we no longer
need to deal with pongs.
In this case, we use Lua's tostring(x) semantics (well, modulo
metamethods), instead of Java's Object.toString(x) call. This ensures
that values are formatted (mostly) consistently between Lua and Java
methods.
- Add IArguments.getStringCoerced, which uses Lua's tostring semantics.
- Add a Coerced<T> wrapper type, which says to use the .getXCoerced
methods. I'm not thrilled about this interface - there's definitely
an argument for using annotations - but this is probably more
consistent for now.
- Convert existing methods to use this call.
Closes#1445
This is a horrible commit: It's a breaking change in a pretty subtle
way, which means it won't be visible while updating. Fortunately I think
the only mod on 1.19.4 is Plethora, but other mods (Mek, Advanced
Peripherals) may be impacted when they update. Sorry!
For some motivation behind the original issue:
The default IArguments implementation (VarargArguments) lazily converts
Lua arguments to Java ones. This is mostly important when passing tables
to Java functions, as we can avoid the conversion entirely if the
function uses IArguments.getTableUnsafe.
However, this lazy conversion breaks down if IArguments is accessed on a
separate thread, as Lua values are not thread-safe. Thus we need to
perform this conversion before the cross-thread sharing occurs.
Now, ideally this would be an implementation detail and entirely
invisible to the user. One approach here would be to only perform this
lazy conversion for methods annotated with @LuaFunction(unsafe=true),
and have it be eager otherwise.
However, the peripheral API gets in the way here, as it means we can no
longer inspect the "actual" method being invoked. And so, alas, this
must leak into the public API.
TLDR: If you're getting weird errors about scope, add an
IArguments.escapes() call before sharing the arguments between threads.
Closes#1384
- Fix several inaccuracies with several files not marking Dan's
authorship. Most of these are new files, where the code was moved from
somewhere else:
- In the public API: IDynamicLuaObject, ILuaAPI, TaskCallbakc,
IDynamicPeripheral, UpgradeBase
- In the ROM: fs, http, require
- Do not mark Dan as an author for entirely new code. This affects
DetailHelpers, DropConsumer, FluidData, InventoryMethods, ItemDetails,
MonitorRenderState, NoTermComputerScreen, Palette, PlatformHelperImpl,
UploadFileMessage, the Terminal tests, and any speaker-related files.
- Relicence many files under the MPL where we have permission to do
so. See #1339 for further details.
Thank you to everyone who has contributed so far! Cannot overstate how
appreciated it is <3.
Trying to play a non-DFPWM (or WAV) file will generate terrible noise,
which in turns generates confused users. Instead, fail to play the audio
file and redirect them to the docs.
- Consult __name in native code too. Closes#1355. This has the added
advantage that unconvertable values (i.e. functions) will now
correctly be reported as their original type, not just nil.
- Fix the error message in cc.expect, so it matches the rest of Lua.
This has been bugging me for years, and I keep forgetting to change
it.
- Several files where @MCJack123 is the exclusive contributor. He has
signed over all contributions to "any OSI-approved license". Thank
you!
- Various the file handle classes: Looking at these again, I don't
think they contain any of the original code.
- Timeouts are now driven by an interrupt system, rather than polling.
While we do not impose memory limits, this should close#1333.
- Update the table library to largely match Lua 5.4:
- Add table.move
- Table methods (with the exception of foreach/foreachi) now use
metamethods (closes#1088).
There's still some remaining quirks (for instance, table.insert
accepts values out-of-bounds), but I think that's fine.
- Cobalt's threaded-coroutine system is gone (load now supports
yielding), so we no longer track coroutine metrics.
- Type errors now use __name. See #1355, though this does not apply to
CC methods (either on the Java or CraftOS side), so is not enough to
resolve it.
See https://github.com/SquidDev/Cobalt/compare/v0.6.0...v0.7.0 for the
full delta.
This adds SPDX license headers to all source code files, following the
REUSE[1] specification. This does not include any asset files (such as
generated JSON files, or textures). While REUSE does support doing so
with ".license" files, for now we define these licences using the
.reuse/dep5 file.
[1]: https://reuse.software/
See the discussion in #1352 - Netty uses the system one by default,
so no sense creating our own.
Also make sure we through the HTTP error every time, not just on the
first failure. Otherwise we get cryptic connection dropped errors.
Given an input like f(x), which is both a valid statement and
expression, both parsers would accept the whole input. However, this was
treated the same as both parsers rejecting the input, resulting in a
crash when trying to print the error.
We now return immediately when any parser accepts the input.
Fixes#1354
If someone had a recursive table (created with an IIFE), then we'd throw
an error inside reserialize. We now catch this error and silently drop
the value.
I'm not thrilled by this behaviour - there's an argument we should
return false instead - but it's consistent with what we currently do.
Closes#1337.
Previously it was possible to access all methods of the multishell
redirect by calling term.current(). This is definitely not intended, as
it leaks all sorts of internals to the user.
Also bump illuaminate - the new version is about twice as fast on my
machine.
Woops!
- Fix the REPL not printing values, as exception.try didn't return
values. It did originally, and then I tried to simplify it >_>
- Change repl_exprs to run an expression and program parser in
parallel, rather than handling the parallelism on the grammar side -
that has a few shift/reduce conflicts which result in bad parse
errors.
- Bump Cobalt to 0.6.0. We now track both line and column numbers of
each bytecode instruction, allowing us to map an error to a concrete
position.
- `loadfile` (and similar functions) now use the full path, rather than
the file name. Cobalt truncates this to 30 characters (rather than
the previous 60) so this should be less noisy.
- The shell, edit and Lua REPL now display the corresponding source
code alongside an error.
Note this is incredibly limited right now - it won't cope with errors
which cross coroutine boundaries. Supporting this is on the roadmap,
but requires some careful API design.
