Also add a test for rednet message sending. Hopefully gives some of the
modem and networking code a little bit of coverage (which is clearly the
same as being right :p).
These are largely copied across from Cobalt's test suite, with some
minor tweaks. It actually exposed one bug in Cobalt, which is pretty
nice.
One interesting thing from the coroutine tests, is that Lua 5.4 (and
one assumes 5.2/5.3) doesn't allow yielding from within the error
handler of xpcall - I rather thought it might.
This doesn't add any of the PUC Lua tests yet - I got a little
distracted.
Also:
- Allow skipping "keyword" tests, in the style of busted. This is
implemented on the Java side for now.
- Fix a bug with os.date("%I", _) not being 2 characters wide.
This probably fails "responsible disclosure", but it's not an RCE and
frankly the whole bug is utterly hilarious so here we are...
It's possible to open a file on a disk drive and continue to read/write
to them after the disk has been removed:
local disk = peripheral.find("drive")
local input = fs.open(fs.combine(disk.getMountPath(), "stream"), "rb")
local output = fs.open(fs.combine(disk.getMountPath(), "stream"), "wb")
disk.ejectDisk()
-- input/output can still be interacted with.
This is pretty amusing, as now it allows us to move the disk somewhere
else and repeat - we've now got a private tunnel which two computers can
use to communicate.
Fixing this is intuitively quite simple - just close any open files
belonging to this mount. However, this is where things get messy thanks
to the wonderful joy of how CC's streams are handled.
As things stand, the filesystem effectively does the following flow::
- There is a function `open : String -> Channel' (file modes are
irrelevant here).
- Once a file is opened, we transform it into some <T extends
Closeable>. This is, for instance, a BufferedReader.
- We generate a "token" (i.e. FileSystemWrapper<T>), which we generate
a week reference to and map it to a tuple of our Channel and T. If
this token is ever garbage collected (someone forgot to call close()
on a file), then we close our T and Channel.
- This token and T are returned to the calling function, which then
constructs a Lua object.
The problem here is that if we close the underlying Channel+T before the
Lua object calls .close(), then it won't know the underlying channel is
closed, and you get some pretty ugly errors (e.g. "Stream Closed"). So
we've moved the "is open" state into the FileSystemWrapper<T>.
The whole system is incredibly complex at this point, and I'd really
like to clean it up. Ideally we could treat the HandleGeneric as the
token instead - this way we could potentially also clean up
FileSystemWrapperMount.
BBut something to play with in the future, and not when it's 10:30pm.
---
All this wall of text, and this isn't the only bug I've found with disks
today :/.
Name a more iconic duo than @SquidDev and over-engineered test
frameworks.
This uses Minecraft's test core[1] plus a home-grown framework to run
tests against computers in-world.
The general idea is:
- Build a structure in game.
- Save the structure to a file. This will be spawned in every time the
test is run.
- Write some code which asserts the structure behaves in a particular
way. This is done in Kotlin (shock, horror), as coroutines give us a
nice way to run asynchronous code while still running on the main
thread.
As with all my testing efforts, I still haven't actually written any
tests! It'd be good to go through some of the historic ones and write
some tests though. Turtle block placing and computer redstone
interactions are probably a good place to start.
[1]: https://www.youtube.com/watch?v=vXaWOJTCYNg
ForgeGradle (probably sensibly) yells at me about doing this. However:
- There's a reasonable number of mods doing this, which establishes
some optimistic precedent.
- The licence update in Aug 2020 now allows you to use them for
"development purposes". I guess source code counts??
- I'm fairly sure this is also compatible with the CCPL - there's an
exception for Minecraft code.
The main motivation for this is to make the Fabric port a little
easier. Hopefully folks (maybe me in the future, we'll see) will no
longer have to deal with mapping hell when merging - only mod loader
hell.
The HTTP filtering system becomes even more complex! Though in this
case, it's pretty minimal, and definitely worth doing.
For instance, the following rule will allow connecting to localhost on
port :8080.
[[http.rules]]
host = "127.0.0.1"
port = 8080
action = "allow"
# Other rules as before.
Closes#540
This allows for configuring the size of computers and pocket computers,
as well as the max size of monitors.
There's several limitations with the current implementation, but it's
still "good enough" for an initial release:
- Turtles cannot be resized.
- GUIs do not scale themselves, so "large" sizes will not render within
the default resolution.
We never added back replacing of ${version} strings, which means that CC
was reporting incorrect version numbers in _HOST, the user agent and
network versions. This meant we would allow connections even on
mismatched versions (#464).
We shift all version handling into ComputerCraftAPI(Impl) - this now
relies on Forge code, so we don't want to run it in emulators.
This moves monitor networking into its own packet, rather than serialising
using NBT. This allows us to be more flexible with how monitors are
serialised.
We now compress terminal data using gzip. This reduces the packet size
of a max-sized-monitor from ~25kb to as little as 100b.
On my test set of images (what I would consider to be the extreme end of
the "reasonable" case), we have packets from 1.4kb bytes up to 12kb,
with a mean of 6kb. Even in the worst case, this is a 2x reduction in
packet size.
While this is a fantastic win for the common case, it is not abuse-proof.
One can create a terminal with high entropy (and so uncompressible). This
will still be close to the original packet size.
In order to prevent any other abuse, we also limit the amount of monitor
data a client can possibly receive to 1MB (configurable).
timetout, max_upload, max_download and max_websocket_message may now be
configured on a domain-by-domain basis. This uses the same system that
we use for the block/allow-list from before:
Example:
[[http.rules]]
host = "*"
action = "allow"
max_upload = 4194304
max_download = 16777216
timeout = 30000
When creating a peripheral or custom Lua object, one must implement two
methods:
- getMethodNames(): String[] - Returns the name of the methods
- callMethod(int, ...): Object[] - Invokes the method using an index in
the above array.
This has a couple of problems:
- It's somewhat unwieldy to use - you need to keep track of array
indices, which leads to ugly code.
- Functions which yield (for instance, those which run on the main
thread) are blocking. This means we need to spawn new threads for
each CC-side yield.
We replace this system with a few changes:
- @LuaFunction annotation: One may annotate a public instance method
with this annotation. This then exposes a peripheral/lua object
method.
Furthermore, this method can accept and return a variety of types,
which often makes functions cleaner (e.g. can return an int rather
than an Object[], and specify and int argument rather than
Object[]).
- MethodResult: Instead of returning an Object[] and having blocking
yields, functions return a MethodResult. This either contains an
immediate return, or an instruction to yield with some continuation
to resume with.
MethodResult is then interpreted by the Lua runtime (i.e. Cobalt),
rather than our weird bodgey hacks before. This means we no longer
spawn new threads when yielding within CC.
- Methods accept IArguments instead of a raw Object array. This has a
few benefits:
- Consistent argument handling - people no longer need to use
ArgumentHelper (as it doesn't exist!), or even be aware of its
existence - you're rather forced into using it.
- More efficient code in some cases. We provide a Cobalt-specific
implementation of IArguments, which avoids the boxing/unboxing when
handling numbers and binary strings.
- Use jacoco for Java-side coverage. Our Java coverage is /terrible
(~10%), as we only really test the core libraries. Still a good thing
to track for regressions though.
- mcfly now tracks Lua side coverage. This works in several stages:
- Replace loadfile to include the whole path
- Add a debug hook which just tracks filename->(lines->count). This
is then submitted to the Java test runner.
- On test completion, we emit a luacov.report.out file.
As the debug hook is inserted by mcfly, this does not include any
computer startup (such as loading apis, or the root of bios.lua),
despite they're executed.
This would be possible to do (for instance, inject a custom header
into bios.lua). However, we're not actually testing any of the
behaviour of startup (aside from "does it not crash"), so I'm not
sure whether to include it or not. Something I'll most likely
re-evaluate.
This is relatively unoptimised right now, but should be efficient enough
for most practical applications.
- Add textutils.json_null. This will be serialized into a literal
`null`. When deserializing, and parse_null is true, this will be
returned instead of a nil.
- Add textutils.unserializeJSON (and textutils.unserializeJSON). This
is a standard compliant JSON parser (hopefully).
- Passing in nbt_style to textutils.unserializeJSON will handle
stringified NBT (no quotes around object keys, numeric suffices). We
don't currently support byte/long/int arrays - something to add in
a future commit.
- Remove stub for table.pack/table.unpack.
- Remove Lua 5.3 bitlib stub. We're not on 5.3, there's no
point emulating it.
- Change peripheral.call to correctly adjust the error level. This is a
terrible hack, but I believe the only good option.
It'd be good to remove load as well, but it's a little more complex due
to our injecting of _ENV.
Closes#363
- Return EPOCH if a zip entry's creation/modification/access time is
missing.
- If a BasicFileAttributes.*Time method returns null, use 0 as our
time. This shouldn't happen, but is a good sanity check.
Fixes#371
- fs.getCapacity just returns the capacity of the current drive, if
available. This will be nil on rom mounts.
- fs.attributes returns an lfs like table of various file attributes.
Currently, this contains:
- access, modification, created: When this file was last accessed,
modified and created. Time is measured in milliseconds since the
epoch, same as os.epoch("utc") and what is accepted by os.date.
- size: Same as fs.getSize
- isDir: same as fs.isDir
Closes#262
- contains now performs a case-insensitive comparison. While this is a
little dubious, it's required for systems like Windows, where foo and
Foo are the same folder.
- Impose a depth limit on copyRecursive. If there are any other cases
where we may try to copy a folder into itself, this should prevent
the computer entirely crashing.
- Remove *Stream methods on IMount/IWritableMount, and make the channel
ones the primary.
- Fix location of AbstractTurtleUpgrade
- Make IComputerAccess.getAvailablePeripheral and .getMainThreadMonitor
mandatory.
- IComputerAccess throws a specialised NotAttachedException
Before it would remain the same across world reloads, and thus would be
out-of-date after leaving the first world. This architecture technically
allows for running multiple servers at once, though that's not going to
matter that soon.
Jonathan Coates