So very little works, but it compiles and runs.
Things to resolve over the next few days:
- Horrible mappings (should largely be resolved by tomorrow).
- Cannot send extra data over containers - we'll have to see what Forge
does here.
- Turtle models are broken
- No block drops yet - this will largely be cherry-picking whatever I
did on Fabric.
- Weird inventory desyncs (items don't show up initially when
interacting with a CC inventory).
- Probably lots of other things.
- Adds a CheckStyle configuration which is pretty similar to CC's
existing one.
- Add the Gradle license plugin.
- Ensure the existing source code is compatible with these additional
checks.
See #239
- os.time, when given a table, will act the same as PUC Lua - returning
the seconds since the epoch. We preserve the previous string/nil
behaviour though - os.epoch("local") is equivalent to PUC's
os.time().
- os.date will now act accept a string and (optional) time, returning
an appropriate table.
Somewhat resolves the madness which was dan200/ComputerCraft#183, and
hopefully (though probably not) makes @Vexatos happy.
This changes the previous behaviour a little, but hopefully is more
sane:
- Only require the socket to be open when first calling receive. This
means if it closes while receving, you won't get an error.
This behaviour is still not perfect - the socket could have closed,
but the event not reached the user yet, but it's better.
- Listen to websocket_close events while receiving, and return null
should it match ours.
See #201
We were using += instead of =, meaning the budget always grew,
rather than growing while there was still space. As a result, computers
were never correctly rate limited.
Further more, if a computer went into a deficit, we would continue to
increase the budget by a negative amount, exponentially decreasing until
overflowing!
Yes, this is a very embarrassing mistake. I'd been aware that rate
limiting wasn't working as expected for a while, I hadn't realised
the problem would be this stupid.
I promise! The joys of using -SNAPSHOT I guess...
This will now correctly cause orphaned threads to be cleaned up,
reducing the risk of thread saturation.
Previously we just relied on magic int values, which was confusing and
potentially error-prone. We could use EnumFacing, but that's a)
dependent on MC and b) incorrect, as we're referring to local
coordinates.
Look, I originally had this split into several commits, but lots of
other cleanups got mixed in. I then backported some of the cleanups to
1.12, did other tidy ups there, and eventually the web of merges was
unreadable.
Yes, this is a horrible mess, but it's still nicer than it was. Anyway,
changes:
- Flatten everything. For instance, there are now three instances of
BlockComputer, two BlockTurtle, ItemPocketComputer. There's also no
more BlockPeripheral (thank heavens) - there's separate block classes
for each peripheral type.
- Remove pretty much all legacy code. As we're breaking world
compatibility anyway, we can remove all the code to load worlds from
1.4 days.
- The command system is largely rewriten to take advantage of 1.13's
new system. It's very fancy!
- WidgetTerminal now uses Minecraft's "GUI listener" system.
- BREAKING CHANGE: All the codes in keys.lua are different, due to the
move to LWJGL 3. Hopefully this won't have too much of an impact.
I don't want to map to the old key codes on the Java side, as there
always ends up being small but slight inconsistencies. IMO it's
better to make a clean break - people should be using keys rather
than hard coding the constants anyway.
- commands.list now allows fetching sub-commands. The ROM has already
been updated to allow fancy usage such as commands.time.set("noon").
- Turtles, modems and cables can be waterlogged.
Are most of these changes small and petty? Yes. However, IMO they do
make the code more readable. Anyway, a summary of some of the more
interesting changes:
- Expose Abstract*Upgrade classes in the API
- Fix the spelling of Jonathan in the API docs (*shakes fist*)
- Fix bug with printout not working in the offhand.
- Rename any argments/variables accidentally named "m_*", and add an
inspection to prevent it happening again.
- Remove most of the Block*.Properties classes - just inline them in
the parent class.
- Return super.writeToNBT instead of reassigning at the top.
OK, so let's get this out of the way, there's some actual changes mixed
in here too. I'm really sorry:
- Turtles can now not be renamed with unnamed item tags (previously it
would clear the name, this seemed a little unideal).
- commands.getBlock(s)Data will also include NBT.
Now, onto the horror story which is these inspection changes:
- Make a lot of methods static
- Typo fixes
- Make utility classes final + private constructor
- Lots of reformatting (ifs -> ternary, invert control flow, etc...)
- ???
- Profit!
I'm so going to regret this - can pretty much guarantee this is going to
break something.
This uses a similar approach to ComputerThread: executors store how long
they've spent executing tasks. We then use that time to prioritise
executors.
One should note that we use the current runtime at the point of adding
to the queue - external tasks will not contribute towards it until a
later execution.
This effectively acts as a public interface to canExecuteExternal() and
consumeTime(). It's hopefully sufficiently general that we can mess
around with the backend as much as we like in the future.
One thing to note here is that this is based on a polling API, as it's
largely intended for people running work every tick. It would be
possible to adapt this with callbacks for when work is available,
etc..., but that was not needed immediately.
This also removes IComputerOwned, as Plethora no longer needs it.
Unlike ComputerThread, we do not have a single source of tasks, and so
need a smarter way to handle scheduling and rate limiting. This
introduces a cooldown system, which works on both a global and
per-computer level:
Each computer is allowed to do some work for 5ms. If they go over that
budget, then they are marked as "hot", and will not execute work on the
next tick, until they have cooled down. This ensures that _on average_
computers perform at most 5ms of work per tick.
Obviously this is a rather large time span, so we also apply a global
10ms to all computers. This uses the same cooldown principle, meaning we
keep to an average of 10ms, even if we go over budget.
We attempted to simplify this 0bfb7049b0,
but that change now means that minimumVirtualRuntime is not updated. As
a result, new tasks will have a runtime of 0 when the queue is empty.
Oh goodness, this is going to painful to update to 1.13.
We now translate:
- Computer/Disk ID tooltips
- /computercraft descriptions, synopsises and usages. The last of these
may not always be translated when in SMP, as it is sometimes done on
the server, but the alternative would be more complex than I'm happy
with.
- Tracking field names. Might be worth adding descriptions too in the
future.
Also cleanup a couple of other translation keys, so they're more
consistent with Minecraft.
Closes#141
- Share the ILuaContext across all method calls, as well as shifting it
into an anonymous class.
- Move the load/loadstring prefixing into bios.lua
- Be less militant in prefixing chunk names:
- load will no longer do any auto-prefixing.
- loadstring will not prefix when there no chunk name is supplied.
Before we would do `"=" .. supplied_program`, which made no sense.
- Only update all runtimes and the minimum runtime when queuing new
exectors. We only need to update the current executor's runtime.
- Fix overflows when comparing times within TimeoutState.
System.nanotime() may (though probably won't) return negative values.
- Hopefully explain how the scheduler works a little bit.
- Runners would set their active executor before starting resetting the
time, meaning it would be judged as running and terminated.
- Similarly, the cumulative time start was reset to 0, meaning the
computer had been judged to run for an impossibly long time.
- If a computer hit the terminate threshold, but not the hard abort
one, then we'd print the stack trace of the terminated thread - we
now do it before interrupting.
There's still race conditions here when terminating a computer, but
hopefully these changes will mean they never occur under normal
operations (only when a computer has run for far too long).
- Fix the timeout error message displaying utter rot.
- Don't resize the runner array. We don't handle this correctly, so
we shouldn't handle it at all.
- Increment virtualRuntime after a task has executed.
- The computer queue is a priority queue sorted by "virtual runtime".
- Virtual runtime is based on the time this task has executed, divided
by the number of pending tasks.
- We try to execute every task within a given period. Each computer is
allocated a fair share of that period, depending how many tasks are
in the queue. Once a computer has used more than that period, the
computer is paused and the next one resumed.
TimeoutState now introduces a TIMESLICE, which is the maximum period of
time a computer can run before we will look into pausing it.
When we have executed a task for more than this period, and if there are
other computers waiting to execute work, then we will suspend the
machine.
Suspending the machine sets a flag on the ComputerExecutor, and pauses
the "cumulative" time - the time spent handling this particular event.
When resuming the machine, we restart our timer and resume the machine.
Oh goodness, when will it end?
- Computer errors are shown in red.
- Lua machine operations provide whether they succeeded, and an
optional error message (reason bios failed to load, timeout error,
another Lua error), which is then shown to the user.
- Clear the Cobalt "thrown soft abort" flag when resuming, rather than
every n instructions.
- Computers will clear their "should start" flag once the time has
expired, irrespective of whether it turned on or not. Before
computers would immediately restart after shutting down if the flag
had been set much earlier.
Errors within the Lua machine are displayed in a more friendly
When closing a BufferedWriter, we close the underlying writer. As we're
using channels, this is an instance of sun.nio.cs.StreamEncoder. This
will attempt to flush the pending character.
However, if throwing an exception within .write errors, the flush will
fail and so the underlying stream is not closed. This was causing us to
leak file descriptors.
We fix this by introducing ChannelWrappers - this holds the wrapper
object (say, a BufferedWriter) and underlying channel. When closed, we
dispose of the wrapper, and then the channel. You could think of this as
doing a nested try-with-resources, rather than a single one.
Note, this is not related to JDK-6378948 - this occurs in the underlying
stream encoder instead.
- TimeoutState uses nanoseconds rather than milliseconds. While this is
slightly less efficient on Windows, it's a) not the bottleneck of Lua
execution and b) we need a monotonic counter, otherwise we could
fail to terminate computers if the time changes.
- Add an exception handler to all threads.
- Document several classes a little better - I'm not sure how useful
all of these are, but _hopefully_ it'll make the internals a little
more accessible.
- Move state management (turnOn, shutdown, etc...) event handling and
the command queue into a ComputerExecutor
- This means the computer thread now just handles running "work" on
computer executors, rather than managing a separate command queue +
requeuing it.
- Instead of setting soft/hard timeouts on the ILuaMachine, we instead
provide it with a TimeoutState instance. This holds the current abort
flags, which can then be polled within debug hooks.
This means the Lua machine has to do less state management, but also
allows a more flexible implementation of aborts.
- Soft aborts are now handled by the TimeoutState - we track when the
task was started, and now only need to check we're more than 7s since
then.
Note, these timers work with millisecond granularity, rather than
nano, as this invokes substantially less overhead.
- Instead of having n runners being observed with n managers, we now
have n runners and 1 manager (or Monitor).
The runners are now responsible for pulling work from the queue. When
the start to execute a task, they set the time execution commenced.
The monitor then just checks each runner every 0.1s and handles hard
aborts (or killing the thread if need be).
- Rename unload -> close to be a little more consistent
- Make pollAndResetChanged be atomic, so we don't need to aquire a lock
- Get the computer queue from the task owner, rather than a separate
argument.
Ideally we'd add a couple more tests in the future, but this'll do for
now.
The bootstrap class is largely yoinked from CCTweaks-Lua, so is a tad
ugly. It works though.
Prior to this change we would schedule a new task which attached
peripherals on the ComputerThread on the empty task queue. This had a
couple of issues:
- Slow running tasks on the computer thread could result in delays in
peripherals being attached (technically, though rarely seen in
practice).
- Now that the ComputerThread runs tasks at once, there was a race
condition in computers being turned on/off and peripherals being
attached/detached.
Note, while the documentation said that peripherals would only be
(at|de)tached on the computer thread, wired modems would attach on the
server thread, so this was not the case in practice.
One should be aware that peripherals are still detached on the
computer thread, most notably when turning a computer on/off.
This is almost definitely going to break some less well-behaved mods,
and possible some of the well behaved ones. I've tested this on SC, so
it definitely works fine with Computronics and Plethora.
The Computer class currently has several resposiblities such as storing
id/label, managing redstone/peirpherals, handling management of the
computer (on/off/events) and updating the output.
In order to simplify this a little bit, we move our IAPIEnvironment
implementation into a separate file, and store all "world state"
(redstone + peripherals) in there. While we still need to have some
level of updating them within the main Computer instance, it's
substantially simpler.
- Fire close events instead of failure when open websockets error.
- Handle ping events. I thought I was doing this already, but this
requires a WebsocketProtocolHandler. Fixes#118
The latest version of Cobalt has several major changes, which I'm
looking forward to taking advantage of in the coming months:
- The Lua interpreter has been split up from the actual LuaClosure
instance. It now runs multiple functions within one loop, handling
pushing/popping and resuming method calls correctly.
This means we have a theoretically infinite call depth, as we're no
longer bounded by Java's stack size. In reality, this is limited to
32767 (Short.MAX_VALUE), as that's a mostly equivalent to the limits
PUC Lua exposes.
- The stack is no longer unwound in the event of errors. This both
simplifies error handling (not that CC:T needs to care about that)
but also means one can call debug.traceback on a now-dead coroutine
(which is more useful for debugging than using xpcall).
- Most significantly, coroutines are no longer each run on a dedicated
thread. Instead, yielding or resuming throws an exception to unwind
the Java stack and switches to a different coroutine.
In order to preserve compatability with CC's assumption about LuaJ's
threading model (namely that yielding blocks the thread), we also
provide a yieldBlock method (which CC:T consumes). This suspends the
current thread and switches execution to a new thread (see
SquidDev/Cobalt@b5ddf164f1 for more
details). While this does mean we need to use more than 1 thread,
it's still /substantially/ less than would otherwise be needed.
We've been running these changes on SwitchCraft for a few days now and
haven't seen any issues. One nice thing to observe is that the number of
CC thread has gone down from ~1.9k to ~100 (of those, ~70 are dedicated
to running coroutines). Similarly, the server has gone from generating
~15k threads over its lifetime, to ~3k. While this is still a lot, it's
a substantial improvement.
- Convert terminals from a polling-based system to a more event-driven
one: they now accept an onChanged callback, which marks the parent as
dirty.
- Schedule ticks when monitors are marked as dirty.
- Add several missing @Overrides. This has nothing to do with the rest
of the changes, but I'm bad at good git practice.
FileSystemMount was originally added to allow using ReadableByteChannels
instead of InputStreams. However, as zip files do not allow seeking,
there is no benefit of using them over the original JarMount (which we
need to preserve for backwards compatibility).
Instead of maintaining two near-identical mounts, we remove the
FileSystemMount and rewrite the JarMount implementation with several
improvements:
- Rewrite the jar scanning algorithm to be closer to 1.13+'s data pack
mount. This means we no longer require the jar file to have
directories before the file (though this was not a problem in
practice).
- Add all JarMounts to a ReferenceQueue, closing up the ZipFile when
they have been garbage collected (fixes#100).
- Cache the contents of all files for 60 seconds (with some constraints
on size). This allows us to seek on ROM files too (assuming they are
small), by reading the whole thing into memory.
The cache is shared across all mounts, and has a 64MiB limit, and
thus should not have an adverse impact on memory.
- We now error if there are too many websockets, instead of queuing
them up. As these have a more explicit "lifetime", it could be
confusing if http.websocket just blocks indefinitely.
- Fix a CCME when cleaning up resources.
- Move all HTTP tasks to a unified "MonitoredResource" model. This
provides a uniform way of tracking object's lifetimes and disposing
of them when complete.
- Rewrite HTTP requests to use Netty instead of standard Java. This
offers several advantages:
- We have access to more HTTP verbs (mostly PATCH).
- We can now do http -> https redirects.
- We no longer need to spawn in a new thread for each HTTP request.
While we do need to run some tasks off-thread in order to resolve
IPs, it's generally a much shorter task, and so is less likely to
inflate the thread pool.
- Introduce several limits for the http API:
- There's a limit on how many HTTP requests and websockets may exist
at the same time. If the limit is reached, additional ones will be
queued up until pending requests have finished.
- HTTP requests may upload a maximum of 4Mib and download a maximum
of 16Mib (configurable).
- .getResponseCode now returns the status text, as well as the status
code.
- Provide whether a message was binary or text in websocket_message
and handle.receive(). (Fixes#96)
- Provide an optional reason and status code within the websocket_close
event.
Off topic, but also cleanup the file handles a little.
- Remove redundant constructors and super calls
- Standardise naming of texture fields
- Always use postfix notations for loops
- Cleanup several peripheral classes
There's several reasons for this change:
- Try to make ComputerCraft.java less monolithic by moving
functionality into separate module-specific classes.
- Hopefully make the core class less Minecraft dependent, meaning
emulators are a little less dependent on anything outside of /core.
Note we still need /some/ methods in the main ComputerCraft class in
order to maintain backwards compatibility with Plethora and
Computronics.
- Only generate resource pack mounts if the desired directory exists.
- Allow mounting files, as well as directories (fixes#90).
As always, also a wee bit of cleanup to some of the surrounding code.
- Remove a redundant logger
- Provide a getter for the ComputerCraft thread group. This allows us
to monitor child threads within prometheus.
- Replace a deprecated call with a fastutils alternative.
Some mods (*cough* Computronics *cough*) directly access this class,
rather than using the API. We add this back to ensure they still behave
as expected.
Truth be told, I can't really complain, as Plethora also does dodgy
thing with CC internals.
- Keep track of the number of created and destroyed coroutines for each
computer.
- Run coroutines with a thread pool executor, which will keep stale
threads around for 60 seconds. This substantially reduces the
pressure from short-lived coroutines.
- Update to the latest Cobalt version.
Java configured the charset decoders/encoders for streams to REPLACE
malformed characters rather than the default REPORT. It does not do the
same for channels, and so we were catching an IO exception and returning
null.
- Rename openStreamFor* methods to more accurate openChannelFor*
- Fix ArrayByteChannel having an incorrect .position() implementation
Cherry-picked from the PR against dan200/ComputerCraft
- Add an argument to send which controls whether it's a binary message
or not. This is a little ugly, but it's probably more effective than
anything else.
- Fix binary frames not correctly queueing the correct data in the
message event.
Closes#69
This replaces the existing IMount openFor* method with openChannelFor*
ones, which return an appropriate byte channel instead.
As channels are not correctly closed when GCed, we introduce a
FileSystemWrapper. We store a weak reference to this, and when it is
GCed or the file closed, we will remove it from our "open file" set and
ensure any underlying buffers are closed.
While this change may seem a little odd, it does introduce some
benefits:
- We can replace JarMount with a more general FileSystemMount. This
does assume a read-only file system, but could technically be used
for other sources.
- Add support for seekable (binary) handles. We can now look for
instances of SeekableByteChannel and dynamically add it. This works
for all binary filesystem and HTTP streams.
- Rewrite the io library to more accurately emulate PUC Lua's
implementation. We do not correctly implement some elements (most
noticably "*n", but it's a definite improvement.
Effectively shift extracting the computer away from Plethora into CC:T.
Ideally we wouldn't need this at all, but Plethora does some funky
things with tick timings.
See SquidDev-CC/plethora#125
This uses a custom ComputerCraft packet to send chat messages to the
client. When received, we delete all messages of the same category
before sending the new ones.
This avoids cluttering the chat with near-identical messages, and helps
make working with the "individual dump" command easier, as the previous
computer's dump output is deleted.
Also change the max height of the TextTable to 18, so it fits within
Minecraft's default chat limit.
This implements an argument format similar to LuaReqeust, as described
in dan200/ComputerCraft#515. The Lua argument checking code is a little
verbose and repetitive, but I'm not sure how to avoid that - we should
look into improving it in the future.
Closes#21
This allows us to track how much work various peripherals are doing.
This will not work with all systems, such as Plethora, as that has its
own execution system.
The limit was added to prevent people creating arbitrarily large buffers
(for instance, handle.read(2^31) would create a 2GB char array). For
"large" counts, we now read in blocks of size 8192, adding to an
extendable buffer.
