These capture the line and column number of the current position
in the matched text. This is useful for error reporting as well
as indentation checking.
This works by lazily creating an index on first use that stores all
newline character indices in order. We can then do a binary search on
this to get both line number and column number in log(n) time.
This is good enough for most use cases and doesn't slow down the common case at all
- these will not be commonly used patterns in a hot loop so it is not worth to try and
optimize this at all. Constant time look up should be possible but at
the cost of complicating code and slowing down all matching to check for
new lines.
This also changes the api of servers slightly -
in light of having support for ev tasks, it is probably better
to remove the "simple" server code and replace it with some Janet
or remove it all together. While convenient, it has issues with error
handling and rigidity.
Fix outdated code in macex1, such as checking for unquote-splicing,
which no longer exists. Also fix macex1 for quasiquoted tables and
structs. macex1 is not the macro expander used by the compiler, so
these bugs only affected code which called macex manually, such as
the short-fn macro.
Inpsired by the REBOL operators of the same name, these
combinators match bytes up to or inculding a given pattern.
(to patt) is (almost) equalivalent to (any (if-not patt 1)), and
(thru patt) is equivalent to (* (to patt) patt). The one difference
is that if the end of the input is reached and patt is not
matched, the entire pattern does not match.
This makes these operatios use constant stack space rather
than linear stackspace given the size of the inputs. This is important
to prevent certain parser input from causing a stack overflow - in
general, we try to avoid unbounded recursion.
Simpler and more flexible interface, and also lets
us use epoll more easily on linux, which is the most important
plantform to optimize for network performance.
Lazy verification makes it easier to not leave funcenvs
in an invalid state, as well as be more precise with the validation.
We needed to verify the FuncEnvs actually pointed to a stack frame if
they were of the "on-stack" variant. There was some minor checking
before, but it was not enough to prevent func envs from pointing to
memory that was off of the fiber stack, overlapping stack frames, etc.
Using a bitset to indicate which stack values are upvalues, we
can more accurately track when a reference to a stack value
persists after the stack frame exits.
User friendly delimited continuations. While this was doable with
signals before, this does not require C and will play nicely with
existing error handling, defers, and with statements.
While C functions are not re-entrant, signaling from a C function
can be used to implement async returns. When resuming a fiber that
signalled from within a C function, the fiber is started after the
instruction that emitted the signal. The resume argument is used
as the return result from the c function.
This unifies equality and comparison checking. Before, we had
separate functions and vm opcodes for comparing general values vs.
for comparing numbers, where the numberic functions were polymorphic and
had special cases for handling NaNs. By unfiying them, abstract types
can now better integrate with other number types and behave as keys.
For now, the old functions are aliased but will eventually be removed.
This adds several common patterns, which are defined in
boot.janet. This essentially gives more primitive patterns
to work with out of the box.
Fix build when JANET_REDUCED_OS is defined.
True top level unquote currently requires basically double compilation
as it currently stands. Also, implementing such double compilation
looses all source mapping information. This is a compromise
implementation that makes it clear that this works differently than
a true top-level unquote.
The new RNG wraps up state for random number generation, so
one can have many rngs and even marshal and unmarshal them.
Adds math/rng, math/rng-uniform, and math/rng-int.
Also introduce `in` and change semantics for
indexing out of range. This commit enforces stricter
invariants on keys when indexing via a function call
on the data structure, or the new `in` function.
The `get` function is now more lax about keys, and will
not throw an error when a bad key is used for a data structure, instead
returning the default value.
This helps for temporarily setting vars in a safe
manner that is guaranteed not to leave vars in a bad state
(assuming that a fiber does not emit debug or use signal and
is never resumed).
The print family of functions now writes output
to an optional buffer instead of a file bound to :out.
This means output can be more easily captured an redirected.
This should help catch a number of errors, but it
is a very shallow implementation of type checking. It will
catch some common misuses of functions at compile time
rather than runtime.
This will prevent these functions from being run
with empty strings, which usually produces useless
output, as the internal string search algorithm will
never "find" empty strings. This is by design, as it is
not always obvious which empty strings should be found in
the search text.
This should be friendlier to most users. It does, however, mean
we lose range information. However, range information could be
recovered by re-parsing, as janet's grammar is simple enough to do this.
Also adds the janet_lengthv API call. This is
needed because janet_length returns a 32 bit integer, where
as lengthv lets us return larger values (useful for typed arrays).
janet_mcall is an api function that should make it easier to call
a janet method from C code. It shares a similar signature with
janet_call.
A keyword reference only counts as visited if we have
it as cached in the memoized->table, and we know it was
originally referenced from the same grammar table. If these
two conditions are true, then compilation must work correctly.
Also add janet_table_get_ex.
(backmatch [tag?]) is similar to a back reference in regular expressions
(NOT to backwards capture in a peg). It only matches a pattern if
it exactly matches the text of the last capture. It does not consume
or push any captures to the capture stack.
This allows better stacktraces when manually intercepting
signals to clean up resources. Also allows functionality
from Common Lisp's unwind-protect, such as calling cleanup code
while unwindinding the stack, restarting on certain signals, and
just in general having more control over signal and signal propagation.
Also fix a bug encountered while implementing with-resource in the
compiler. Desturcturing arguments that were not the last argument
would often result in bad code generation, as slots used to destructure
the earlier arguments would invalidate the later parameters. This is
fixed by allocating all named parameters before doing any destructuring.
This lets us build a smaller binary. The minimal tested
binary on x86-64 (with -Os, -s, and all options that shrink binary size
turned on) is about 240 kB.
- Allow passing a table to fibers, which make fiber level scope easier.
- Add fiber/getenv, fiber/setenv, dyn, and setdyn
- Remove meta, *env*, and *doc-width*
- Some functions changed dignatures, and no longer take an env
janet_vm_return_reg should only be set when janet_continue
is called. Otherwise, a panic may dump it's error message in
the wrong place, resulting in undefined behavior (often showing
the last return value or worse, segfaulting).
A lot of refactoring larger integer types. Fix a number
of casting errors, but mostly rename things. Also try to
limit use of template-like macros as they bloat the binary
if not used in moderation. We were able to reduce the size of
typed array code as well by using a single view types.
