Prepare for patch release.

Fix typo in docstring

CHANGELOG.md

@@ -1,6 +1,20 @@
 # Changelog
 All notable changes to this project will be documented in this file.
 
+## 1.18.1 - 2021-10-16
+- Fix some documentation typos
+- Fix - Set pipes passed to subprocess to blocking mode.
+- Fix `-r` switch in repl.
+
+## 1.18.0 - 2021-10-10
+- Allow `ev/cancel` to work on already scheduled fibers.
+- Fix bugs with ev/ module.
+- Add optional `base` argument to scan-number
+- Add `-i` flag to janet binary to make it easier to run image files from the command line
+- Remove `thread/` module.
+- Add `(number ...)` pattern to peg for more efficient number parsing using Janet's
+  scan-number function without immediate string creation.
+
 ## 1.17.2 - 2021-09-18
 - Remove include of windows.h from janet.h. This caused issues on certain projects.
 - Fix formatting in doc-format to better handle special characters in signatures.
@@ -8,7 +22,7 @@ All notable changes to this project will be documented in this file.
 - Add optional Makefile target to install jpm as well.
 - Supervisor channels in threads will no longer include a wasteful copy of the fiber in every
   message across a thread.
-- Allow passing a closure to `ev/thead` as well as a whole fiber.
+- Allow passing a closure to `ev/thread` as well as a whole fiber.
 - Allow passing a closure directly to `ev/go` to spawn fibers on the event loop.
 
 ## 1.17.1 - 2021-08-29

Makefile

@@ -67,6 +67,10 @@ ifeq ($(UNAME), Haiku)
 	LDCONFIG:=true
 	LDFLAGS=-Wl,--export-dynamic
 endif
+# For Android (termux)
+ifeq ($(shell uname -o), Android)
+	CLIBS:=$(CLIBS) -landroid-spawn
+endif
 
 $(shell mkdir -p build/core build/c build/boot)
 all: $(JANET_TARGET) $(JANET_LIBRARY) $(JANET_STATIC_LIBRARY) build/janet.h
@@ -120,7 +124,6 @@ JANET_CORE_SOURCES=src/core/abstract.c \
 				   src/core/struct.c \
 				   src/core/symcache.c \
 				   src/core/table.c \
-				   src/core/thread.c \
 				   src/core/tuple.c \
 				   src/core/util.c \
 				   src/core/value.c \
@@ -159,7 +162,7 @@ build/c/janet.c: build/janet_boot src/boot/boot.janet
 ##### Amalgamation #####
 ########################
 
-SONAME=libjanet.so.1.17
+SONAME=libjanet.so.1.18
 
 build/c/shell.c: src/mainclient/shell.c
 	cp $< $@

README.md

@@ -30,6 +30,7 @@ Lua, but smaller than GNU Guile or Python.
 
 ## Features
 
+* Configurable at build time - turn features on or off for a smaller or more featureful build
 * Minimal setup - one binary and you are good to go!
 * First-class closures
 * Garbage collection
@@ -39,6 +40,8 @@ Lua, but smaller than GNU Guile or Python.
 * Mutable and immutable hashtables (table/struct)
 * Mutable and immutable strings (buffer/string)
 * Macros
+* Multithreading
+* Per-thread event loop for efficient evented IO
 * Byte code interpreter with an assembly interface, as well as bytecode verification
 * Tail call Optimization
 * Direct interop with C via abstract types and C functions
@@ -238,23 +241,19 @@ Gitter provides Matrix and irc bridges as well.
 
 ## FAQ
 
-### Why is my terminal spitting out junk when I run the REPL?
-
-Make sure your terminal supports ANSI escape codes. Most modern terminals will
-support these, but some older terminals, Windows consoles, or embedded terminals
-will not. If your terminal does not support ANSI escape codes, run the REPL with
-the `-n` flag, which disables color output. You can also try the `-s` if further issues
-ensue.
-
 ### Where is (favorite feature from other language)?
 
 It may exist, it may not. If you want to propose major language features, go ahead and open an issue, but
 they will likely by closed as "will not implement". Often, such features make one usecase simpler at the expense
 of 5 others by making the language more complicated.
 
-### Where is the example code?
+### Is there a language spec?
 
-In the examples directory.
+There is not currently a spec besides the documentation at https://janet-lang.org.
+
+### Is this Scheme/Common Lisp? Where are the cons cells?
+
+Nope. There are no cons cells here.
 
 ### Is this a Clojure port?
 
@@ -266,14 +265,35 @@ Internally, Janet is not at all like Clojure.
 No. They are immutable arrays and hash tables. Don't try and use them like Clojure's vectors
 and maps, instead they work well as table keys or other identifiers.
 
+### Can I do Object Oriented programming with Janet?
+
+To some extent, yes. However, it is not the recommended method of abstraction, and performance may suffer.
+That said, tables can be used to make mutable objects with inheritance and polymorphism, where object
+methods are implemeted with keywords.
+
+```
+(def Car @{:honk (fn [self msg] (print "car " self " goes " msg)) })
+(def my-car (table/setproto @{} Car))
+(:honk my-car "Beep!")
+```
+
 ### Why can't we add (feature from Clojure) into the core?
 
 Usually, one of a few reasons:
 - Often, it already exists in a different form and the Clojure port would be redundant.
 - Clojure programs often generate a lot of garbage and rely on the JVM to clean it up.
-  Janet does not run on the JVM. We admittedly have a much more primitive GC.
+  Janet does not run on the JVM, and has a more primitive garbage collector.
 - We want to keep the Janet core small. With Lisps, usually a feature can be added as a library
-  without feeling "bolted on", especially when compared to ALGOL like languages.
+  without feeling "bolted on", especially when compared to ALGOL like languages. Adding features
+  to the core also makes it a bit more difficult keep Janet maximally portable.
+
+### Why is my terminal spitting out junk when I run the REPL?
+
+Make sure your terminal supports ANSI escape codes. Most modern terminals will
+support these, but some older terminals, Windows consoles, or embedded terminals
+will not. If your terminal does not support ANSI escape codes, run the REPL with
+the `-n` flag, which disables color output. You can also try the `-s` if further issues
+ensue.
 
 ## Why is it called "Janet"?
 

build_win.bat

@@ -18,8 +18,14 @@
 
 @rem Set compile and link options here
 @setlocal
+
+@rem Example use asan
+@rem set JANET_COMPILE=cl /nologo /Isrc\include /Isrc\conf /c /O2 /W3 /D_CRT_SECURE_NO_WARNINGS /MD /fsanitize=address /Zi
+@rem set JANET_LINK=link /nologo clang_rt.asan_dynamic-x86_64.lib clang_rt.asan_dynamic_runtime_thunk-x86_64.lib
+
 @set JANET_COMPILE=cl /nologo /Isrc\include /Isrc\conf /c /O2 /W3 /D_CRT_SECURE_NO_WARNINGS /MD
 @set JANET_LINK=link /nologo
+
 @set JANET_LINK_STATIC=lib /nologo
 
 @rem Add janet build tag
@@ -81,7 +87,7 @@ exit /b 1
 @echo command prompt.
 exit /b 0
 
-@rem Clean build artifacts 
+@rem Clean build artifacts
 :CLEAN
 del *.exe *.lib *.exp
 rd /s /q build

janet.1

@@ -3,7 +3,7 @@
 janet \- run the Janet language abstract machine
 .SH SYNOPSIS
 .B janet
-[\fB\-hvsrpnqk\fR]
+[\fB\-hvsrpnqik\fR]
 [\fB\-e\fR \fISOURCE\fR]
 [\fB\-E\fR \fISOURCE ...ARGUMENTS\fR]
 [\fB\-l\fR \fIMODULE\fR]
@@ -213,6 +213,11 @@ Precompiles Janet source code into an image, a binary dump that can be efficient
 Source should be a path to the Janet module to compile, and output should be the file path of
 resulting image. Output should usually end with the .jimage extension.
 
+.TP
+.BR \-i
+When this flag is passed, a script passed to the interpreter will be treated as a janet image file
+rather than a janet source file.
+
 .TP
 .BR \-l\ lib
 Import a Janet module before running a script or repl. Multiple files can be loaded

meson.build

@@ -20,7 +20,7 @@
 
 project('janet', 'c',
   default_options : ['c_std=c99', 'build.c_std=c99', 'b_lundef=false', 'default_library=both'],
-  version : '1.17.2')
+  version : '1.18.1')
 
 # Global settings
 janet_path = join_paths(get_option('prefix'), get_option('libdir'), 'janet')
@@ -30,6 +30,7 @@ header_path = join_paths(get_option('prefix'), get_option('includedir'), 'janet'
 cc = meson.get_compiler('c')
 m_dep = cc.find_library('m', required : false)
 dl_dep = cc.find_library('dl', required : false)
+android_spawn_dep = cc.find_library('android-spawn', required : false)
 thread_dep = dependency('threads')
 
 # Link options
@@ -74,7 +75,6 @@ conf.set('JANET_NO_PROCESSES', not get_option('processes'))
 conf.set('JANET_SIMPLE_GETLINE', get_option('simple_getline'))
 conf.set('JANET_EV_NO_EPOLL', not get_option('epoll'))
 conf.set('JANET_EV_NO_KQUEUE', not get_option('kqueue'))
-conf.set('JANET_NO_THREADS', get_option('threads'))
 conf.set('JANET_NO_INTERPRETER_INTERRUPT', not get_option('interpreter_interrupt'))
 if get_option('os_name') != ''
   conf.set('JANET_OS_NAME', get_option('os_name'))
@@ -136,7 +136,6 @@ core_src = [
   'src/core/struct.c',
   'src/core/symcache.c',
   'src/core/table.c',
-  'src/core/thread.c',
   'src/core/tuple.c',
   'src/core/util.c',
   'src/core/value.c',
@@ -162,7 +161,7 @@ mainclient_src = [
 janet_boot = executable('janet-boot', core_src, boot_src,
   include_directories : incdir,
   c_args : '-DJANET_BOOTSTRAP',
-  dependencies : [m_dep, dl_dep, thread_dep],
+  dependencies : [m_dep, dl_dep, thread_dep, android_spawn_dep],
   native : true)
 
 # Build janet.c
@@ -175,7 +174,7 @@ janetc = custom_target('janetc',
     'JANET_PATH', janet_path
   ])
 
-janet_dependencies = [m_dep, dl_dep]
+janet_dependencies = [m_dep, dl_dep, android_spawn_dep]
 if not get_option('single_threaded')
   janet_dependencies += thread_dep
 endif

meson_options.txt

@@ -1,7 +1,6 @@
 option('git_hash', type : 'string', value : 'meson')
 
 option('single_threaded', type : 'boolean', value : false)
-option('threads', type : 'boolean', value : true)
 option('nanbox', type : 'boolean', value : true)
 option('dynamic_modules', type : 'boolean', value : true)
 option('docstrings', type : 'boolean', value : true)

src/boot/boot.janet

@@ -3508,6 +3508,12 @@
 # conditional compilation for reduced os
 (def- getenv-alias (if-let [entry (in root-env 'os/getenv)] (entry :value) (fn [&])))
 
+(defn- run-main
+  [env subargs arg]
+  (if-let [main (get (in env 'main) :value)]
+    (let [thunk (compile [main ;subargs] env arg)]
+      (if (function? thunk) (thunk) (error (thunk :error))))))
+
 (defn cli-main
   `Entrance for the Janet CLI tool. Call this function with the command line
   arguments as an array or tuple of strings to invoke the CLI interface.`
@@ -3515,17 +3521,18 @@
 
   (setdyn :args args)
 
-  (var *should-repl* false)
-  (var *no-file* true)
-  (var *quiet* false)
-  (var *raw-stdin* false)
-  (var *handleopts* true)
-  (var *exit-on-error* true)
-  (var *colorize* true)
-  (var *debug* false)
-  (var *compile-only* false)
-  (var *warn-level* nil)
-  (var *error-level* nil)
+  (var should-repl false)
+  (var no-file true)
+  (var quiet false)
+  (var raw-stdin false)
+  (var handleopts true)
+  (var exit-on-error true)
+  (var colorize true)
+  (var debug-flag false)
+  (var compile-only false)
+  (var warn-level nil)
+  (var error-level nil)
+  (var expect-image false)
 
   (if-let [jp (getenv-alias "JANET_PATH")] (setdyn :syspath jp))
   (if-let [jprofile (getenv-alias "JANET_PROFILE")] (setdyn :profilepath jprofile))
@@ -3555,8 +3562,9 @@
                -k : Compile scripts but do not execute (flycheck)
                -m syspath : Set system path for loading global modules
                -c source output : Compile janet source code into an image
+               -i : Load the script argument as an image file instead of source code
                -n : Disable ANSI color output in the REPL
-               -l lib : Import a module before processing more arguments
+               -l lib : Use a module before processing more arguments
                -w level : Set the lint warning level - default is "normal"
                -x level : Set the lint error level - default is "none"
                -- : Stop handling options
@@ -3564,29 +3572,31 @@
            (os/exit 0)
            1)
      "v" (fn [&] (print janet/version "-" janet/build) (os/exit 0) 1)
-     "s" (fn [&] (set *raw-stdin* true) (set *should-repl* true) 1)
-     "r" (fn [&] (set *should-repl* true) 1)
-     "p" (fn [&] (set *exit-on-error* false) 1)
-     "q" (fn [&] (set *quiet* true) 1)
-     "k" (fn [&] (set *compile-only* true) (set *exit-on-error* false) 1)
-     "n" (fn [&] (set *colorize* false) 1)
+     "s" (fn [&] (set raw-stdin true) (set should-repl true) 1)
+     "r" (fn [&] (set should-repl true) 1)
+     "p" (fn [&] (set exit-on-error false) 1)
+     "q" (fn [&] (set quiet true) 1)
+     "i" (fn [&] (set expect-image true) 1)
+     "k" (fn [&] (set compile-only true) (set exit-on-error false) 1)
+     "n" (fn [&] (set colorize false) 1)
      "m" (fn [i &] (setdyn :syspath (in args (+ i 1))) 2)
      "c" (fn c-switch [i &]
-           (def e (dofile (in args (+ i 1))))
+           (def path (in args (+ i 1)))
+           (def e (dofile path))
            (spit (in args (+ i 2)) (make-image e))
-           (set *no-file* false)
+           (set no-file false)
            3)
-     "-" (fn [&] (set *handleopts* false) 1)
+     "-" (fn [&] (set handleopts false) 1)
      "l" (fn l-switch [i &]
            (import* (in args (+ i 1))
-                    :prefix "" :exit *exit-on-error*)
+                    :prefix "" :exit exit-on-error)
            2)
      "e" (fn e-switch [i &]
-           (set *no-file* false)
+           (set no-file false)
            (eval-string (in args (+ i 1)))
            2)
      "E" (fn E-switch [i &]
-           (set *no-file* false)
+           (set no-file false)
            (def subargs (array/slice args (+ i 2)))
            (def src ~|,(parse (in args (+ i 1))))
            (def thunk (compile src))
@@ -3594,9 +3604,9 @@
              ((thunk) ;subargs)
              (error (get thunk :error)))
            math/inf)
-     "d" (fn [&] (set *debug* true) 1)
-     "w" (fn [i &] (set *warn-level* (get-lint-level i)) 2)
-     "x" (fn [i &] (set *error-level* (get-lint-level i)) 2)
+     "d" (fn [&] (set debug-flag true) 1)
+     "w" (fn [i &] (set warn-level (get-lint-level i)) 2)
+     "x" (fn [i &] (set error-level (get-lint-level i)) 2)
      "R" (fn [&] (setdyn :profilepath nil) 1)})
 
   (defn- dohandler [n i &]
@@ -3608,29 +3618,37 @@
   (def lenargs (length args))
   (while (< i lenargs)
     (def arg (in args i))
-    (if (and *handleopts* (= "-" (string/slice arg 0 1)))
+    (if (and handleopts (= "-" (string/slice arg 0 1)))
       (+= i (dohandler (string/slice arg 1) i))
       (do
-        (set *no-file* false)
-        (def env (make-env))
         (def subargs (array/slice args i))
-        (put env :args subargs)
-        (put env :lint-error *error-level*)
-        (put env :lint-warn *warn-level*)
-        (if *compile-only*
-          (flycheck arg :exit *exit-on-error* :env env)
+        (set no-file false)
+        (if expect-image
           (do
-            (dofile arg :exit *exit-on-error* :env env)
-            (if-let [main (get (in env 'main) :value)]
-              (let [thunk (compile [main ;(tuple/slice args i)] env arg)]
-                (if (function? thunk) (thunk) (error (thunk :error)))))))
+            (def env (load-image (slurp arg)))
+            (put env :args subargs)
+            (put env :lint-error error-level)
+            (put env :lint-warn warn-level)
+            (if debug-flag (put env :debug true))
+            (run-main env subargs arg))
+          (do
+            (def env (make-env))
+            (put env :args subargs)
+            (put env :lint-error error-level)
+            (put env :lint-warn warn-level)
+            (if debug-flag (put env :debug true))
+            (if compile-only
+              (flycheck arg :exit exit-on-error :env env)
+              (do
+                (dofile arg :exit exit-on-error :env env)
+                (run-main env subargs arg)))))
         (set i lenargs))))
 
-  (if (or *should-repl* *no-file*)
+  (if (or should-repl no-file)
     (if
-      *compile-only* (flycheck stdin :source "stdin" :exit *exit-on-error*)
+      compile-only (flycheck stdin :source "stdin" :exit exit-on-error)
       (do
-        (if-not *quiet*
+        (if-not quiet
           (print "Janet " janet/version "-" janet/build " " (os/which) "/" (os/arch) " - '(doc)' for help"))
         (flush)
         (defn getprompt [p]
@@ -3644,15 +3662,15 @@
         (when-let [profile.janet (dyn :profilepath)]
             (def new-env (dofile profile.janet :exit true))
             (merge-module env new-env "" false))
-        (if *debug* (put env :debug true))
-        (def getter (if *raw-stdin* getstdin getline))
+        (if debug-flag (put env :debug true))
+        (def getter (if raw-stdin getstdin getline))
         (defn getchunk [buf p]
           (getter (getprompt p) buf env))
-        (setdyn :pretty-format (if *colorize* "%.20Q" "%.20q"))
-        (setdyn :err-color (if *colorize* true))
-        (setdyn :doc-color (if *colorize* true))
-        (setdyn :lint-error *error-level*)
-        (setdyn :lint-warn *error-level*)
+        (setdyn :pretty-format (if colorize "%.20Q" "%.20q"))
+        (setdyn :err-color (if colorize true))
+        (setdyn :doc-color (if colorize true))
+        (setdyn :lint-error error-level)
+        (setdyn :lint-warn error-level)
         (repl getchunk nil env)))))
 
 ###
@@ -3672,10 +3690,8 @@
         (put into k (x k))))
     into)
 
-  # Deprecate thread library
-  (loop [[k v] :in (pairs root-env)
-         :when (symbol? k)
-         :when (string/has-prefix? "thread/" k)]
+  # Deprecate file/popen
+  (when-let [v (get root-env 'file/popen)]
     (put v :deprecated true))
 
   # Modify root-env to remove private symbols and
@@ -3762,7 +3778,6 @@
      "src/core/struct.c"
      "src/core/symcache.c"
      "src/core/table.c"
-     "src/core/thread.c"
      "src/core/tuple.c"
      "src/core/util.c"
      "src/core/value.c"

src/conf/janetconf.h

@@ -4,10 +4,10 @@
 #define JANETCONF_H
 
 #define JANET_VERSION_MAJOR 1
-#define JANET_VERSION_MINOR 17
-#define JANET_VERSION_PATCH 2
+#define JANET_VERSION_MINOR 18
+#define JANET_VERSION_PATCH 1
 #define JANET_VERSION_EXTRA ""
-#define JANET_VERSION "1.17.2"
+#define JANET_VERSION "1.18.1"
 
 /* #define JANET_BUILD "local" */
 

src/core/array.c

@@ -241,6 +241,11 @@ JANET_CORE_FN(cfun_array_concat,
                 int32_t j, len = 0;
                 const Janet *vals = NULL;
                 janet_indexed_view(argv[i], &vals, &len);
+                if (array->data == vals) {
+                    int32_t newcount = array->count + len;
+                    janet_array_ensure(array, newcount, 2);
+                    janet_indexed_view(argv[i], &vals, &len);
+                }
                 for (j = 0; j < len; j++)
                     janet_array_push(array, vals[j]);
             }

src/core/corelib.c

@@ -137,7 +137,7 @@ static const char *janet_dyncstring(const char *name, const char *dflt) {
     const uint8_t *jstr = janet_unwrap_string(x);
     const char *cstr = (const char *)jstr;
     if (strlen(cstr) != (size_t) janet_string_length(jstr)) {
-        janet_panicf("string %v contains embedded 0s");
+        janet_panicf("string %v contains embedded 0s", x);
     }
     return cstr;
 }
@@ -398,15 +398,21 @@ JANET_CORE_FN(janet_core_is_abstract,
 }
 
 JANET_CORE_FN(janet_core_scannumber,
-              "(scan-number str)",
-              "Parse a number from a byte sequence an return that number, either and integer "
+              "(scan-number str &opt base)",
+              "Parse a number from a byte sequence and return that number, either an integer "
               "or a real. The number "
               "must be in the same format as numbers in janet source code. Will return nil "
-              "on an invalid number.") {
+              "on an invalid number. Optionally provide a base - if a base is provided, no "
+              "radix specifier is expected at the beginning of the number.") {
     double number;
-    janet_fixarity(argc, 1);
+    janet_arity(argc, 1, 2);
     JanetByteView view = janet_getbytes(argv, 0);
-    if (janet_scan_number(view.bytes, view.len, &number))
+    int32_t base = janet_optinteger(argv, argc, 1, 0);
+    int valid = base == 0 || (base >= 2 && base <= 36);
+    if (!valid) {
+        janet_panicf("expected base between 2 and 36, got %d", base);
+    }
+    if (janet_scan_number_base(view.bytes, view.len, base, &number))
         return janet_wrap_nil();
     return janet_wrap_number(number);
 }
@@ -979,9 +985,6 @@ static void janet_load_libs(JanetTable *env) {
 #ifdef JANET_INT_TYPES
     janet_lib_inttypes(env);
 #endif
-#ifdef JANET_THREADS
-    janet_lib_thread(env);
-#endif
 #ifdef JANET_EV
     janet_lib_ev(env);
 #endif

src/core/ev.c

@@ -75,6 +75,7 @@ typedef struct {
     JanetFiber *fiber;
     Janet value;
     JanetSignal sig;
+    uint32_t expected_sched_id; /* If the fiber has been rescheduled this loop, don't run first scheduling. */
 } JanetTask;
 
 /* Wrap return value by pairing it with the callback used to handle it
@@ -447,10 +448,7 @@ const JanetAbstractType janet_stream_type = {
 
 /* Register a fiber to resume with value */
 void janet_schedule_signal(JanetFiber *fiber, Janet value, JanetSignal sig) {
-    if (fiber->flags & JANET_FIBER_FLAG_SCHEDULED) return;
-    fiber->flags |= JANET_FIBER_FLAG_SCHEDULED;
-    fiber->sched_id++;
-    JanetTask t = { fiber, value, sig };
+    JanetTask t = { fiber, value, sig, ++fiber->sched_id };
     janet_q_push(&janet_vm.spawn, &t, sizeof(t));
 }
 
@@ -1206,9 +1204,9 @@ JanetFiber *janet_loop1(void) {
 
     /* Run scheduled fibers */
     while (janet_vm.spawn.head != janet_vm.spawn.tail) {
-        JanetTask task = {NULL, janet_wrap_nil(), JANET_SIGNAL_OK};
+        JanetTask task = {NULL, janet_wrap_nil(), JANET_SIGNAL_OK, 0};
         janet_q_pop(&janet_vm.spawn, &task, sizeof(task));
-        task.fiber->flags &= ~JANET_FIBER_FLAG_SCHEDULED;
+        if (task.expected_sched_id != task.fiber->sched_id) continue;
         Janet res;
         JanetSignal sig = janet_continue_signal(task.fiber, task.value, &res, task.sig);
         void *sv = task.fiber->supervisor_channel;
@@ -1513,8 +1511,8 @@ void janet_loop1_impl(int has_timeout, JanetTimestamp timeout) {
             JanetStream *stream = p;
             int mask = events[i].events;
             JanetListenerState *state = stream->state;
-            state->event = events + i;
             while (NULL != state) {
+                state->event = events + i;
                 JanetListenerState *next_state = state->_next;
                 JanetAsyncStatus status1 = JANET_ASYNC_STATUS_NOT_DONE;
                 JanetAsyncStatus status2 = JANET_ASYNC_STATUS_NOT_DONE;
@@ -1576,32 +1574,19 @@ void janet_ev_deinit(void) {
  * NetBSD uses intptr_t while others use void * for .udata */
 #define EV_SETx(ev, a, b, c, d, e, f) EV_SET((ev), (a), (b), (c), (d), (e), ((__typeof__((ev)->udata))(f)))
 #define JANET_KQUEUE_TF (EV_ADD | EV_ENABLE | EV_CLEAR | EV_ONESHOT)
-
-/* NOTE:
- * NetBSD doesn't like intervals less than 1 millisecond so lets make that the
- * default anywhere JANET_KQUEUE_TS will be used. */
-#ifdef __NetBSD__
-#define JANET_KQUEUE_MIN_INTERVAL 1
-#else
 #define JANET_KQUEUE_MIN_INTERVAL 0
-#endif
 
-#ifdef __FreeBSD__
-#define JANET_KQUEUE_TS(timestamp) (timestamp)
-#else
 /* NOTE:
- * NetBSD and OpenBSD expect things are always intervals, so fake that we have
- * abstime capability by changing how a timestamp is used in all kqueue calls
- * and defining absent macros. Additionally NetBSD expects intervals be
- * greater than 1 millisecond, so correct all intervals to be at least 1
- * millisecond under NetBSD. */
-JanetTimestamp fix_interval(const JanetTimestamp ts) {
+ * NetBSD and OpenBSD expect things are always intervals, and FreeBSD doesn't
+ * like an ABSTIME in the past so just use intervals always. Introduces a
+ * calculation to determine the minimum timeout per timeout requested of
+ * kqueue. Also note that NetBSD doesn't accept timeout intervals less than 1
+ * millisecond, so correct all intervals on that platform to be at least 1
+ * millisecond.*/
+JanetTimestamp to_interval(const JanetTimestamp ts) {
     return ts >= JANET_KQUEUE_MIN_INTERVAL ? ts : JANET_KQUEUE_MIN_INTERVAL;
 }
-#define JANET_KQUEUE_TS(timestamp) (fix_interval((timestamp - ts_now())))
-#define NOTE_MSECONDS 0
-#define NOTE_ABSTIME 0
-#endif
+#define JANET_KQUEUE_INTERVAL(timestamp) (to_interval((timestamp - ts_now())))
 
 
 /* TODO: make this available be we using kqueue or epoll, instead of
@@ -1614,6 +1599,12 @@ static JanetTimestamp ts_now(void) {
     return res;
 }
 
+/* NOTE: Assumes Janet's timestamp precision is in milliseconds. */
+static void timestamp2timespec(struct timespec *t, JanetTimestamp ts) {
+    t->tv_sec = ts == 0 ? 0 : ts / 1000;
+    t->tv_nsec = ts == 0 ? 0 : (ts % 1000) * 1000000;
+}
+
 void add_kqueue_events(const struct kevent *events, int length) {
     /* NOTE: Status should be equal to the amount of events added, which isn't
      * always known since deletions or modifications occur. Can't use the
@@ -1680,45 +1671,46 @@ static void janet_unlisten(JanetListenerState *state, int is_gc) {
     janet_unlisten_impl(state, is_gc);
 }
 
-#define JANET_KQUEUE_TIMER_IDENT 1
 #define JANET_KQUEUE_MAX_EVENTS 64
 
 void janet_loop1_impl(int has_timeout, JanetTimestamp timeout) {
-    /* Construct our timer which is a definite time on the clock, not an
-     * interval, in milliseconds as that is `JanetTimestamp`'s precision. */
-    struct kevent timer;
-    if (janet_vm.timer_enabled || has_timeout) {
-        EV_SETx(&timer,
-                JANET_KQUEUE_TIMER_IDENT,
-                EVFILT_TIMER,
-                JANET_KQUEUE_TF,
-                NOTE_MSECONDS | NOTE_ABSTIME,
-                JANET_KQUEUE_TS(timeout), &janet_vm.timer);
-        add_kqueue_events(&timer, 1);
-    }
-    janet_vm.timer_enabled = has_timeout;
-
     /* Poll for events */
+    /* NOTE:
+     * We calculate the timeout interval per iteration. When the interval
+     * drops to 0 or negative, we effect a timeout of 0. Effecting a timeout
+     * of infinity will not work and could make other fibers with timeouts
+     * miss their timeouts if we did so.
+     * JANET_KQUEUE_INTERVAL insures we have a timeout of no less than 0. */
     int status;
+    struct timespec ts;
     struct kevent events[JANET_KQUEUE_MAX_EVENTS];
     do {
-        status = kevent(janet_vm.kq, NULL, 0, events, JANET_KQUEUE_MAX_EVENTS, NULL);
+        if (janet_vm.timer_enabled || has_timeout) {
+            timestamp2timespec(&ts, JANET_KQUEUE_INTERVAL(timeout));
+            status = kevent(janet_vm.kq, NULL, 0, events,
+                            JANET_KQUEUE_MAX_EVENTS, &ts);
+        } else {
+            status = kevent(janet_vm.kq, NULL, 0, events,
+                            JANET_KQUEUE_MAX_EVENTS, NULL);
+        }
     } while (status == -1 && errno == EINTR);
     if (status == -1)
         JANET_EXIT("failed to poll events");
 
+    /* Make sure timer is set accordingly. */
+    janet_vm.timer_enabled = has_timeout;
+
     /* Step state machines */
     for (int i = 0; i < status; i++) {
         void *p = (void *) events[i].udata;
-        if (&janet_vm.timer == p) {
-            /* Timer expired, ignore */;
-        } else if (janet_vm.selfpipe == p) {
+        if (janet_vm.selfpipe == p) {
             /* Self-pipe handling */
             janet_ev_handle_selfpipe();
         } else {
             JanetStream *stream = p;
             JanetListenerState *state = stream->state;
-            if (NULL != state) {
+            while (NULL != state) {
+                JanetListenerState *next_state = state->_next;
                 state->event = events + i;
                 JanetAsyncStatus statuses[4];
                 for (int i = 0; i < 4; i++)
@@ -1739,6 +1731,8 @@ void janet_loop1_impl(int has_timeout, JanetTimestamp timeout) {
                         statuses[2] == JANET_ASYNC_STATUS_DONE ||
                         statuses[3] == JANET_ASYNC_STATUS_DONE)
                     janet_unlisten(state, 0);
+
+                state = next_state;
             }
         }
     }
@@ -1750,16 +1744,9 @@ void janet_ev_init(void) {
     janet_vm.kq = kqueue();
     janet_vm.timer_enabled = 0;
     if (janet_vm.kq == -1) goto error;
-    struct kevent events[2];
-    /* Don't use JANET_KQUEUE_TS here, as even under FreeBSD we use intervals
-     * here. */
-    EV_SETx(&events[0],
-            JANET_KQUEUE_TIMER_IDENT,
-            EVFILT_TIMER,
-            JANET_KQUEUE_TF,
-            NOTE_MSECONDS, JANET_KQUEUE_MIN_INTERVAL, &janet_vm.timer);
-    EV_SETx(&events[1], janet_vm.selfpipe[0], EVFILT_READ, EV_ADD | EV_ENABLE, 0, 0, janet_vm.selfpipe);
-    add_kqueue_events(events, 2);
+    struct kevent event;
+    EV_SETx(&event, janet_vm.selfpipe[0], EVFILT_READ, EV_ADD | EV_ENABLE, 0, 0, janet_vm.selfpipe);
+    add_kqueue_events(&event, 1);
     return;
 error:
     JANET_EXIT("failed to initialize event loop");
@@ -2235,7 +2222,7 @@ JanetAsyncStatus ev_machine_read(JanetListenerState *s, JanetAsyncEvent event) {
         case JANET_ASYNC_EVENT_READ: {
             JanetBuffer *buffer = state->buf;
             int32_t bytes_left = state->bytes_left;
-            int32_t read_limit = bytes_left > 4096 ? 4096 : bytes_left;
+            int32_t read_limit = state->is_chunk ? (bytes_left > 4096 ? 4096 : bytes_left) : bytes_left;
             janet_buffer_extra(buffer, read_limit);
             ssize_t nread;
 #ifdef JANET_NET
@@ -2571,15 +2558,16 @@ void janet_ev_sendto_string(JanetStream *stream, JanetString str, void *dest, in
 static volatile long PipeSerialNumber;
 #endif
 
+/*
+ * mode = 0: both sides non-blocking.
+ * mode = 1: only read side non-blocking: write side sent to subprocess
+ * mode = 2: only write side non-blocking: read side sent to subprocess
+ */
 int janet_make_pipe(JanetHandle handles[2], int mode) {
 #ifdef JANET_WINDOWS
     /*
      * On windows, the built in CreatePipe function doesn't support overlapped IO
      * so we lift from the windows source code and modify for our own version.
-     *
-     * mode = 0: both sides non-blocking.
-     * mode = 1: only read side non-blocking: write side sent to subprocess
-     * mode = 2: only write side non-blocking: read side sent to subprocess
      */
     JanetHandle shandle, chandle;
     UCHAR PipeNameBuffer[MAX_PATH];
@@ -2629,10 +2617,9 @@ int janet_make_pipe(JanetHandle handles[2], int mode) {
     }
     return 0;
 #else
-    (void) mode;
     if (pipe(handles)) return -1;
-    if (fcntl(handles[0], F_SETFL, O_NONBLOCK)) goto error;
-    if (fcntl(handles[1], F_SETFL, O_NONBLOCK)) goto error;
+    if (mode != 2 && fcntl(handles[0], F_SETFL, O_NONBLOCK)) goto error;
+    if (mode != 1 && fcntl(handles[1], F_SETFL, O_NONBLOCK)) goto error;
     return 0;
 error:
     close(handles[0]);
@@ -2903,7 +2890,7 @@ JANET_CORE_FN(cfun_ev_deadline,
 
 JANET_CORE_FN(cfun_ev_cancel,
               "(ev/cancel fiber err)",
-              "Cancel a suspended fiber in the event loop. Differs from cancel in that it returns the canceled fiber immediately") {
+              "Cancel a suspended fiber in the event loop. Differs from cancel in that it returns the canceled fiber immediately.") {
     janet_fixarity(argc, 2);
     JanetFiber *fiber = janet_getfiber(argv, 0);
     Janet err = argv[1];

src/core/fiber.h

@@ -47,7 +47,6 @@
 #define JANET_FIBER_MASK_USER 0x3FF0
 
 #define JANET_FIBER_STATUS_MASK 0x3F0000
-#define JANET_FIBER_FLAG_SCHEDULED 0x800000
 #define JANET_FIBER_RESUME_SIGNAL 0x400000
 #define JANET_FIBER_STATUS_OFFSET 16
 

src/core/net.c

@@ -74,6 +74,15 @@ const JanetAbstractType janet_address_type = {
 #endif
 #endif
 
+/* maximum number of bytes in a socket address host (post name resolution) */
+#ifdef JANET_WINDOWS
+#define SA_ADDRSTRLEN (INET6_ADDRSTRLEN + 1)
+typedef unsigned short in_port_t;
+#else
+#define JANET_SA_MAX(a, b) (((a) > (b))? (a) : (b))
+#define SA_ADDRSTRLEN JANET_SA_MAX(INET6_ADDRSTRLEN + 1, (sizeof ((struct sockaddr_un *)0)->sun_path) + 1)
+#endif
+
 static JanetStream *make_stream(JSock handle, uint32_t flags);
 
 /* We pass this flag to all send calls to prevent sigpipe */
@@ -314,12 +323,13 @@ static struct addrinfo *janet_get_addrinfo(Janet *argv, int32_t offset, int sock
  */
 
 JANET_CORE_FN(cfun_net_sockaddr,
-              "(net/address host port &opt type)",
+              "(net/address host port &opt type multi)",
               "Look up the connection information for a given hostname, port, and connection type. Returns "
               "a handle that can be used to send datagrams over network without establishing a connection. "
               "On Posix platforms, you can use :unix for host to connect to a unix domain socket, where the name is "
               "given in the port argument. On Linux, abstract "
-              "unix domain sockets are specified with a leading '@' character in port.") {
+              "unix domain sockets are specified with a leading '@' character in port. If `multi` is truthy, will "
+              "return all address that match in an array instead of just the first.") {
     janet_arity(argc, 2, 4);
     int socktype = janet_get_sockettype(argv, argc, 2);
     int is_unix = 0;
@@ -631,268 +641,92 @@ JANET_CORE_FN(cfun_net_listen,
     }
 }
 
-/* Definitions from:
- *   https://github.com/wahern/cqueues/blog/master/src/lib/socket.h
- *     SO_MAX, SA_PORT_NONE, SO_MIN, SA_ADDRSTRLEN, sa_ntoa, sa_family,
- *     sa_port */
-#define SO_MAX(a, b) (((a) > (b))? (a) : (b))
-#define SA_PORT_NONE (&(in_port_t){ 0 })
-#define SO_MIN(a, b) (((a) < (b))? (a) : (b))
-#ifndef JANET_WINDOWS
-#define SA_ADDRSTRLEN SO_MAX(INET6_ADDRSTRLEN, (sizeof ((struct sockaddr_un *)0)->sun_path) + 1)
-#else
-#define SA_ADDRSTRLEN (INET6_ADDRSTRLEN + 1)
-#endif
-#define sa_ntoa(sa)  sa_ntoa_((char [SA_ADDRSTRLEN]){ 0 }, SA_ADDRSTRLEN, (sa))
-#define sa_family(...) sa_family(__VA_ARGS__)
-#define sa_port(...) sa_port(__VA_ARGS__)
-#ifdef JANET_WINDOWS
-typedef short sa_family_t; /* added to silence warnings */
-typedef unsigned short in_port_t; /* added to silence warnings */
-#endif
+/* Types of socket's we need to deal with - relevant type puns below.
+struct sockaddr *sa;           // Common base structure
+struct sockaddr_storage *ss;   // Size of largest socket address type
+struct sockaddr_in *sin;       // IPv4 address + port
+struct sockaddr_in6 *sin6;     // IPv6 address + port
+struct sockaddr_un *sun;       // Unix Domain Socket Address
+*/
 
-/* Definition from:
- *   https://github.com/wahern/cqueues/blog/master/src/lib/socket.h */
-union sockaddr_arg {
-    struct sockaddr *sa;
-    const struct sockaddr *c_sa;
-    struct sockaddr_storage *ss;
-    struct sockaddr_storage *c_ss;
-    struct sockaddr_in *sin;
-    struct sockaddr_in *c_sin;
-    struct sockaddr_in6 *sin6;
-    struct sockaddr_in6 *c_sin6;
-#ifndef JANET_WINDOWS
-    struct sockaddr_un *sun;
-#endif
-    struct sockaddr_un *c_sun;
-    union sockaddr_any *any;
-    union sockaddr_any *c_any;
-
-    void *ptr;
-    void *c_ptr;
-};
-
-/* Definition from:
- *   https://github.com/wahern/cqueues/blog/master/src/lib/socket.h */
-union sockaddr_any {
-    struct sockaddr sa;
-    struct sockaddr_storage ss;
-    struct sockaddr_in sin;
-    struct sockaddr_in6 sin6;
-#ifndef JANET_WINDOWS
-    struct sockaddr_un sun;
-#endif
-};
-
-/* Definition from:
- *   https://github.com/wahern/cqueues/blog/master/src/lib/socket.h */
-static inline union sockaddr_arg sockaddr_ref(void *arg) {
-    return (union sockaddr_arg) {
-        arg
-    };
-}
-
-/* Definition from:
- *   https://github.com/wahern/cqueues/blog/master/src/lib/socket.h */
-static inline sa_family_t *(sa_family)(void *arg) {
-    return &sockaddr_ref(arg).sa->sa_family;
-}
-
-/* Definition from:
- *   https://github.com/wahern/cqueues/blog/master/src/lib/socket.h */
-static inline in_port_t *(sa_port)(void *arg, const in_port_t *def, int *error) {
-    switch (*sa_family(arg)) {
-        case AF_INET:
-            return &sockaddr_ref(arg).sin->sin_port;
-        case AF_INET6:
-            return &sockaddr_ref(arg).sin6->sin6_port;
+/* Turn a socket address into a host, port pair (port is optional).
+ * For unix domain sockets, returned tuple will have only a single element, the path string. */
+static Janet janet_so_getname(const void *sa_any) {
+    const struct sockaddr *sa = sa_any;
+    char buffer[SA_ADDRSTRLEN];
+    switch (sa->sa_family) {
         default:
-            if (error)
-                *error = EAFNOSUPPORT;
-
-            return (in_port_t *)def;
-    }
-}
-
-/* Definition from:
- *   https://github.com/wahern/cqueues/blog/master/src/lib/socket.c
- * Original was dns_strlcpy */
-size_t janet_socket_strlcpy(char *dst, const char *src, size_t lim) {
-    char *d     = dst;
-    char *e     = &dst[lim];
-    const char *s   = src;
-
-    if (d < e) {
-        do {
-            if ('\0' == (*d++ = *s++))
-                return s - src - 1;
-        } while (d < e);
-
-        d[-1]   = '\0';
-    }
-
-    while (*s++ != '\0')
-        ;;
-
-    return s - src - 1;
-}
-
-/* Definition from:
- *   https://github.com/wahern/cqueues/blog/master/src/lib/socket.c */
-char *sa_ntop(char *dst, size_t lim, const void *src, const char *def, int *_error) {
-    union sockaddr_any *any = (void *)src;
-    const char *unspec = "0.0.0.0";
-    char text[SA_ADDRSTRLEN];
-    int error;
-
-    switch (*sa_family(&any->sa)) {
-        case AF_INET:
-            unspec = "0.0.0.0";
-
-            if (!inet_ntop(AF_INET, &any->sin.sin_addr, text, sizeof text))
-                goto syerr;
-
-            break;
-        case AF_INET6:
-            unspec = "::";
-
-            if (!inet_ntop(AF_INET6, &any->sin6.sin6_addr, text, sizeof text))
-                goto syerr;
-
-            break;
-#ifndef JANET_WINDOWS
-        case AF_UNIX:
-            unspec = "/nonexistent";
-
-            memset(text, 0, sizeof text);
-            memcpy(text, any->sun.sun_path, SO_MIN(sizeof text - 1, sizeof any->sun.sun_path));
-
-            break;
-#endif
-        default:
-            error = EAFNOSUPPORT;
-
-            goto error;
-    }
-
-    if (janet_socket_strlcpy(dst, text, lim) >= lim) {
-        error = ENOSPC;
-
-        goto error;
-    }
-
-    return dst;
-syerr:
-    error = errno;
-error:
-    if (_error)
-        *_error = error;
-
-    /*
-     * NOTE: Always write something in case caller ignores errors, such
-     * as when caller is using the sa_ntoa() macro.
-     */
-    safe_memcpy(dst, (def) ? def : unspec, lim);
-
-    return (char *)def;
-}
-
-/* Definition from:
- *   https://github.com/wahern/cqueues/blog/master/src/lib/socket.h */
-static inline char *sa_ntoa_(char *dst, size_t lim, const void *src) {
-    return sa_ntop(dst, lim, src, NULL, &(int) {
-        0
-    }), dst;
-}
-
-/* Definition from:
- *   https://github.com/wahern/cqueues/blog/master/src/lib/socket.c
- * Originaly was lso_pushname */
-static Janet janet_so_getname(const struct sockaddr_storage *ss, socklen_t slen) {
-    uint8_t *hn = NULL;
-    uint16_t hp = 0;
-    size_t plen = SA_ADDRSTRLEN;
-
-    switch (ss->ss_family) {
-        case AF_INET:
-        /* fall through */
-        case AF_INET6:
-            /* hn = hostname, hp = hostport */
-            hn = (uint8_t *)sa_ntoa(ss);
-            hp = ntohs(*sa_port((void *)ss, SA_PORT_NONE, NULL));
-            break;
-#ifndef JANET_WINDOWS
-        case AF_UNIX:
-            /* support nameless sockets, linux-ism */
-            if (slen > offsetof(struct sockaddr_un, sun_path)) {
-                struct sockaddr_un *sun = (struct sockaddr_un *)ss;
-                char *pe = (char *)sun + SO_MIN(sizeof * sun, slen);
-
-                while (pe > sun->sun_path && pe[-1] == '\0')
-                    --pe;
-
-                if ((plen = pe - sun->sun_path) > 0) {
-                    hn = (uint8_t *)sun->sun_path;
-                } else {
-                    hn = (uint8_t *)"@";
-                    plen = 1;
-                }
-            } else {
-                hn = (uint8_t *)"@";
-                plen = 1;
+            janet_panic("unknown address family");
+        case AF_INET: {
+            const struct sockaddr_in *sai = sa_any;
+            if (!inet_ntop(AF_INET, &(sai->sin_addr), buffer, sizeof(buffer))) {
+                janet_panic("unable to decode ipv4 host address");
             }
-            break;
+            Janet pair[2] = {janet_cstringv(buffer), janet_wrap_integer(ntohs(sai->sin_port))};
+            return janet_wrap_tuple(janet_tuple_n(pair, sai->sin_port ? 2 : 1));
+        }
+        case AF_INET6: {
+            const struct sockaddr_in6 *sai6 = sa_any;
+            if (!inet_ntop(AF_INET6, &(sai6->sin6_addr), buffer, sizeof(buffer))) {
+                janet_panic("unable to decode ipv4 host address");
+            }
+            Janet pair[2] = {janet_cstringv(buffer), janet_wrap_integer(ntohs(sai6->sin6_port))};
+            return janet_wrap_tuple(janet_tuple_n(pair, sai6->sin6_port ? 2 : 1));
+        }
+#ifndef JANET_WINDOWS
+        case AF_UNIX: {
+            const struct sockaddr_un *sun = sa_any;
+            Janet pathname;
+            if (sun->sun_path[0] == '\0') {
+                memcpy(buffer, sun->sun_path, sizeof(sun->sun_path));
+                buffer[0] = '@';
+                pathname = janet_cstringv(buffer);
+            } else {
+                pathname = janet_cstringv(sun->sun_path);
+            }
+            return janet_wrap_tuple(janet_tuple_n(&pathname, 1));
+        }
 #endif
-        default:
-            hn = (uint8_t *)"";
-            plen = 0;
-            break;
     }
-
-    Janet name[2];
-    int32_t len = 1;
-    name[0] = janet_wrap_string(janet_cstring((const char *)hn));
-    if (hp > 0) {
-        len++;
-        name[1] = janet_wrap_integer(hp);
-    }
-
-    return janet_wrap_tuple(janet_tuple_n(name, len));
 }
 
 JANET_CORE_FN(cfun_net_getsockname,
               "(net/localname stream)",
               "Gets the local address and port in a tuple in that order.") {
-    janet_arity(argc, 1, 1);
+    janet_fixarity(argc, 1);
     JanetStream *js = janet_getabstract(argv, 0, &janet_stream_type);
     struct sockaddr_storage ss;
-    socklen_t slen = sizeof ss;
+    socklen_t slen = sizeof(ss);
     memset(&ss, 0, slen);
-
-    int error;
-    if (0 != (error = getsockname((JSock)js->handle, (struct sockaddr *) &ss, &slen)))
-        janet_panicf("Failed to get peername on fd %d, error: %s", js->handle, janet_ev_lasterr());
-
-    return janet_so_getname(&ss, slen);
+    if (getsockname((JSock)js->handle, (struct sockaddr *) &ss, &slen)) {
+        janet_panicf("Failed to get localname on %v: %V", argv[0], janet_ev_lasterr());
+    }
+    janet_assert(slen <= sizeof(ss), "socket address truncated");
+    return janet_so_getname(&ss);
 }
 
-
 JANET_CORE_FN(cfun_net_getpeername,
               "(net/peername stream)",
               "Gets the remote peer's address and port in a tuple in that order.") {
-    janet_arity(argc, 1, 1);
+    janet_fixarity(argc, 1);
     JanetStream *js = janet_getabstract(argv, 0, &janet_stream_type);
     struct sockaddr_storage ss;
-    socklen_t slen = sizeof ss;
+    socklen_t slen = sizeof(ss);
     memset(&ss, 0, slen);
-
-    int error;
-    if (0 != (error = getpeername((JSock)js->handle, (struct sockaddr *)&ss, &slen))) {
-        janet_panicf("Failed to get peername on fd %d, error: %s", js->handle, janet_ev_lasterr());
+    if (getpeername((JSock)js->handle, (struct sockaddr *)&ss, &slen)) {
+        janet_panicf("Failed to get peername on %v: %V", argv[0], janet_ev_lasterr());
     }
+    janet_assert(slen <= sizeof(ss), "socket address truncated");
+    return janet_so_getname(&ss);
+}
 
-    return janet_so_getname(&ss, slen);
+JANET_CORE_FN(cfun_net_address_unpack,
+              "(net/address-unpack address)",
+              "Given an address returned by net/adress, return a host, port pair. Unix domain sockets "
+              "will have only the path in the returned tuple.") {
+    janet_fixarity(argc, 1);
+    struct sockaddr *sa = janet_getabstract(argv, 0, &janet_address_type);
+    return janet_so_getname(sa);
 }
 
 JANET_CORE_FN(cfun_stream_accept_loop,
@@ -1066,6 +900,7 @@ void janet_lib_net(JanetTable *env) {
         JANET_CORE_REG("net/shutdown", cfun_net_shutdown),
         JANET_CORE_REG("net/peername", cfun_net_getpeername),
         JANET_CORE_REG("net/localname", cfun_net_getsockname),
+        JANET_CORE_REG("net/address-unpack", cfun_net_address_unpack),
         JANET_REG_END
     };
     janet_core_cfuns_ext(env, NULL, net_cfuns);

src/core/peg.c

@@ -387,6 +387,25 @@ tail:
             return result;
         }
 
+        case RULE_CAPTURE_NUM: {
+            down1(s);
+            const uint8_t *result = peg_rule(s, s->bytecode + rule[1], text);
+            up1(s);
+            if (!result) return NULL;
+            /* check number parsing */
+            double x = 0.0;
+            int32_t base = (int32_t) rule[2];
+            if (janet_scan_number_base(text, (int32_t)(result - text), base, &x)) return NULL;
+            /* Specialized pushcap - avoid intermediate string creation */
+            if (!s->has_backref && s->mode == PEG_MODE_ACCUMULATE) {
+                janet_buffer_push_bytes(s->scratch, text, (int32_t)(result - text));
+            } else {
+                uint32_t tag = rule[3];
+                pushcap(s, janet_wrap_number(x), tag);
+            }
+            return result;
+        }
+
         case RULE_ACCUMULATE: {
             uint32_t tag = rule[2];
             int oldmode = s->mode;
@@ -975,6 +994,25 @@ static void spec_unref(Builder *b, int32_t argc, const Janet *argv) {
     spec_cap1(b, argc, argv, RULE_UNREF);
 }
 
+static void spec_capture_number(Builder *b, int32_t argc, const Janet *argv) {
+    peg_arity(b, argc, 1, 3);
+    Reserve r = reserve(b, 4);
+    uint32_t base = 0;
+    if (argc >= 2) {
+        if (!janet_checktype(argv[1], JANET_NIL)) {
+            if (!janet_checkint(argv[1])) goto error;
+            base = (uint32_t) janet_unwrap_integer(argv[1]);
+            if (base < 2 || base > 36) goto error;
+        }
+    }
+    uint32_t tag = (argc == 3) ? emit_tag(b, argv[2]) : 0;
+    uint32_t rule = peg_compile1(b, argv[0]);
+    emit_3(r, RULE_CAPTURE_NUM, rule, base, tag);
+    return;
+error:
+    peg_panicf(b, "expected integer between 2 and 36, got %v", argv[2]);
+}
+
 static void spec_reference(Builder *b, int32_t argc, const Janet *argv) {
     peg_arity(b, argc, 1, 2);
     Reserve r = reserve(b, 3);
@@ -1118,6 +1156,7 @@ static const SpecialPair peg_specials[] = {
     {"line", spec_line},
     {"look", spec_look},
     {"not", spec_not},
+    {"number", spec_capture_number},
     {"opt", spec_opt},
     {"position", spec_position},
     {"quote", spec_capture},
@@ -1419,6 +1458,12 @@ static void *peg_unmarshal(JanetMarshalContext *ctx) {
                 if (rule[1] >= clen) goto bad;
                 i += 3;
                 break;
+            case RULE_CAPTURE_NUM:
+                /* [rule, base, tag] */
+                if (rule[1] >= blen) goto bad;
+                op_flags[rule[1]] |= 0x01;
+                i += 4;
+                break;
             case RULE_ACCUMULATE:
             case RULE_GROUP:
             case RULE_CAPTURE:

src/core/run.c

@@ -79,7 +79,9 @@ int janet_dobytes(JanetTable *env, const uint8_t *bytes, int32_t len, const char
                 const char *e = janet_parser_error(&parser);
                 errflags |= 0x04;
                 ret = janet_cstringv(e);
-                janet_eprintf("parse error in %s: %s\n", sourcePath, e);
+                int32_t line = parser.line;
+                int32_t col = parser.column;
+                janet_eprintf("%s:%d:%d: parse error: %s\n", sourcePath, line, col, e);
                 done = 1;
                 break;
             }

src/core/state.h

@@ -54,14 +54,6 @@ typedef struct {
     int is_error;
 } JanetTimeout;
 
-#ifdef JANET_THREADS
-typedef struct {
-    JanetMailbox *original;
-    JanetMailbox *newbox;
-    uint64_t flags;
-} JanetMailboxPair;
-#endif
-
 /* Registry table for C functions - containts metadata that can
  * be looked up by cfunction pointer. All strings here are pointing to
  * static memory not managed by Janet. */
@@ -145,13 +137,6 @@ struct JanetVM {
     JanetTraversalNode *traversal_top;
     JanetTraversalNode *traversal_base;
 
-    /* Threading */
-#ifdef JANET_THREADS
-    JanetMailbox *mailbox;
-    JanetThread *thread_current;
-    JanetTable *thread_decode;
-#endif
-
     /* Event loop and scheduler globals */
 #ifdef JANET_EV
     size_t tq_count;
@@ -186,12 +171,6 @@ struct JanetVM {
 
 extern JANET_THREAD_LOCAL JanetVM janet_vm;
 
-/* Setup / teardown */
-#ifdef JANET_THREADS
-void janet_threads_init(void);
-void janet_threads_deinit(void);
-#endif
-
 #ifdef JANET_NET
 void janet_net_init(void);
 void janet_net_deinit(void);

src/core/string.c

@@ -218,7 +218,7 @@ JANET_CORE_FN(cfun_string_repeat,
 
 JANET_CORE_FN(cfun_string_bytes,
               "(string/bytes str)",
-              "Returns an array of integers that are the byte values of the string.") {
+              "Returns a tuple of integers that are the byte values of the string.") {
     janet_fixarity(argc, 1);
     JanetByteView view = janet_getbytes(argv, 0);
     Janet *tup = janet_tuple_begin(view.len);

src/core/strtod.c

@@ -246,15 +246,15 @@ static double convert(
 }
 
 /* Scan a real (double) from a string. If the string cannot be converted into
- * and integer, set *err to 1 and return 0. */
-int janet_scan_number(
+ * and integer, return 0. */
+int janet_scan_number_base(
     const uint8_t *str,
     int32_t len,
+    int32_t base,
     double *out) {
     const uint8_t *end = str + len;
     int seenadigit = 0;
     int ex = 0;
-    int base = 10;
     int seenpoint = 0;
     int foundexp = 0;
     int neg = 0;
@@ -278,21 +278,28 @@ int janet_scan_number(
     }
 
     /* Check for leading 0x or digit digit r */
-    if (str + 1 < end && str[0] == '0' && str[1] == 'x') {
-        base = 16;
-        str += 2;
-    } else if (str + 1 < end  &&
-               str[0] >= '0' && str[0] <= '9' &&
-               str[1] == 'r') {
-        base = str[0] - '0';
-        str += 2;
-    } else if (str + 2 < end  &&
-               str[0] >= '0' && str[0] <= '9' &&
-               str[1] >= '0' && str[1] <= '9' &&
-               str[2] == 'r') {
-        base = 10 * (str[0] - '0') + (str[1] - '0');
-        if (base < 2 || base > 36) goto error;
-        str += 3;
+    if (base == 0) {
+        if (str + 1 < end && str[0] == '0' && str[1] == 'x') {
+            base = 16;
+            str += 2;
+        } else if (str + 1 < end  &&
+                   str[0] >= '0' && str[0] <= '9' &&
+                   str[1] == 'r') {
+            base = str[0] - '0';
+            str += 2;
+        } else if (str + 2 < end  &&
+                   str[0] >= '0' && str[0] <= '9' &&
+                   str[1] >= '0' && str[1] <= '9' &&
+                   str[2] == 'r') {
+            base = 10 * (str[0] - '0') + (str[1] - '0');
+            if (base < 2 || base > 36) goto error;
+            str += 3;
+        }
+    }
+
+    /* If still base is 0, set to default (10) */
+    if (base == 0) {
+        base = 10;
     }
 
     /* Skip leading zeros */
@@ -376,6 +383,13 @@ error:
     return 1;
 }
 
+int janet_scan_number(
+    const uint8_t *str,
+    int32_t len,
+    double *out) {
+    return janet_scan_number_base(str, len, 0, out);
+}
+
 #ifdef JANET_INT_TYPES
 
 static int scan_uint64(

src/core/thread.c

@@ -1,739 +0,0 @@
-/*
-* Copyright (c) 2021 Calvin Rose
-*
-* Permission is hereby granted, free of charge, to any person obtaining a copy
-* of this software and associated documentation files (the "Software"), to
-* deal in the Software without restriction, including without limitation the
-* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
-* sell copies of the Software, and to permit persons to whom the Software is
-* furnished to do so, subject to the following conditions:
-*
-* The above copyright notice and this permission notice shall be included in
-* all copies or substantial portions of the Software.
-*
-* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
-* IN THE SOFTWARE.
-*/
-
-#ifndef JANET_AMALG
-#include "features.h"
-#include <janet.h>
-#include "gc.h"
-#include "util.h"
-#include "state.h"
-#endif
-
-#ifdef JANET_THREADS
-
-#include <math.h>
-#ifdef JANET_WINDOWS
-#include <windows.h>
-#else
-#include <setjmp.h>
-#include <time.h>
-#include <pthread.h>
-#endif
-
-/* typedefed in janet.h */
-struct JanetMailbox {
-
-    /* Synchronization */
-#ifdef JANET_WINDOWS
-    CRITICAL_SECTION lock;
-    CONDITION_VARIABLE cond;
-#else
-    pthread_mutex_t lock;
-    pthread_cond_t cond;
-#endif
-
-    /* Memory management - reference counting */
-    int refCount;
-    int closed;
-
-    /* Store messages */
-    uint16_t messageCapacity;
-    uint16_t messageCount;
-    uint16_t messageFirst;
-    uint16_t messageNext;
-
-    /* Buffers to store messages. These buffers are manually allocated, so
-     * are not owned by any thread's GC. */
-    JanetBuffer messages[];
-};
-
-#define JANET_THREAD_HEAVYWEIGHT 0x1
-#define JANET_THREAD_ABSTRACTS 0x2
-#define JANET_THREAD_CFUNCTIONS 0x4
-static const char janet_thread_flags[] = "hac";
-
-static JanetTable *janet_thread_get_decode(void) {
-    if (janet_vm.thread_decode == NULL) {
-        janet_vm.thread_decode = janet_get_core_table("load-image-dict");
-        if (NULL == janet_vm.thread_decode) {
-            janet_vm.thread_decode = janet_table(0);
-        }
-        janet_gcroot(janet_wrap_table(janet_vm.thread_decode));
-    }
-    return janet_vm.thread_decode;
-}
-
-static JanetMailbox *janet_mailbox_create(int refCount, uint16_t capacity) {
-    JanetMailbox *mailbox = janet_malloc(sizeof(JanetMailbox) + sizeof(JanetBuffer) * (size_t) capacity);
-    if (NULL == mailbox) {
-        JANET_OUT_OF_MEMORY;
-    }
-#ifdef JANET_WINDOWS
-    InitializeCriticalSection(&mailbox->lock);
-    InitializeConditionVariable(&mailbox->cond);
-#else
-    pthread_mutex_init(&mailbox->lock, NULL);
-    pthread_cond_init(&mailbox->cond, NULL);
-#endif
-    mailbox->refCount = refCount;
-    mailbox->closed = 0;
-    mailbox->messageCount = 0;
-    mailbox->messageCapacity = capacity;
-    mailbox->messageFirst = 0;
-    mailbox->messageNext = 0;
-    for (uint16_t i = 0; i < capacity; i++) {
-        janet_buffer_init(mailbox->messages + i, 0);
-    }
-    return mailbox;
-}
-
-static void janet_mailbox_destroy(JanetMailbox *mailbox) {
-#ifdef JANET_WINDOWS
-    DeleteCriticalSection(&mailbox->lock);
-#else
-    pthread_mutex_destroy(&mailbox->lock);
-    pthread_cond_destroy(&mailbox->cond);
-#endif
-    for (uint16_t i = 0; i < mailbox->messageCapacity; i++) {
-        janet_buffer_deinit(mailbox->messages + i);
-    }
-    janet_free(mailbox);
-}
-
-static void janet_mailbox_lock(JanetMailbox *mailbox) {
-#ifdef JANET_WINDOWS
-    EnterCriticalSection(&mailbox->lock);
-#else
-    pthread_mutex_lock(&mailbox->lock);
-#endif
-}
-
-static void janet_mailbox_unlock(JanetMailbox *mailbox) {
-#ifdef JANET_WINDOWS
-    LeaveCriticalSection(&mailbox->lock);
-#else
-    pthread_mutex_unlock(&mailbox->lock);
-#endif
-}
-
-/* Assumes you have the mailbox lock already */
-static void janet_mailbox_ref_with_lock(JanetMailbox *mailbox, int delta) {
-    mailbox->refCount += delta;
-    if (mailbox->refCount <= 0) {
-        janet_mailbox_unlock(mailbox);
-        janet_mailbox_destroy(mailbox);
-    } else {
-        janet_mailbox_unlock(mailbox);
-    }
-}
-
-static void janet_mailbox_ref(JanetMailbox *mailbox, int delta) {
-    janet_mailbox_lock(mailbox);
-    janet_mailbox_ref_with_lock(mailbox, delta);
-}
-
-static void janet_close_thread(JanetThread *thread) {
-    if (thread->mailbox) {
-        janet_mailbox_ref(thread->mailbox, -1);
-        thread->mailbox = NULL;
-    }
-}
-
-static int thread_gc(void *p, size_t size) {
-    (void) size;
-    JanetThread *thread = (JanetThread *)p;
-    janet_close_thread(thread);
-    return 0;
-}
-
-static int thread_mark(void *p, size_t size) {
-    (void) size;
-    JanetThread *thread = (JanetThread *)p;
-    if (thread->encode) {
-        janet_mark(janet_wrap_table(thread->encode));
-    }
-    return 0;
-}
-
-static JanetMailboxPair *make_mailbox_pair(JanetMailbox *original, uint64_t flags) {
-    JanetMailboxPair *pair = janet_malloc(sizeof(JanetMailboxPair));
-    if (NULL == pair) {
-        JANET_OUT_OF_MEMORY;
-    }
-    pair->original = original;
-    janet_mailbox_ref(original, 1);
-    pair->newbox = janet_mailbox_create(1, 16);
-    pair->flags = flags;
-    return pair;
-}
-
-static void destroy_mailbox_pair(JanetMailboxPair *pair) {
-    janet_mailbox_ref(pair->original, -1);
-    janet_mailbox_ref(pair->newbox, -1);
-    janet_free(pair);
-}
-
-/* Abstract waiting for timeout across windows/posix */
-typedef struct {
-    int timedwait;
-    int nowait;
-#ifdef JANET_WINDOWS
-    DWORD interval;
-    DWORD ticksLeft;
-#else
-    struct timespec ts;
-#endif
-} JanetWaiter;
-
-static void janet_waiter_init(JanetWaiter *waiter, double sec) {
-    waiter->timedwait = 0;
-    waiter->nowait = 0;
-
-    if (sec <= 0.0 || isnan(sec)) {
-        waiter->nowait = 1;
-        return;
-    }
-    waiter->timedwait = sec > 0.0 && !isinf(sec);
-
-    /* Set maximum wait time to 30 days */
-    if (sec > (60.0 * 60.0 * 24.0 * 30.0)) {
-        sec = 60.0 * 60.0 * 24.0 * 30.0;
-    }
-
-#ifdef JANET_WINDOWS
-    if (waiter->timedwait) {
-        waiter->ticksLeft = waiter->interval = (DWORD) floor(1000.0 * sec);
-    }
-#else
-    if (waiter->timedwait) {
-        /* N seconds -> timespec of (now + sec) */
-        struct timespec now;
-        janet_gettime(&now);
-        time_t tvsec = (time_t) floor(sec);
-        long tvnsec = (long) floor(1000000000.0 * (sec - ((double) tvsec)));
-        tvsec += now.tv_sec;
-        tvnsec += now.tv_nsec;
-        if (tvnsec >= 1000000000L) {
-            tvnsec -= 1000000000L;
-            tvsec += 1;
-        }
-        waiter->ts.tv_sec = tvsec;
-        waiter->ts.tv_nsec = tvnsec;
-    }
-#endif
-}
-
-static int janet_waiter_wait(JanetWaiter *wait, JanetMailbox *mailbox) {
-    if (wait->nowait) return 1;
-#ifdef JANET_WINDOWS
-    if (wait->timedwait) {
-        if (wait->ticksLeft == 0) return 1;
-        DWORD startTime = GetTickCount();
-        int status = !SleepConditionVariableCS(&mailbox->cond, &mailbox->lock, wait->ticksLeft);
-        DWORD dTick = GetTickCount() - startTime;
-        /* Be careful about underflow */
-        wait->ticksLeft = dTick > wait->ticksLeft ? 0 : dTick;
-        return status;
-    } else {
-        SleepConditionVariableCS(&mailbox->cond, &mailbox->lock, INFINITE);
-        return 0;
-    }
-#else
-    if (wait->timedwait) {
-        return pthread_cond_timedwait(&mailbox->cond, &mailbox->lock, &wait->ts);
-    } else {
-        pthread_cond_wait(&mailbox->cond, &mailbox->lock);
-        return 0;
-    }
-#endif
-}
-
-static void janet_mailbox_wakeup(JanetMailbox *mailbox) {
-#ifdef JANET_WINDOWS
-    WakeConditionVariable(&mailbox->cond);
-#else
-    pthread_cond_signal(&mailbox->cond);
-#endif
-}
-
-static int mailbox_at_capacity(JanetMailbox *mailbox) {
-    return mailbox->messageCount >= mailbox->messageCapacity;
-}
-
-/* Returns 1 if could not send (encode error or timeout), 2 for mailbox closed, and
- * 0 otherwise. Will not panic.  */
-int janet_thread_send(JanetThread *thread, Janet msg, double timeout) {
-
-    /* Ensure mailbox is not closed. */
-    JanetMailbox *mailbox = thread->mailbox;
-    if (NULL == mailbox) return 2;
-    janet_mailbox_lock(mailbox);
-    if (mailbox->closed) {
-        janet_mailbox_ref_with_lock(mailbox, -1);
-        thread->mailbox = NULL;
-        return 2;
-    }
-
-    /* Back pressure */
-    if (mailbox_at_capacity(mailbox)) {
-        JanetWaiter wait;
-        janet_waiter_init(&wait, timeout);
-
-        if (wait.nowait) {
-            janet_mailbox_unlock(mailbox);
-            return 1;
-        }
-
-        /* Retry loop, as there can be multiple writers */
-        while (mailbox_at_capacity(mailbox)) {
-            if (janet_waiter_wait(&wait, mailbox)) {
-                janet_mailbox_unlock(mailbox);
-                janet_mailbox_wakeup(mailbox);
-                return 1;
-            }
-        }
-    }
-
-    /* Hack to capture all panics from marshalling. This works because
-     * we know janet_marshal won't mess with other essential global state. */
-    jmp_buf buf;
-    jmp_buf *old_buf = janet_vm.signal_buf;
-    janet_vm.signal_buf = &buf;
-    int32_t oldmcount = mailbox->messageCount;
-
-    int ret = 0;
-    if (setjmp(buf)) {
-        ret = 1;
-        mailbox->messageCount = oldmcount;
-    } else {
-        JanetBuffer *msgbuf = mailbox->messages + mailbox->messageNext;
-        msgbuf->count = 0;
-
-        /* Start panic zone */
-        janet_marshal(msgbuf, msg, thread->encode, JANET_MARSHAL_UNSAFE);
-        /* End panic zone */
-
-        mailbox->messageNext = (mailbox->messageNext + 1) % mailbox->messageCapacity;
-        mailbox->messageCount++;
-    }
-
-    /* Cleanup */
-    janet_vm.signal_buf = old_buf;
-    janet_mailbox_unlock(mailbox);
-
-    /* Potentially wake up a blocked thread */
-    janet_mailbox_wakeup(mailbox);
-
-    return ret;
-}
-
-/* Returns 0 on successful message. Returns 1 if timedout */
-int janet_thread_receive(Janet *msg_out, double timeout) {
-    JanetMailbox *mailbox = janet_vm.mailbox;
-    janet_mailbox_lock(mailbox);
-
-    /* For timeouts */
-    JanetWaiter wait;
-    janet_waiter_init(&wait, timeout);
-
-    for (;;) {
-
-        /* Check for messages waiting for us */
-        if (mailbox->messageCount > 0) {
-
-            /* Hack to capture all panics from marshalling. This works because
-             * we know janet_marshal won't mess with other essential global state. */
-            jmp_buf buf;
-            jmp_buf *old_buf = janet_vm.signal_buf;
-            janet_vm.signal_buf = &buf;
-
-            /* Handle errors */
-            if (setjmp(buf)) {
-                /* Cleanup jmp_buf, return error.
-                 * Do not ignore bad messages as before. */
-                janet_vm.signal_buf = old_buf;
-                *msg_out = *janet_vm.return_reg;
-                janet_mailbox_unlock(mailbox);
-                return 2;
-            } else {
-                JanetBuffer *msgbuf = mailbox->messages + mailbox->messageFirst;
-                mailbox->messageCount--;
-                mailbox->messageFirst = (mailbox->messageFirst + 1) % mailbox->messageCapacity;
-
-                /* Read from beginning of channel */
-                const uint8_t *nextItem = NULL;
-                Janet item = janet_unmarshal(
-                                 msgbuf->data, msgbuf->count,
-                                 JANET_MARSHAL_UNSAFE, janet_thread_get_decode(), &nextItem);
-                *msg_out = item;
-
-                /* Cleanup */
-                janet_vm.signal_buf = old_buf;
-                janet_mailbox_unlock(mailbox);
-
-                /* Potentially wake up pending threads */
-                janet_mailbox_wakeup(mailbox);
-
-                return 0;
-            }
-        }
-
-        if (wait.nowait) {
-            janet_mailbox_unlock(mailbox);
-            return 1;
-        }
-
-        /* Wait for next message */
-        if (janet_waiter_wait(&wait, mailbox)) {
-            janet_mailbox_unlock(mailbox);
-            return 1;
-        }
-    }
-}
-
-static int janet_thread_getter(void *p, Janet key, Janet *out);
-static Janet janet_thread_next(void *p, Janet key);
-
-const JanetAbstractType janet_thread_type = {
-    "core/thread",
-    thread_gc,
-    thread_mark,
-    janet_thread_getter,
-    NULL, /* put */
-    NULL, /* marshal */
-    NULL, /* unmarshal */
-    NULL, /* tostring */
-    NULL, /* compare */
-    NULL, /* hash */
-    janet_thread_next,
-    JANET_ATEND_NEXT
-};
-
-static JanetThread *janet_make_thread(JanetMailbox *mailbox, JanetTable *encode) {
-    JanetThread *thread = janet_abstract(&janet_thread_type, sizeof(JanetThread));
-    janet_mailbox_ref(mailbox, 1);
-    thread->mailbox = mailbox;
-    thread->encode = encode;
-    return thread;
-}
-
-JanetThread *janet_getthread(const Janet *argv, int32_t n) {
-    return (JanetThread *) janet_getabstract(argv, n, &janet_thread_type);
-}
-
-/* Runs in new thread */
-static int thread_worker(JanetMailboxPair *pair) {
-    JanetFiber *fiber = NULL;
-    Janet out;
-
-    /* Init VM */
-    janet_init();
-
-    /* Use the mailbox we were given */
-    janet_vm.mailbox = pair->newbox;
-    janet_mailbox_ref(pair->newbox, 1);
-
-    /* Get dictionaries for default encode/decode */
-    JanetTable *encode;
-    if (pair->flags & JANET_THREAD_HEAVYWEIGHT) {
-        encode = janet_get_core_table("make-image-dict");
-    } else {
-        encode = NULL;
-        janet_vm.thread_decode = janet_table(0);
-        janet_gcroot(janet_wrap_table(janet_vm.thread_decode));
-    }
-
-    /* Create parent thread */
-    JanetThread *parent = janet_make_thread(pair->original, encode);
-    Janet parentv = janet_wrap_abstract(parent);
-
-    /* Unmarshal the abstract registry */
-    if (pair->flags & JANET_THREAD_ABSTRACTS) {
-        Janet reg;
-        int status = janet_thread_receive(&reg, INFINITY);
-        if (status) goto error;
-        if (!janet_checktype(reg, JANET_TABLE)) goto error;
-        janet_gcunroot(janet_wrap_table(janet_vm.abstract_registry));
-        janet_vm.abstract_registry = janet_unwrap_table(reg);
-        janet_gcroot(janet_wrap_table(janet_vm.abstract_registry));
-    }
-
-    /* Unmarshal the function */
-    Janet funcv;
-    int status = janet_thread_receive(&funcv, INFINITY);
-    if (status) goto error;
-    if (!janet_checktype(funcv, JANET_FUNCTION)) goto error;
-    JanetFunction *func = janet_unwrap_function(funcv);
-
-    /* Arity check */
-    if (func->def->min_arity > 1 || func->def->max_arity < 1) {
-        goto error;
-    }
-
-    /* Call function */
-    Janet argv[1] = { parentv };
-    fiber = janet_fiber(func, 64, 1, argv);
-    if (pair->flags & JANET_THREAD_HEAVYWEIGHT) {
-        fiber->env = janet_table(0);
-        fiber->env->proto = janet_core_env(NULL);
-    }
-    JanetSignal sig = janet_continue(fiber, janet_wrap_nil(), &out);
-    if (sig != JANET_SIGNAL_OK && sig < JANET_SIGNAL_USER0) {
-        janet_eprintf("in thread %v: ", janet_wrap_abstract(janet_make_thread(pair->newbox, encode)));
-        janet_stacktrace(fiber, out);
-    }
-
-#ifdef JANET_EV
-    janet_loop();
-#endif
-
-    /* Normal exit */
-    destroy_mailbox_pair(pair);
-    janet_deinit();
-    return 0;
-
-    /* Fail to set something up */
-error:
-    destroy_mailbox_pair(pair);
-    janet_eprintf("\nthread failed to start\n");
-    janet_deinit();
-    return 1;
-}
-
-#ifdef JANET_WINDOWS
-
-static DWORD WINAPI janet_create_thread_wrapper(LPVOID param) {
-    thread_worker((JanetMailboxPair *)param);
-    return 0;
-}
-
-static int janet_thread_start_child(JanetMailboxPair *pair) {
-    HANDLE handle = CreateThread(NULL, 0, janet_create_thread_wrapper, pair, 0, NULL);
-    int ret = NULL == handle;
-    /* Does not kill thread, simply detatches */
-    if (!ret) CloseHandle(handle);
-    return ret;
-}
-
-#else
-
-static void *janet_pthread_wrapper(void *param) {
-    thread_worker((JanetMailboxPair *)param);
-    return NULL;
-}
-
-static int janet_thread_start_child(JanetMailboxPair *pair) {
-    pthread_t handle;
-    int error = pthread_create(&handle, NULL, janet_pthread_wrapper, pair);
-    if (error) {
-        return 1;
-    } else {
-        pthread_detach(handle);
-        return 0;
-    }
-}
-
-#endif
-
-/*
- * Setup/Teardown
- */
-
-void janet_threads_init(void) {
-    janet_vm.mailbox = janet_mailbox_create(1, 10);
-    janet_vm.thread_decode = NULL;
-    janet_vm.thread_current = NULL;
-}
-
-void janet_threads_deinit(void) {
-    janet_mailbox_lock(janet_vm.mailbox);
-    janet_vm.mailbox->closed = 1;
-    janet_mailbox_ref_with_lock(janet_vm.mailbox, -1);
-    janet_vm.mailbox = NULL;
-    janet_vm.thread_current = NULL;
-    janet_vm.thread_decode = NULL;
-}
-
-JanetThread *janet_thread_current(void) {
-    if (NULL == janet_vm.thread_current) {
-        janet_vm.thread_current = janet_make_thread(janet_vm.mailbox, janet_get_core_table("make-image-dict"));
-        janet_gcroot(janet_wrap_abstract(janet_vm.thread_current));
-    }
-    return janet_vm.thread_current;
-}
-
-/*
- * Cfuns
- */
-
-JANET_CORE_FN(cfun_thread_current,
-              "(thread/current)",
-              "Get the current running thread.") {
-    (void) argv;
-    janet_fixarity(argc, 0);
-    return janet_wrap_abstract(janet_thread_current());
-}
-
-JANET_CORE_FN(cfun_thread_new,
-              "(thread/new func &opt capacity flags)",
-              "Start a new thread that will start immediately. "
-              "If capacity is provided, that is how many messages can be stored in the thread's mailbox before blocking senders. "
-              "The capacity must be between 1 and 65535 inclusive, and defaults to 10. "
-              "Can optionally provide flags to the new thread - supported flags are:\n\n"
-              "* `:h` - Start a heavyweight thread. This loads the core environment by default, so may use more memory initially. Messages may compress better, though.\n"
-              "* `:a` - Allow sending over registered abstract types to the new thread\n"
-              "* `:c` - Send over cfunction information to the new thread (no longer supported).\n"
-              "Returns a handle to the new thread.") {
-    janet_arity(argc, 1, 3);
-    /* Just type checking */
-    janet_getfunction(argv, 0);
-    int32_t cap = janet_optinteger(argv, argc, 1, 10);
-    if (cap < 1 || cap > UINT16_MAX) {
-        janet_panicf("bad slot #1, expected integer in range [1, 65535], got %d", cap);
-    }
-    uint64_t flags = argc >= 3 ? janet_getflags(argv, 2, janet_thread_flags) : JANET_THREAD_ABSTRACTS;
-    JanetTable *encode;
-    if (flags & JANET_THREAD_HEAVYWEIGHT) {
-        encode = janet_get_core_table("make-image-dict");
-    } else {
-        encode = NULL;
-    }
-
-    JanetMailboxPair *pair = make_mailbox_pair(janet_vm.mailbox, flags);
-    JanetThread *thread = janet_make_thread(pair->newbox, encode);
-    if (janet_thread_start_child(pair)) {
-        destroy_mailbox_pair(pair);
-        janet_panic("could not start thread");
-    }
-
-    if (flags & JANET_THREAD_ABSTRACTS) {
-        if (janet_thread_send(thread, janet_wrap_table(janet_vm.abstract_registry), INFINITY)) {
-            janet_panic("could not send abstract registry to thread");
-        }
-    }
-
-    /* If thread started, send the worker function. */
-    if (janet_thread_send(thread, argv[0], INFINITY)) {
-        janet_panicf("could not send worker function %v to thread", argv[0]);
-    }
-
-    return janet_wrap_abstract(thread);
-}
-
-JANET_CORE_FN(cfun_thread_send,
-              "(thread/send thread msgi &opt timeout)",
-              "Send a message to the thread. By default, the timeout is 1 second, but an optional timeout "
-              "in seconds can be provided. Use math/inf for no timeout. "
-              "Will throw an error if there is a problem sending the message.") {
-    janet_arity(argc, 2, 3);
-    JanetThread *thread = janet_getthread(argv, 0);
-    int status = janet_thread_send(thread, argv[1], janet_optnumber(argv, argc, 2, 1.0));
-    switch (status) {
-        default:
-            break;
-        case 1:
-            janet_panicf("failed to send message %v", argv[1]);
-        case 2:
-            janet_panic("thread mailbox is closed");
-    }
-    return argv[0];
-}
-
-JANET_CORE_FN(cfun_thread_receive,
-              "(thread/receive &opt timeout)",
-              "Get a message sent to this thread. If timeout (in seconds) is provided, an error "
-              "will be thrown after the timeout has elapsed but "
-              "no messages are received. The default timeout is 1 second, and math/inf cam be passed to "
-              "turn off the timeout.") {
-    janet_arity(argc, 0, 1);
-    double wait = janet_optnumber(argv, argc, 0, 1.0);
-    Janet out;
-    int status = janet_thread_receive(&out, wait);
-    switch (status) {
-        default:
-            break;
-        case 1:
-            janet_panicf("timeout after %f seconds", wait);
-        case 2:
-            janet_panicf("failed to receive message: %v", out);
-    }
-    return out;
-}
-
-JANET_CORE_FN(cfun_thread_close,
-              "(thread/close thread)",
-              "Close a thread, unblocking it and ending communication with it. Note that closing "
-              "a thread is idempotent and does not cancel the thread's operation. Returns nil.") {
-    janet_fixarity(argc, 1);
-    JanetThread *thread = janet_getthread(argv, 0);
-    janet_close_thread(thread);
-    return janet_wrap_nil();
-}
-
-JANET_CORE_FN(cfun_thread_exit,
-              "(thread/exit &opt code)",
-              "Exit from the current thread. If no more threads are running, ends the process, but otherwise does "
-              "not end the current process.") {
-    (void) argv;
-    janet_arity(argc, 0, 1);
-#if defined(JANET_WINDOWS)
-    int32_t flag = janet_optinteger(argv, argc, 0, 0);
-    ExitThread(flag);
-#else
-    pthread_exit(NULL);
-#endif
-    return janet_wrap_nil();
-}
-
-static const JanetMethod janet_thread_methods[] = {
-    {"send", cfun_thread_send},
-    {"close", cfun_thread_close},
-    {NULL, NULL}
-};
-
-static int janet_thread_getter(void *p, Janet key, Janet *out) {
-    (void) p;
-    if (!janet_checktype(key, JANET_KEYWORD)) return 0;
-    return janet_getmethod(janet_unwrap_keyword(key), janet_thread_methods, out);
-}
-
-static Janet janet_thread_next(void *p, Janet key) {
-    (void) p;
-    return janet_nextmethod(janet_thread_methods, key);
-}
-
-/* Module entry point */
-void janet_lib_thread(JanetTable *env) {
-    JanetRegExt threadlib_cfuns[] = {
-        JANET_CORE_REG("thread/current", cfun_thread_current),
-        JANET_CORE_REG("thread/new", cfun_thread_new),
-        JANET_CORE_REG("thread/send", cfun_thread_send),
-        JANET_CORE_REG("thread/receive", cfun_thread_receive),
-        JANET_CORE_REG("thread/close", cfun_thread_close),
-        JANET_CORE_REG("thread/exit", cfun_thread_exit),
-        JANET_REG_END
-    };
-    janet_core_cfuns_ext(env, NULL, threadlib_cfuns);
-    janet_register_abstract_type(&janet_thread_type);
-}
-
-#endif

src/core/util.h

@@ -145,9 +145,6 @@ void janet_lib_typed_array(JanetTable *env);
 #ifdef JANET_INT_TYPES
 void janet_lib_inttypes(JanetTable *env);
 #endif
-#ifdef JANET_THREADS
-void janet_lib_thread(JanetTable *env);
-#endif
 #ifdef JANET_NET
 void janet_lib_net(JanetTable *env);
 extern const JanetAbstractType janet_address_type;

src/core/vm.c

@@ -1493,7 +1493,9 @@ JanetSignal janet_pcall(
 
 Janet janet_mcall(const char *name, int32_t argc, Janet *argv) {
     /* At least 1 argument */
-    if (argc < 1) janet_panicf("method :%s expected at least 1 argument");
+    if (argc < 1) {
+        janet_panicf("method :%s expected at least 1 argument", name);
+    }
     /* Find method */
     Janet method = janet_method_lookup(argv[0], name);
     if (janet_checktype(method, JANET_NIL)) {
@@ -1557,9 +1559,6 @@ int janet_init(void) {
     janet_vm.root_fiber = NULL;
     janet_vm.stackn = 0;
 
-#ifdef JANET_THREADS
-    janet_threads_init();
-#endif
 #ifdef JANET_EV
     janet_ev_init();
 #endif
@@ -1586,9 +1585,6 @@ void janet_deinit(void) {
     janet_vm.root_fiber = NULL;
     janet_free(janet_vm.registry);
     janet_vm.registry = NULL;
-#ifdef JANET_THREADS
-    janet_threads_deinit();
-#endif
 #ifdef JANET_EV
     janet_ev_deinit();
 #endif

src/include/janet.h

@@ -144,11 +144,6 @@ extern "C" {
 #define JANET_NO_UTC_MKTIME
 #endif
 
-/* Check thread library */
-#ifndef JANET_NO_THREADS
-#define JANET_THREADS
-#endif
-
 /* Define how global janet state is declared */
 /* Also enable the thread library only if not single-threaded */
 #ifdef JANET_SINGLE_THREADED
@@ -1527,6 +1522,7 @@ JANET_API int janet_loop_fiber(JanetFiber *fiber);
 
 /* Number scanning */
 JANET_API int janet_scan_number(const uint8_t *str, int32_t len, double *out);
+JANET_API int janet_scan_number_base(const uint8_t *str, int32_t len, int32_t base, double *out);
 JANET_API int janet_scan_int64(const uint8_t *str, int32_t len, int64_t *out);
 JANET_API int janet_scan_uint64(const uint8_t *str, int32_t len, uint64_t *out);
 
@@ -1543,6 +1539,7 @@ JANET_API JanetRNG *janet_default_rng(void);
 JANET_API void janet_rng_seed(JanetRNG *rng, uint32_t seed);
 JANET_API void janet_rng_longseed(JanetRNG *rng, const uint8_t *bytes, int32_t len);
 JANET_API uint32_t janet_rng_u32(JanetRNG *rng);
+JANET_API double janet_rng_double(JanetRNG *rng);
 
 /* Array functions */
 JANET_API JanetArray *janet_array(int32_t capacity);
@@ -2040,7 +2037,8 @@ typedef enum {
     RULE_READINT,      /* [(signedness << 4) | (endianess << 5) | bytewidth, tag] */
     RULE_LINE,         /* [tag] */
     RULE_COLUMN,       /* [tag] */
-    RULE_UNREF         /* [rule, tag] */
+    RULE_UNREF,        /* [rule, tag] */
+    RULE_CAPTURE_NUM   /* [rule, tag] */
 } JanetPegOpcod;
 
 typedef struct {

test/suite0008.janet

@@ -344,4 +344,8 @@ neldb\0\0\0\xD8\x05printG\x01\0\xDE\xDE\xDE'\x03\0marshal_tes/\x02
 (assert (deep= @[] (peg/match '(* "test" (any 1)) @"test")) "peg empty pattern 5")
 (assert (deep= @[] (peg/match '(* "test" (any 1)) (buffer "test"))) "peg empty pattern 6")
 
+# number pattern
+(assert (deep= @[111] (peg/match '(number :d+) "111")) "simple number capture 1")
+(assert (deep= @[255] (peg/match '(number :w+) "0xff")) "simple number capture 2")
+
 (end-suite)

test/suite0009.janet

@@ -152,41 +152,36 @@
   (:close s))
 
 # Test localname and peername
-(repeat 10
-  (defn check-matching-names [stream &opt direction]
-    "Checks that the remote agrees with the local about ip/port"
-    (let [[my-ip my-port]         (net/localname stream)
-          [remote-ip remote-port] (net/peername stream)
-          to-write                (string/join
-                                    @[my-ip (string my-port)
-                                      remote-ip (string remote-port)]
-                                    " ")
-          buffer                  @""]
-      (if (= direction :write)
-        (do (net/write stream to-write) (net/read stream 1024 buffer))
-        (do (net/read stream 1024 buffer) (net/write stream to-write)))
-      (def comparison (string/split " " buffer))
-      (assert (and (= my-ip (get comparison 2))
-                   (= (string my-port) (get comparison 3))
-                   (= remote-ip (get comparison 0))
-                   (= (string remote-port) (get comparison 1)))
-              "localname does not match peername")))
+(repeat 20
+
+  (defn check-matching-names [stream]
+    (def [my-ip my-port] (net/localname stream))
+    (def [remote-ip remote-port] (net/peername stream))
+    (def msg (string my-ip " " my-port " " remote-ip " " remote-port))
+    (def buf @"")
+    (ev/gather
+      (net/write stream msg)
+      (net/read stream 1024 buf))
+    (def comparison (string/split " " buf))
+    (assert (and (= my-ip (get comparison 2))
+                 (= (string my-port) (get comparison 3))
+                 (= remote-ip (get comparison 0))
+                 (= (string remote-port) (get comparison 1)))
+            "localname should match peername"))
+
+  # Test on both server and client
   (defn names-handler
-    "Simple handler for connections."
     [stream]
     (defer (:close stream)
       (check-matching-names stream)))
+  (with [s (net/server "127.0.0.1" "8000" names-handler)]
+    (defn test-names []
+      (with [conn (net/connect "127.0.0.1" "8000")]
+        (check-matching-names conn)))
+    (test-names)
+    (test-names))
 
-  (def s (net/server "127.0.0.1" "8000" names-handler))
-  (assert s "made server 1")
-
-  (defn test-names []
-    (with [conn (net/connect "127.0.0.1" "8000")]
-          (check-matching-names conn :write)))
-
-  (test-names)
-  (test-names)
-  (:close s))
+  (gccollect))
 
 # Create pipe