Allow IO redirection on windows.

This should help cover a number of common cases for
use of subprocesses. This should also eventually work well
with the ev branch via

.builds/meson.yml

@@ -10,3 +10,5 @@ tasks:
     cd build
     ninja
     ninja test
+    doas ninja install
+    doas jpm --verbose install circlet

.builds/meson2.yml

@@ -0,0 +1,15 @@
+image: openbsd/latest
+sources:
+- https://git.sr.ht/~bakpakin/janet
+packages:
+- meson
+tasks:
+- build: |
+    cd janet
+    meson setup build --buildtype=release
+    cd build
+    meson configure -Dprf=true
+    ninja
+    ninja test
+    doas ninja install
+    doas jpm --verbose install circlet

.builds/meson_min.yml

@@ -17,7 +17,6 @@ tasks:
     meson configure -Dpeg=false
     meson configure -Dassembler=false
     meson configure -Dint_types=false
-    meson configure -Dtyped_arrays=false
+    meson configure -Dtyped_array=false
     meson configure -Dreduced_os=true
-    meson configure -Dprf=false
     ninja # will not pass tests but should build

.gitattributes

@@ -0,0 +1 @@
+*.janet linguist-language=Clojure

CHANGELOG.md

@@ -1,6 +1,100 @@
 # Changelog
 All notable changes to this project will be documented in this file.
 
+## Unreleased - ???
+- Add `os/proc-wait` and `os/proc-kill` for interacting with processes.
+- Add `janet_getjfile` to C API.
+- Allow redirection of stdin, stdout, and stderr by passing keywords in the env table.
+- Add `:a` flag to `os/execute` to get a core/process back instead of an exit code.
+  When called like this, `os/execute` returns immediately.
+- Add `:x` flag to os/execute to raise error when exit code is non-zero.
+- Don't run `main` when flychecking.
+- Add `:n` flag to `file/open` to raise an error if file cannot be opened.
+- Fix import macro to not try and coerce everything to a string.
+- Allow passing a second argument to `disasm`.
+- Add `cancel`. Resumes a fiber but makes it immediately error at the yield point.
+- Allow multi-line paste into built in repl.
+- Add `(curenv)`.
+- Change `net/read`, `net/chunk`, and `net/write` to raise errors in the case of failures.
+- Add `janet_continue_signal` to C API. This indirectly enables C functions that yield to the event loop
+  to raise errors or other signals.
+- Update meson build script to fix bug on Debian's version of meson
+- Add `xprint`, `xprin`, `xprintf`, and `xprinf`.
+- `net/write` now raises an error message if write fails.
+- Fix issue with SIGPIPE on macOS and BSDs.
+
+## 1.11.3 - 2020-08-03
+- Add `JANET_HASHSEED` environment variable when `JANET_PRF` is enabled.
+- Expose `janet_cryptorand` in C API.
+- Properly initialize PRF in default janet program
+- Add `index-of` to core library.
+- Add `-fPIC` back to core CFLAGS (non-optional when compiling default client with Makefile)
+- Fix defaults on Windows for ARM
+- Fix defaults on NetBSD.
+
+## 1.11.1 - 2020-07-25
+- Fix jpm and git with multiple git installs on Windows
+- Fix importing a .so file in the current directory
+- Allow passing byte sequence types directly to typed-array constructors.
+- Fix bug sending files between threads.
+- Disable PRF by default.
+- Update the soname.
+
+## 1.11.0 - 2020-07-18
+- Add `forever` macro.
+- Add `any?` predicate to core.
+- Add `jpm list-pkgs` subcommand to see which package aliases are in the listing.
+- Add `jpm list-installed` subcommand to see which packages are installed.
+- Add `math/int-min`, `math/int-max`, `math/int32-min`, and `math/int32-max` for getting integer limits.
+- The gc interval is now autotuned, to prevent very bad gc behavior.
+- Improvements to the bytecode compiler, Janet will now generate more efficient bytecode.
+- Add `peg/find`, `peg/find-all`, `peg/replace`, and `peg/replace-all`
+- Add `math/nan`
+- Add `forv` macro
+- Add `symbol/slice`
+- Add `keyword/slice`
+- Allow cross compilation with Makefile.
+- Change `compare-primitve` to `cmp` and make it more efficient.
+- Add `reverse!` for reversing an array or buffer in place.
+- `janet_dobytes` and `janet_dostring` return parse errors in \*out
+- Add `repeat` macro for iterating something n times.
+- Add `eachy` (each yield) macro for iterating a fiber.
+- Fix `:generate` verb in loop macro to accept non symbols as bindings.
+- Add `:h`, `:h+`, and `:h*` in `default-peg-grammar` for hexidecimal digits.
+- Fix `%j` formatter to print numbers precisely (using the `%.17g` format string to printf).
+
+## 1.10.1 - 2020-06-18
+- Expose `janet_table_clear` in API.
+- Respect `JANET_NO_PROCESSES` define when building
+- Fix `jpm` rules having multiple copies of the same dependency.
+- Fix `jpm` install in some cases.
+- Add `array/trim` and `buffer/trim` to shrink the backing capacity of these types
+  to their current length.
+
+## 1.10.0 - 2020-06-14
+- Hardcode default jpm paths on install so env variables are needed in fewer cases.
+- Add `:no-compile` to `create-executable` option for jpm.
+- Fix bug with the `trace` function.
+- Add `:h`, `:a`, and `:c` flags to `thread/new` for creating new kinds of threads.
+  By default, threads will now consume much less memory per thread, but sending data between
+  threads may cost more.
+- Fix flychecking when using the `use` macro.
+- CTRL-C no longer exits the repl, and instead cancels the current form.
+- Various small bug fixes
+- New MSI installer instead of NSIS based installer.
+- Make `os/realpath` work on windows.
+- Add polymorphic `compare` functions for comparing numbers.
+- Add `to` and `thru` peg combinators.
+- Add `JANET_GIT` environment variable to jpm to use a specific git binary (useful mainly on windows).
+- `asm` and `disasm` functions now use keywords instead of macros for keys. Also
+  some slight changes to the way constants are encoded (remove wrapping `quote` in some cases).
+- Expose current macro form inside macros as (dyn :macro-form)
+- Add `tracev` macro.
+- Fix compiler bug that emitted incorrect code in some cases for while loops that create closures.
+- Add `:fresh` option to `(import ...)` to overwrite the module cache.
+- `(range x y 0)` will return an empty array instead of hanging forever.
+- Rename `jpm repl` to `jpm debug-repl`.
+
 ## 1.9.1 - 2020-05-12
 - Add :prefix option to declare-source
 - Re-enable minimal builds with the debugger.

CONTRIBUTING.md

@@ -35,8 +35,9 @@ may require changes before being merged.
   [astyle](http://astyle.sourceforge.net/astyle.html). You will probably need
   to install this, but it can be installed with most package managers.
 
-  For janet code, the use lisp indentation with 2 spaces. One can use janet.vim to
+  For janet code, use lisp indentation with 2 spaces. One can use janet.vim to
-  do this indentation, or approximate as close as possible.
+  do this indentation, or approximate as close as possible. There is a janet formatter
+  in [spork](https://github.com/janet-lang/spork.git) that can be used to format code as well.
 
 ## C style
 

Makefile

@@ -33,13 +33,20 @@ JANET_TARGET=build/janet
 JANET_LIBRARY=build/libjanet.so
 JANET_STATIC_LIBRARY=build/libjanet.a
 JANET_PATH?=$(LIBDIR)/janet
-MANPATH?=$(PREFIX)/share/man/man1/
+JANET_MANPATH?=$(PREFIX)/share/man/man1/
-PKG_CONFIG_PATH?=$(LIBDIR)/pkgconfig
+JANET_PKG_CONFIG_PATH?=$(LIBDIR)/pkgconfig
 DEBUGGER=gdb
 SONAME_SETTER=-Wl,-soname,
 
-CFLAGS:=$(CFLAGS) -std=c99 -Wall -Wextra -Isrc/include -Isrc/conf -fPIC -O2 -fvisibility=hidden
+# For cross compilation
-LDFLAGS:=$(LDFLAGS) -rdynamic
+HOSTCC?=$(CC)
+HOSTAR?=$(AR)
+CFLAGS?=-O2
+LDFLAGS?=-rdynamic
+
+COMMON_CFLAGS:=-std=c99 -Wall -Wextra -Isrc/include -Isrc/conf -fvisibility=hidden -fPIC
+BOOT_CFLAGS:=-DJANET_BOOTSTRAP -DJANET_BUILD=$(JANET_BUILD) -O0 -g $(COMMON_CFLAGS)
+BUILD_CFLAGS:=$(CFLAGS) $(COMMON_CFLAGS)
 
 # For installation
 LDCONFIG:=ldconfig "$(LIBDIR)"
@@ -131,7 +138,6 @@ JANET_BOOT_HEADERS=src/boot/tests.h
 ##########################################################
 
 JANET_BOOT_OBJECTS=$(patsubst src/%.c,build/%.boot.o,$(JANET_CORE_SOURCES) $(JANET_BOOT_SOURCES))
-BOOT_CFLAGS:=-DJANET_BOOTSTRAP -DJANET_BUILD=$(JANET_BUILD) $(CFLAGS)
 
 $(JANET_BOOT_OBJECTS): $(JANET_BOOT_HEADERS)
 
@@ -149,7 +155,7 @@ build/janet.c: build/janet_boot src/boot/boot.janet
 ##### Amalgamation #####
 ########################
 
-SONAME=libjanet.so.1.9
+SONAME=libjanet.so.1.11
 
 build/shell.c: src/mainclient/shell.c
 	cp $< $@
@@ -161,24 +167,26 @@ build/janetconf.h: src/conf/janetconf.h
 	cp $< $@
 
 build/janet.o: build/janet.c build/janet.h build/janetconf.h
-	$(CC) $(CFLAGS) -c $< -o $@ -I build
+	$(HOSTCC) $(BUILD_CFLAGS) -c $< -o $@ -I build
 
 build/shell.o: build/shell.c build/janet.h build/janetconf.h
-	$(CC) $(CFLAGS) -c $< -o $@ -I build
+	$(HOSTCC) $(BUILD_CFLAGS) -c $< -o $@ -I build
 
 $(JANET_TARGET): build/janet.o build/shell.o
-	$(CC) $(LDFLAGS) $(CFLAGS) -o $@ $^ $(CLIBS)
+	$(HOSTCC) $(LDFLAGS) $(BUILD_CFLAGS) -o $@ $^ $(CLIBS)
 
 $(JANET_LIBRARY): build/janet.o build/shell.o
-	$(CC) $(LDFLAGS) $(CFLAGS) $(SONAME_SETTER)$(SONAME) -shared -o $@ $^ $(CLIBS)
+	$(HOSTCC) $(LDFLAGS) $(BUILD_CFLAGS) $(SONAME_SETTER)$(SONAME) -shared -o $@ $^ $(CLIBS)
 
 $(JANET_STATIC_LIBRARY): build/janet.o build/shell.o
-	$(AR) rcs $@ $^
+	$(HOSTAR) rcs $@ $^
 
 ###################
 ##### Testing #####
 ###################
 
+# Testing assumes HOSTCC=CC
+
 TEST_SCRIPTS=$(wildcard test/suite*.janet)
 
 repl: $(JANET_TARGET)
@@ -233,6 +241,10 @@ build/doc.html: $(JANET_TARGET) tools/gendoc.janet
 ##### Installation #####
 ########################
 
+build/jpm: jpm $(JANET_TARGET)
+	$(JANET_TARGET) tools/patch-jpm.janet jpm build/jpm "--libpath=$(LIBDIR)" "--headerpath=$(INCLUDEDIR)/janet" "--binpath=$(BINDIR)"
+	chmod +x build/jpm
+
 .INTERMEDIATE: build/janet.pc
 build/janet.pc: $(JANET_TARGET)
 	echo 'prefix=$(PREFIX)' > $@
@@ -248,7 +260,7 @@ build/janet.pc: $(JANET_TARGET)
 	echo 'Libs: -L$${libdir} -ljanet' >> $@
 	echo 'Libs.private: $(CLIBS)' >> $@
 
-install: $(JANET_TARGET) build/janet.pc
+install: $(JANET_TARGET) build/janet.pc build/jpm
 	mkdir -p '$(DESTDIR)$(BINDIR)'
 	cp $(JANET_TARGET) '$(DESTDIR)$(BINDIR)/janet'
 	mkdir -p '$(DESTDIR)$(INCLUDEDIR)/janet'
@@ -259,12 +271,12 @@ install: $(JANET_TARGET) build/janet.pc
 	cp $(JANET_STATIC_LIBRARY) '$(DESTDIR)$(LIBDIR)/libjanet.a'
 	ln -sf $(SONAME) '$(DESTDIR)$(LIBDIR)/libjanet.so'
 	ln -sf libjanet.so.$(shell $(JANET_TARGET) -e '(print janet/version)') $(DESTDIR)$(LIBDIR)/$(SONAME)
-	cp -rf jpm '$(DESTDIR)$(BINDIR)'
+	cp -rf build/jpm '$(DESTDIR)$(BINDIR)'
-	mkdir -p '$(DESTDIR)$(MANPATH)'
+	mkdir -p '$(DESTDIR)$(JANET_MANPATH)'
-	cp janet.1 '$(DESTDIR)$(MANPATH)'
+	cp janet.1 '$(DESTDIR)$(JANET_MANPATH)'
-	cp jpm.1 '$(DESTDIR)$(MANPATH)'
+	cp jpm.1 '$(DESTDIR)$(JANET_MANPATH)'
-	mkdir -p '$(DESTDIR)$(PKG_CONFIG_PATH)'
+	mkdir -p '$(DESTDIR)$(JANET_PKG_CONFIG_PATH)'
-	cp build/janet.pc '$(DESTDIR)$(PKG_CONFIG_PATH)/janet.pc'
+	cp build/janet.pc '$(DESTDIR)$(JANET_PKG_CONFIG_PATH)/janet.pc'
 	[ -z '$(DESTDIR)' ] && $(LDCONFIG) || true
 
 uninstall:
@@ -272,9 +284,9 @@ uninstall:
 	-rm '$(DESTDIR)$(BINDIR)/jpm'
 	-rm -rf '$(DESTDIR)$(INCLUDEDIR)/janet'
 	-rm -rf '$(DESTDIR)$(LIBDIR)'/libjanet.*
-	-rm '$(DESTDIR)$(PKG_CONFIG_PATH)/janet.pc'
+	-rm '$(DESTDIR)$(JANET_PKG_CONFIG_PATH)/janet.pc'
-	-rm '$(DESTDIR)$(MANPATH)/janet.1'
+	-rm '$(DESTDIR)$(JANET_MANPATH)/janet.1'
-	-rm '$(DESTDIR)$(MANPATH)/jpm.1'
+	-rm '$(DESTDIR)$(JANET_MANPATH)/jpm.1'
 	# -rm -rf '$(DESTDIR)$(JANET_PATH)'/* - err on the side of correctness here
 
 #################
@@ -306,5 +318,27 @@ test-install:
 	cd test/install && jpm --verbose --test --modpath=./modpath install https://github.com/janet-lang/path.git
 	cd test/install && jpm --verbose --test --modpath=./modpath install https://github.com/janet-lang/argparse.git
 
+help:
+	@echo
+	@echo 'Janet: A Dynamic Language & Bytecode VM'
+	@echo
+	@echo Usage:
+	@echo '   make            Build Janet'
+	@echo '   make repl       Start a REPL from a built Janet'
+	@echo
+	@echo '   make test       Test a built Janet'
+	@echo '   make valgrind   Assess Janet with Valgrind'
+	@echo '   make callgrind  Assess Janet with Valgrind, using Callgrind'
+	@echo '   make valtest    Run the test suite with Valgrind to check for memory leaks'
+	@echo '   make dist       Create a distribution tarball'
+	@echo '   make docs       Generate documentation'
+	@echo '   make debug      Run janet with GDB or LLDB'
+	@echo '   make install    Install into the current filesystem'
+	@echo '   make uninstall  Uninstall from the current filesystem'
+	@echo '   make clean      Clean intermediate build artifacts'
+	@echo "   make format     Format Janet's own source files"
+	@echo '   make grammar    Generate a TextMate language grammar'
+	@echo
+
 .PHONY: clean install repl debug valgrind test \
-	valtest emscripten dist uninstall docs grammar format
+	valtest dist uninstall docs grammar format help

README.md

@@ -2,15 +2,15 @@
  
 [![Appveyor Status](https://ci.appveyor.com/api/projects/status/bjraxrxexmt3sxyv/branch/master?svg=true)](https://ci.appveyor.com/project/bakpakin/janet/branch/master)
 [![Build Status](https://travis-ci.org/janet-lang/janet.svg?branch=master)](https://travis-ci.org/janet-lang/janet)
-[![builds.sr.ht status](https://builds.sr.ht/~bakpakin/janet/freebsd.yml.svg)](https://builds.sr.ht/~bakpakin/janet/freebsd.yml?)
+[![builds.sr.ht status](https://builds.sr.ht/~bakpakin/janet/commits/freebsd.yml.svg)](https://builds.sr.ht/~bakpakin/janet/commits/freebsd.yml?)
-[![builds.sr.ht status](https://builds.sr.ht/~bakpakin/janet/openbsd.yml.svg)](https://builds.sr.ht/~bakpakin/janet/openbsd.yml?)
+[![builds.sr.ht status](https://builds.sr.ht/~bakpakin/janet/commits/openbsd.yml.svg)](https://builds.sr.ht/~bakpakin/janet/commits/openbsd.yml?)
-[![builds.sr.ht status](https://builds.sr.ht/~bakpakin/janet/meson.yml.svg)](https://builds.sr.ht/~bakpakin/janet/meson.yml?)
+[![builds.sr.ht status](https://builds.sr.ht/~bakpakin/janet/commits/meson.yml.svg)](https://builds.sr.ht/~bakpakin/janet/commits/meson.yml?)
-[![builds.sr.ht status](https://builds.sr.ht/~bakpakin/janet/meson_min.yml.svg)](https://builds.sr.ht/~bakpakin/janet/meson_min.yml?)
+[![builds.sr.ht status](https://builds.sr.ht/~bakpakin/janet/commits/meson_min.yml.svg)](https://builds.sr.ht/~bakpakin/janet/commits/meson_min.yml?)
 
 <img src="https://raw.githubusercontent.com/janet-lang/janet/master/assets/janet-w200.png" alt="Janet logo" width=200 align="left">
 
 **Janet** is a functional and imperative programming language and bytecode interpreter. It is a
-modern lisp, but lists are replaced
+lisp-like language, but lists are replaced
 by other data structures (arrays, tables (hash table), struct (immutable hash table), tuples).
 The language also supports bridging to native code written in C, meta-programming with macros, and bytecode assembly.
 
@@ -37,7 +37,7 @@ Lua, but smaller than GNU Guile or Python.
 * Mutable and immutable arrays (array/tuple)
 * Mutable and immutable hashtables (table/struct)
 * Mutable and immutable strings (buffer/string)
-* Lisp Macros
+* Macros
 * Byte code interpreter with an assembly interface, as well as bytecode verification
 * Tailcall Optimization
 * Direct interop with C via abstract types and C functions
@@ -77,7 +77,7 @@ the SourceHut mirror is actively maintained.
 
 ## Building
 
-### macos and Unix-like
+### macOS and Unix-like
 
 The Makefile is non-portable and requires GNU-flavored make.
 
@@ -88,6 +88,8 @@ make test
 make repl
 ```
 
+Find out more about the available make targets by running `make help`.
+
 ### 32-bit Haiku
 
 32-bit Haiku build instructions are the same as the unix-like build instructions,
@@ -113,6 +115,11 @@ gmake test
 gmake repl
 ```
 
+### NetBSD
+
+NetBSD build instructions are the same as the FreeBSD build instuctions.
+Alternatively, install directly from packages, using `pkgin install janet`.
+
 ### Windows
 
 1. Install [Visual Studio](https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=Community&rel=15#) or [Visual Studio Build Tools](https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=BuildTools&rel=15#)
@@ -120,6 +127,13 @@ gmake repl
 3. Run `build_win` to compile janet.
 4. Run `build_win test` to make sure everything is working.
 
+To build an `.msi` installer executable, in addition to the above steps, you will have to:
+
+5. Install, or otherwise add to your PATH the [WiX 3.11 Toolset](https://github.com/wixtoolset/wix3/releases)
+6. run `build_win dist`
+
+Now you should have an `.msi`. You can run `build_win install` to install the `.msi`, or execute the file itself.
+
 ### Meson
 
 Janet also has a build file for [Meson](https://mesonbuild.com/), a cross platform build
@@ -137,6 +151,7 @@ cd janet
 meson setup build \
           --buildtype release \
           --optimization 2 \
+          --libdir /usr/local/lib \
           -Dgit_hash=$(git log --pretty=format:'%h' -n 1)
 ninja -C build
 
@@ -202,7 +217,7 @@ If installed, you can also run `man janet` and `man jpm` to get usage informatio
 Janet can be embedded in a host program very easily. The normal build
 will create a file `build/janet.c`, which is a single C file
 that contains all the source to Janet. This file, along with
-`src/include/janet.h` and `src/include/janetconf.h` can dragged into any C
+`src/include/janet.h` and `src/conf/janetconf.h` can be dragged into any C
 project and compiled into the project. Janet should be compiled with `-std=c99`
 on most compilers, and will need to be linked to the math library, `-lm`, and
 the dynamic linker, `-ldl`, if one wants to be able to load dynamic modules. If
@@ -222,10 +237,10 @@ Alternatively, check out [the #janet channel on Freenode](https://webchat.freeno
 
 ## FAQ
 
-### Why is my terminal is spitting out junk when I run the repl?
+### Why is my terminal spitting out junk when I run the repl?
 
 Make sure your terminal supports ANSI escape codes. Most modern terminals will
-support these, but some older terminals, windows consoles, or embedded terminals
+support these, but some older terminals, Windows consoles, or embedded terminals
 will not. If your terminal does not support ANSI escape codes, run the repl with
 the `-n` flag, which disables color output. You can also try the `-s` if further issues
 ensue.

appveyor.yml

@@ -19,10 +19,6 @@ init:
 
 install:
     - set JANET_BUILD=%appveyor_repo_commit:~0,7%
-    - choco install nsis -y -pre --version 3.05
-    # Replace makensis.exe and files with special long string build. This should
-    # prevent issues when setting PATH during installation.
-    - 7z e "tools\nsis-3.05-strlen_8192.zip" -o"C:\Program Files (x86)\NSIS\" -y
     - build_win all
     - refreshenv
     # We need to reload vcvars after refreshing
@@ -48,7 +44,7 @@ artifacts:
     - name: "janet-$(janet_outname)-windows-%platform%"
       path: dist
       type: Zip
-    - path: "janet-$(janet_outname)-windows-%platform%-installer.exe"
+    - path: "janet-$(janet_outname)-windows-%platform%-installer.msi"
       type: File
 
 deploy:

build_win.bat

@@ -121,18 +121,29 @@ copy tools\jpm.bat dist\jpm.bat
 
 @rem Create installer
 janet.exe -e "(->> janet/version (peg/match ''(* :d+ `.` :d+ `.` :d+)) first print)" > build\version.txt
-janet.exe -e "(print (= (os/arch) :x64))" > build\64bit.txt
+janet.exe -e "(print (os/arch))" > build\arch.txt
 set /p JANET_VERSION= < build\version.txt
-set /p SIXTYFOUR= < build\64bit.txt
+set /p BUILDARCH= < build\arch.txt
 echo "JANET_VERSION is %JANET_VERSION%"
-"C:\Program Files (x86)\NSIS\makensis.exe" /DVERSION=%JANET_VERSION% /DSIXTYFOUR=%SIXTYFOUR% janet-installer.nsi 
+if defined APPVEYOR_REPO_TAG_NAME (
+    set RELEASE_VERSION=%APPVEYOR_REPO_TAG_NAME%
+) else (
+    set RELEASE_VERSION=%JANET_VERSION%
+)
+if defined CI (
+    set WIXBIN="c:\Program Files (x86)\WiX Toolset v3.11\bin\"
+) else (
+    set WIXBIN=
+)
+%WIXBIN%candle.exe tools\msi\janet.wxs -arch %BUILDARCH% -out build\
+%WIXBIN%light.exe "-sice:ICE38" -b tools\msi -ext WixUIExtension build\janet.wixobj -out janet-%RELEASE_VERSION%-windows-%BUILDARCH%-installer.msi
 exit /b 0
 
 @rem Run the installer. (Installs to the local user with default settings)
 :INSTALL
-@echo Running Installer...
+FOR %%a in (janet-*-windows-*-installer.msi) DO (
-FOR %%a in (janet-*-windows-*-installer.exe) DO (
+    @echo Running Installer %%a...
-    %%a /S /CurrentUser
+    %%a /QN
 )
 exit /b 0
 

examples/numarray/numarray.c

@@ -23,7 +23,7 @@ static int num_array_gc(void *p, size_t s) {
     return 0;
 }
 
-Janet num_array_get(void *p, Janet key);
+int num_array_get(void *p, Janet key, Janet *out);
 void num_array_put(void *p, Janet key, Janet value);
 
 static const JanetAbstractType num_array_type = {
@@ -31,7 +31,8 @@ static const JanetAbstractType num_array_type = {
     num_array_gc,
     NULL,
     num_array_get,
-    num_array_put
+    num_array_put,
+    JANET_ATEND_PUT
 };
 
 static Janet num_array_new(int32_t argc, Janet *argv) {
@@ -81,21 +82,20 @@ static const JanetMethod methods[] = {
     {NULL, NULL}
 };
 
-Janet num_array_get(void *p, Janet key) {
+int num_array_get(void *p, Janet key, Janet *out) {
     size_t index;
-    Janet value;
     num_array *array = (num_array *)p;
     if (janet_checktype(key, JANET_KEYWORD))
-        return janet_getmethod(janet_unwrap_keyword(key), methods);
+        return janet_getmethod(janet_unwrap_keyword(key), methods, out);
     if (!janet_checkint(key))
         janet_panic("expected integer key");
     index = (size_t)janet_unwrap_integer(key);
     if (index >= array->size) {
-        value = janet_wrap_nil();
+        return 0;
     } else {
-        value = janet_wrap_number(array->data[index]);
+        *out = janet_wrap_number(array->data[index]);
     }
-    return value;
+    return 1;
 }
 
 static const JanetReg cfuns[] = {

examples/numarray/project.janet

@@ -0,0 +1,7 @@
+(declare-project
+  :name "numarray"
+  :description "Example c lib with abstract type")
+
+(declare-native
+  :name "numarray"
+  :source @["numarray.c"])

examples/numarray/test/numarray_tests.janet

@@ -1,10 +1,4 @@
-(import cook)
+(import build/numarray)
-
-(cook/make-native
-    :name "numarray"
-    :source @["numarray.c"])
-
-(import build/numarray :as numarray)
 
 (def a (numarray/new 30))
 (print (get a 20))

janet-installer.nsi

@@ -1,217 +0,0 @@
-# This file is invoked by build_win.bat
-# Relevant configuration variables are set there.
-
-SetCompressor /FINAL /SOLID lzma
-Unicode True
-
-!echo "Program Files: ${PROGRAMFILES}"
-!addplugindir "tools\"
-
-# Version
-!define PRODUCT_VERSION "${VERSION}.0"
-VIProductVersion "${PRODUCT_VERSION}"
-VIFileVersion "${PRODUCT_VERSION}"
-
-# Use the modern UI
-!define MULTIUSER_EXECUTIONLEVEL Highest
-!define MULTIUSER_MUI
-!define MULTIUSER_INSTALLMODE_COMMANDLINE
-!define MULTIUSER_INSTALLMODE_DEFAULT_REGISTRY_KEY "Software\Janet\${VERSION}"
-!define MULTIUSER_INSTALLMODE_DEFAULT_REGISTRY_VALUENAME ""
-!define MULTIUSER_INSTALLMODE_INSTDIR_REGISTRY_KEY "Software\Janet\${VERSION}"
-!define MULTIUSER_INSTALLMODE_INSTDIR_REGISTRY_VALUENAME ""
-!define MULTIUSER_INSTALLMODE_INSTDIR "Janet-${VERSION}"
-
-!if ${SIXTYFOUR} == "true"
-    !define MULTIUSER_USE_PROGRAMFILES64
-    !define PLATNAME "x64"
-!else
-    !define PLATNAME "x86"
-!endif
-
-# Includes
-!include "MultiUser.nsh"
-!include "MUI2.nsh"
-!include "LogicLib.nsh"
-
-# Basics
-Name "Janet"
-
-# Do some NSIS-fu to figure out at compile time if we are in appveyor
-!define OUTNAME $%APPVEYOR_REPO_TAG_NAME%
-!define "CHECK_${OUTNAME}"
-!define DOLLAR "$"
-!ifdef CHECK_${DOLLAR}%APPVEYOR_REPO_TAG_NAME%
-    # We are not in the appveyor environment, use version name
-    !define OUTNAME_PART ${VERSION}
-!else
-    # We are in appveyor, use git tag name for installer
-    !define OUTNAME_PART ${OUTNAME}
-!endif
-OutFile "janet-${OUTNAME_PART}-windows-${PLATNAME}-installer.exe"
-
-# Some Configuration
-!define APPNAME "Janet"
-!define DESCRIPTION "The Janet Programming Language"
-!define HELPURL "http://janet-lang.org"
-BrandingText "The Janet Programming Language"
-
-# Macros for setting registry values
-!define UNINST_KEY "Software\Microsoft\Windows\CurrentVersion\Uninstall\Janet-${VERSION}"
-!macro WriteEnv key value
-    ${If} $MultiUser.InstallMode == "AllUsers"
-        WriteRegExpandStr HKLM "SYSTEM\CurrentControlSet\Control\Session Manager\Environment" "${key}" "${value}"
-    ${Else}
-        WriteRegExpandStr HKCU "Environment" "${key}" "${value}"
-    ${EndIf}
-!macroend
-!macro DelEnv key
-    ${If} $MultiUser.InstallMode == "AllUsers"
-        DeleteRegValue HKLM "SYSTEM\CurrentControlSet\Control\Session Manager\Environment" "${key}"
-    ${Else}
-        DeleteRegValue HKCU "Environment" "${key}"
-    ${EndIf}
-!macroend
-
-# MUI Configuration
-!define MUI_ICON "assets\icon.ico"
-!define MUI_UNICON "assets\icon.ico"
-!define MUI_HEADERIMAGE
-!define MUI_HEADERIMAGE_BITMAP "assets\janet-w200.png"
-!define MUI_HEADERIMAGE_RIGHT
-!define MUI_ABORTWARNING
-
-# Show a welcome page first
-!insertmacro MUI_PAGE_WELCOME
-!insertmacro MUI_PAGE_LICENSE "LICENSE"
-
-# Pick Install Directory
-!insertmacro MULTIUSER_PAGE_INSTALLMODE
-!insertmacro MUI_PAGE_DIRECTORY
-!insertmacro MUI_PAGE_INSTFILES
-
-# Done
-!insertmacro MUI_PAGE_FINISH
-
-# Need to set a language.
-!insertmacro MUI_LANGUAGE "English"
-
-function .onInit
-    !insertmacro MULTIUSER_INIT
-functionEnd
-
-section "Janet" BfWSection
-
-    createDirectory "$INSTDIR\Library"
-    createDirectory "$INSTDIR\C"
-    createDirectory "$INSTDIR\bin"
-    createDirectory "$INSTDIR\docs"
-    setOutPath "$INSTDIR"
-
-    # Bin files
-    file /oname=bin\janet.exe dist\janet.exe
-    file /oname=logo.ico assets\icon.ico
-    file /oname=bin\jpm.janet .\jpm
-    file /oname=bin\jpm.bat tools\jpm.bat
-
-    # C headers and library files
-    file /oname=C\janet.h dist\janet.h
-    file /oname=C\janetconf.h dist\janetconf.h
-    file /oname=C\janet.lib dist\janet.lib
-    file /oname=C\janet.exp dist\janet.exp
-    file /oname=C\janet.c dist\janet.c
-    file /oname=C\libjanet.lib dist\libjanet.lib
-
-    # Documentation
-    file /oname=docs\docs.html dist\doc.html
-
-    # Other
-    file README.md
-    file LICENSE
-
-    # Uninstaller - See function un.onInit and section "uninstall" for configuration
-    writeUninstaller "$INSTDIR\uninstall.exe"
-
-    # Start Menu
-    createShortCut "$SMPROGRAMS\Janet.lnk" "$INSTDIR\bin\janet.exe" "" "$INSTDIR\logo.ico"
-
-    # Update path
-    ${If} $MultiUser.InstallMode == "AllUsers"
-        EnVar::SetHKLM
-    ${Else}
-        EnVar::SetHKCU
-    ${EndIf}
-    EnVar::AddValue "PATH" "$INSTDIR\bin"
-    Pop $0
-
-    # Set up Environment variables
-    !insertmacro WriteEnv JANET_PATH "$INSTDIR\Library"
-    !insertmacro WriteEnv JANET_HEADERPATH "$INSTDIR\C"
-    !insertmacro WriteEnv JANET_LIBPATH "$INSTDIR\C"
-    !insertmacro WriteEnv JANET_BINPATH "$INSTDIR\bin"
-
-    SendMessage ${HWND_BROADCAST} ${WM_WININICHANGE} 0 "STR:Environment" /TIMEOUT=5000
-
-    # Registry information for add/remove programs
-    WriteRegStr SHCTX "${UNINST_KEY}" "DisplayName" "Janet"
-    WriteRegStr SHCTX "${UNINST_KEY}" "InstallLocation" "$INSTDIR"
-    WriteRegStr SHCTX "${UNINST_KEY}" "DisplayIcon" "$INSTDIR\logo.ico"
-    WriteRegStr SHCTX "${UNINST_KEY}" "Publisher" "Janet-Lang.org"
-    WriteRegStr SHCTX "${UNINST_KEY}" "HelpLink" "${HELPURL}"
-    WriteRegStr SHCTX "${UNINST_KEY}" "URLUpdateInfo" "${HELPURL}"
-    WriteRegStr SHCTX "${UNINST_KEY}" "URLInfoAbout" "${HELPURL}"
-    WriteRegStr SHCTX "${UNINST_KEY}" "DisplayVersion" "${VERSION}"
-    WriteRegDWORD SHCTX "${UNINST_KEY}" "NoModify" 1
-    WriteRegDWORD SHCTX "${UNINST_KEY}" "NoRepair" 1
-    WriteRegDWORD SHCTX "${UNINST_KEY}" "EstimatedSize" 1000
-    # Add uninstall
-    WriteRegStr SHCTX "${UNINST_KEY}" "UninstallString" "$\"$INSTDIR\uninstall.exe$\" /$MultiUser.InstallMode"
-    WriteRegStr SHCTX "${UNINST_KEY}" "QuietUninstallString" "$\"$INSTDIR\uninstall.exe$\" /$MultiUser.InstallMode /S"
-
-sectionEnd
-
-# Uninstaller
-
-function un.onInit
-    !insertmacro MULTIUSER_UNINIT
-functionEnd
-
-section "uninstall"
-
-    # Remove Start Menu launcher
-    delete "$SMPROGRAMS\Janet.lnk"
-
-    # Remove files
-    delete "$INSTDIR\logo.ico"
-    delete "$INSTDIR\README.md"
-    delete "$INSTDIR\LICENSE"
-    rmdir /r "$INSTDIR\Library"
-    rmdir /r "$INSTDIR\bin"
-    rmdir /r "$INSTDIR\C"
-    rmdir /r "$INSTDIR\docs"
-
-    # Remove env vars
-    !insertmacro DelEnv JANET_PATH
-    !insertmacro DelEnv JANET_HEADERPATH
-    !insertmacro DelEnv JANET_LIBPATH
-    !insertmacro DelEnv JANET_BINPATH
-
-    # Unset PATH
-    ${If} $MultiUser.InstallMode == "AllUsers"
-        EnVar::SetHKLM
-    ${Else}
-        EnVar::SetHKCU
-    ${EndIf}
-    EnVar::DeleteValue "PATH" "$INSTDIR\bin"
-    Pop $0
-
-    # make sure windows knows about the change
-    SendMessage ${HWND_BROADCAST} ${WM_WININICHANGE} 0 "STR:Environment" /TIMEOUT=5000
-
-    # Always delete uninstaller as the last action
-    delete "$INSTDIR\uninstall.exe"
-
-    # Remove uninstaller information from the registry
-    DeleteRegKey SHCTX "${UNINST_KEY}"
-
-sectionEnd

janet.1

@@ -13,8 +13,8 @@ janet \- run the Janet language abstract machine
 .BR args ...
 .SH DESCRIPTION
 Janet is a functional and imperative programming language and bytecode interpreter.
-It is a modern lisp, but lists are replaced by other data structures with better utility
+It is a Lisp-like language, but lists are replaced by other data structures
-and performance (arrays, tables, structs, tuples). The language also features bridging
+(arrays, tables, structs, tuples). The language also features bridging
 to native code written in C, meta-programming with macros, and bytecode assembly.
 
 There is a repl for trying out the language, as well as the ability to run script files.
@@ -194,8 +194,8 @@ Source should be a path to the Janet module to compile, and output should be the
 resulting image. Output should usually end with the .jimage extension.
 
 .TP
-.BR \-l\ path
+.BR \-l\ lib
-Load a Janet file before running a script or repl. Multiple files can be loaded
+Import a Janet module before running a script or repl. Multiple files can be loaded
 in this manner, and exports from each file will be made available to the script
 or repl.
 
@@ -213,5 +213,13 @@ find native and source code modules. If no JANET_PATH is set, Janet will look in
 the default location set at compile time.
 .RE
 
+.B JANET_HASHSEED
+.RS
+To disable randomization of Janet's PRF on start up, one can set this variable. This can have the
+effect of making programs deterministic that otherwise would depend on the random seed chosen at program start.
+This variable does nothing in the default configuration of Janet, as PRF is disabled by default. Also, JANET_REDUCED_OS
+cannot be defined for this variable to have an effect.
+.RE
+
 .SH AUTHOR
 Written by Calvin Rose <calsrose@gmail.com>

jpm

@@ -19,6 +19,9 @@
 # Defaults
 #
 
+###START###
+
+# Overriden on some installs.
 (def- exe-dir
   "Directory containing jpm script"
   (do
@@ -26,21 +29,28 @@
     (def i (last (string/find-all sep exe)))
     (slice exe 0 i)))
 
-(def JANET_MODPATH (or (os/getenv "JANET_MODPATH") (dyn :syspath)))
+(defn- install-paths []
+  {:headerpath (os/realpath (string exe-dir "/../include/janet"))
+   :libpath (os/realpath (string exe-dir "/../lib"))
+   :binpath exe-dir})
+
+###END###
 
 # Default based on janet binary location
 (def JANET_HEADERPATH (or (os/getenv "JANET_HEADERPATH")
-                          (string exe-dir "/../include/janet")))
+                          (get (install-paths) :headerpath)))
 (def JANET_LIBPATH (or (os/getenv "JANET_LIBPATH")
-                       (string exe-dir "/../lib")))
+                       (get (install-paths) :libpath)))
-
 # We want setting JANET_PATH to contain installed binaries. However, it is convenient
 # to have globally installed binaries got to the same place as jpm itself, which is on
 # the $PATH.
 (def JANET_BINPATH (or (os/getenv "JANET_BINPATH")
                        (if-let [mp (os/getenv "JANET_MODPATH")] (string mp "/bin"))
                        (if-let [mp (os/getenv "JANET_PATH")] (string mp "/bin"))
-                       exe-dir))
+                       (get (install-paths) :binpath)))
+
+# modpath should only be derived from the syspath being used or an environment variable.
+(def JANET_MODPATH (or (os/getenv "JANET_MODPATH") (dyn :syspath)))
 
 #
 # Utilities
@@ -66,11 +76,13 @@
 (defn rm
   "Remove a directory and all sub directories."
   [path]
-  (if (= (os/lstat path :mode) :directory)
+  (case (os/lstat path :mode)
-    (do
+    :directory (do
       (each subpath (os/dir path)
         (rm (string path sep subpath)))
       (os/rmdir path))
+    nil nil # do nothing if file does not exist
+    # Default, try to remove
     (os/rm path)))
 
 (defn- rimraf
@@ -78,16 +90,24 @@
   [path]
   (if is-win
     # windows get rid of read-only files
-    (os/shell `rmdir /S /Q "` path `"`))
+    (when (os/stat path :mode)
-  (rm path))
+      (os/shell (string `rmdir /S /Q "` path `"`)))
+    (rm path)))
 
 (defn clear-cache
   "Clear the global git cache."
   []
   (def cache (find-cache))
-  (print "clearing " cache "...")
+  (print "clearing cache " cache "...")
   (rimraf cache))
 
+(defn clear-manifest
+  "Clear the global installation manifest."
+  []
+  (def manifest (find-manifest-dir))
+  (print "clearing manifests " manifest "...")
+  (rimraf manifest))
+
 (def- default-pkglist
   (or (os/getenv "JANET_PKGLIST") "https://github.com/janet-lang/pkgs.git"))
 
@@ -134,7 +154,9 @@
   (if is-win
     (let [end (last (peg/match path-splitter src))
           isdir (= (os/stat src :mode) :directory)]
-      (shell "xcopy" src (if isdir (string dest "\\" end) dest) "/y" "/s" "/e" "/i"))
+      (shell "C:\\Windows\\System32\\xcopy.exe"
+             (string/replace "/" "\\" src) (string/replace "/" "\\" (if isdir (string dest "\\" end) dest))
+             "/y" "/s" "/e" "/i"))
     (shell "cp" "-rf" src dest)))
 
 (defn mkdir
@@ -166,9 +188,27 @@
   (unless item (error (string "No rule for target " target)))
   item)
 
+(defn add-dep
+  "Add a dependency to an existing rule. Useful for extending phony
+  rules or extending the dependency graph of existing rules."
+  [target dep]
+  (def [deps] (gettarget target))
+  (unless (find |(= dep $) deps)
+    (array/push deps dep)))
+
+(defn- add-thunk
+  [target more &opt phony]
+  (def item (gettarget target))
+  (def [_ thunks pthunks] item)
+  (array/push (if phony pthunks thunks) more)
+  item)
+
 (defn- rule-impl
   [target deps thunk &opt phony]
-  (put (getrules) target @[(array/slice deps) @[thunk] phony]))
+  (def rules (getrules))
+  (unless (rules target) (put rules target @[(array/slice deps) @[] @[]]))
+  (each d deps (add-dep target d))
+  (add-thunk target thunk phony))
 
 (defmacro rule
   "Add a rule to the rule graph."
@@ -192,20 +232,6 @@
   [target deps & body]
   ~(,rule-impl ,target ,deps (fn [] (,assert (,zero? (,os/shell (,string ,;body))))) true))
 
-(defn add-dep
-  "Add a dependency to an existing rule. Useful for extending phony
-  rules or extending the dependency graph of existing rules."
-  [target dep]
-  (def [deps] (gettarget target))
-  (array/push deps dep))
-
-(defn- add-thunk
-  [target more]
-  (def item (gettarget target))
-  (def [_ thunks] item)
-  (array/push thunks more)
-  item)
-
 (defmacro add-body
   "Add recipe code to an existing rule. This makes existing rules do more but
   does not modify the dependency graph."
@@ -235,9 +261,11 @@
       (error (string "No rule for file " target " found."))))
   (def [deps thunks phony] item)
   (def realdeps (seq [dep :in deps :let [x (do-rule dep)] :when x] x))
-  (when (or phony (needs-build-some target realdeps))
+  (each thunk phony (thunk))
-    (each thunk thunks (thunk)))
+  (unless (empty? thunks)
-  (unless phony target))
+    (when (needs-build-some target realdeps)
+      (each thunk thunks (thunk))
+    target)))
 
 #
 # Importing a file
@@ -301,26 +329,30 @@
 # Detect threads
 (def env (fiber/getenv (fiber/current)))
 (def threads? (not (not (env 'thread/new))))
-
-# Default libraries to link
 (def- thread-flags
   (if is-win []
     (if threads? ["-lpthread"] [])))
 
-# lflags needed for the janet binary.
+# flags needed for the janet binary and compiling standalone
+# executables.
 (def janet-lflags
   (case (os/which)
     :macos ["-ldl" "-lm" ;thread-flags]
     :windows [;thread-flags]
     :linux ["-lm" "-ldl" "-lrt" ;thread-flags]
     ["-lm" ;thread-flags]))
+(def janet-ldflags [])
+(def janet-cflags [])
 
 # Default flags for natives, but not required
+# How can we better detect the need for -pthread?
+# we probably want to better detect compiler
 (def default-lflags (if is-win ["/nologo"] []))
 (def default-cflags
   (if is-win
     ["/nologo" "/MD"]
     ["-std=c99" "-Wall" "-Wextra"]))
+(def default-ldflags [])
 
 # Required flags for dynamic libraries. These
 # are used no matter what for dynamic libraries.
@@ -330,7 +362,7 @@
     ["-fPIC"]))
 (def- dynamic-lflags
   (if is-win
-    ["/DLL" ;thread-flags]
+    ["/DLL"]
     (if is-mac
       ["-shared" "-undefined" "dynamic_lookup" ;thread-flags]
       ["-shared" ;thread-flags])))
@@ -376,8 +408,8 @@
   "Generate strings for adding custom defines to the compiler."
   [define value]
   (if value
-    (string (if is-win "/D" "-D") define "=" value)
+    (string "-D" define "=" value)
-    (string (if is-win "/D" "-D") define)))
+    (string "-D" define)))
 
 (defn- make-defines
   "Generate many defines. Takes a dictionary of defines. If a value is
@@ -389,8 +421,8 @@
   "Generate the c flags from the input options."
   [opts]
   @[;(opt opts :cflags default-cflags)
-    (string (if is-win "/I" "-I") (dyn :headerpath JANET_HEADERPATH))
+    (string "-I" (dyn :headerpath JANET_HEADERPATH))
-    (string (if is-win "/O" "-O") (opt opts :optimize 2))])
+    (string "-O" (opt opts :optimize 2))])
 
 (defn- entry-name
   "Name of symbol that enters static compilation of a module."
@@ -503,6 +535,23 @@
     ```
 
 int main(int argc, const char **argv) {
+
+#if defined(JANET_PRF)
+    uint8_t hash_key[JANET_HASH_KEY_SIZE + 1];
+#ifdef JANET_REDUCED_OS
+    char *envvar = NULL;
+#else
+    char *envvar = getenv("JANET_HASHSEED");
+#endif
+    if (NULL != envvar) {
+        strncpy((char *) hash_key, envvar, sizeof(hash_key) - 1);
+    } else if (janet_cryptorand(hash_key, JANET_HASH_KEY_SIZE) != 0) {
+        fputs("unable to initialize janet PRF hash function.\n", stderr);
+        return 1;
+    }
+    janet_init_hash_key(hash_key);
+#endif
+
     janet_init();
 
     /* Get core env */
@@ -576,7 +625,8 @@ int main(int argc, const char **argv) {
 
   # Create executable's janet image
   (def cimage_dest (string dest ".c"))
-  (rule dest [source]
+  (def no-compile (opts :no-compile))
+  (rule (if no-compile cimage_dest dest) [source]
         (check-cc)
         (print "generating executable c source...")
         (create-dirs dest)
@@ -632,11 +682,11 @@ int main(int argc, const char **argv) {
         # Append main function
         (spit cimage_dest (make-bin-source declarations lookup-into-invocations) :ab)
         # Compile and link final exectable
-        (do
+        (unless no-compile
           (def cc (opt opts :compiler default-compiler))
-          (def ldflags [;dep-ldflags ;(opt opts :ldflags [])])
+          (def ldflags [;dep-ldflags ;(opt opts :ldflags []) ;janet-ldflags])
           (def lflags [;static-libs (libjanet) ;dep-lflags ;(opt opts :lflags default-lflags) ;janet-lflags])
-          (def cflags (getcflags opts))
+          (def cflags [;(getcflags opts) ;janet-cflags])
           (def defines (make-defines (opt opts :defines {})))
           (print "compiling and linking " dest "...")
           (if is-win
@@ -647,6 +697,17 @@ int main(int argc, const char **argv) {
 # Installation and Dependencies
 #
 
+(var- stored-git-path nil)
+(defn- git-path
+    "Get the location of git such that it can be passed as an argument to os/execute."
+    "(Some builds/configurations of windows don't like just the string 'git')"
+    []
+    (if stored-git-path (break stored-git-path))
+    (set stored-git-path
+         (if is-win
+           (or (os/getenv "JANET_GIT") (first (string/split "\n" (pslurp "where git"))))
+           (os/getenv "JANET_GIT" "git"))))
+
 (defn uninstall
   "Uninstall bundle named name"
   [name]
@@ -656,7 +717,8 @@ int main(int argc, const char **argv) {
     (each path (get man :paths [])
       (print "removing " path)
       (rm path))
-    (print "removing " manifest)
+    (print "removing manifest " manifest)
+    (:close f) # I hate windows
     (rm manifest)
     (print "Uninstalled.")))
 
@@ -695,7 +757,7 @@ int main(int argc, const char **argv) {
     (when (mkdir module-dir)
       (set fresh true)
       (print "cloning repository " repo " to " module-dir)
-      (unless (zero? (os/execute ["git" "clone" repo module-dir] :p))
+      (unless (zero? (os/execute [(git-path) "clone" repo module-dir] :p))
         (rimraf module-dir)
         (error (string "could not clone git dependency " repo)))))
   (def olddir (os/cwd))
@@ -707,16 +769,16 @@ int main(int argc, const char **argv) {
                 :binpath (abspath (dyn :binpath JANET_BINPATH))]
       (os/cd module-dir)
       (unless fresh
-        (os/execute ["git" "pull" "origin" "master"] :p))
+        (os/execute [(git-path) "pull" "origin" "master" "--ff-only"] :p))
       (when tag
-        (os/execute ["git" "reset" "--hard" tag] :p))
+        (os/execute [(git-path) "reset" "--hard" tag] :p))
       (unless (dyn :offline)
-        (os/execute ["git" "submodule" "update" "--init" "--recursive"] :p))
+        (os/execute [(git-path) "submodule" "update" "--init" "--recursive"] :p))
-      (import-rules "./project.janet")
+      (import-rules "./project.janet" true)
       (unless no-deps (do-rule "install-deps"))
       (do-rule "build")
       (do-rule "install"))
-    ([err] (print "Error building git repository dependency: " err)))
+    ([err f] (print "Error building git repository dependency: " err) (propagate err f)))
   (os/cd olddir))
 
 (defn install-rule
@@ -726,9 +788,9 @@ int main(int argc, const char **argv) {
   (def name (last parts))
   (def path (string destdir sep name))
   (array/push (dyn :installed-files) path)
-  (add-body "install"
+  (phony "install" []
-            (mkdir destdir)
+          (mkdir destdir)
-            (copy src destdir)))
+          (copy src destdir)))
 
 (defn- make-lockfile
   [&opt filename]
@@ -758,8 +820,15 @@ int main(int argc, const char **argv) {
     (unless made-progress
       (error (string/format "could not resolve package order for: %j"
                             (filter (complement resolved) (map |($ :repo) packages))))))
-  # Write to file
+  # Write to file, manual format for better diffs.
-  (with [f (file/open filename :w)] (with-dyns [:out f] (printf "%j" ordered-packages))))
+  (with [f (file/open filename :w)]
+    (with-dyns [:out f]
+      (prin "@[")
+      (eachk i ordered-packages
+        (unless (zero? i)
+          (prin "\n  "))
+        (prinf "%j" (ordered-packages i)))
+      (print "]"))))
 
 (defn- load-lockfile
   [&opt filename]
@@ -856,25 +925,46 @@ int main(int argc, const char **argv) {
   is marshalled into bytecode which is then embedded in a final executable for distribution.\n\n
   This executable can be installed as well to the --binpath given."
   [&keys {:install install :name name :entry entry :headers headers
-          :cflags cflags :lflags lflags :deps deps :ldflags ldflags}]
+          :cflags cflags :lflags lflags :deps deps :ldflags ldflags
+          :no-compile no-compile}]
   (def name (if is-win (string name ".exe") name))
   (def dest (string "build" sep name))
-  (create-executable @{:cflags cflags :lflags lflags :ldflags ldflags} entry dest)
+  (create-executable @{:cflags cflags :lflags lflags :ldflags ldflags :no-compile no-compile} entry dest)
-  (add-dep "build" dest)
+  (if no-compile
-  (when headers
+    (let [cdest (string dest ".c")]
-    (each h headers (add-dep dest h)))
+      (add-dep "build" cdest))
-  (when deps
+    (do
-    (each d deps (add-dep dest d)))
+      (add-dep "build" dest)
-  (when install
+      (when headers
-    (install-rule dest (dyn :binpath JANET_BINPATH))))
+        (each h headers (add-dep dest h)))
+      (when deps
+        (each d deps (add-dep dest d)))
+      (when install
+        (install-rule dest (dyn :binpath JANET_BINPATH))))))
 
 (defn declare-binscript
   "Declare a janet file to be installed as an executable script. Creates
-  a shim on windows."
+  a shim on windows. If hardcode is true, will insert code into the script
-  [&keys opts]
+  such that it will run correctly even when JANET_PATH is changed."
-  (def main (opts :main))
+  [&keys {:main main :hardcode-syspath hardcode}]
   (def binpath (dyn :binpath JANET_BINPATH))
-  (install-rule main binpath)
+  (if hardcode
+    (let [syspath (dyn :modpath JANET_MODPATH)]
+      (def parts (peg/match path-splitter main))
+      (def name (last parts))
+      (def path (string binpath sep name))
+      (array/push (dyn :installed-files) path)
+      (phony "install" []
+                (def contents
+                  (with [f (file/open main)]
+                    (def first-line (:read f :line))
+                    (def second-line (string/format "(put root-env :syspath %v)\n" syspath))
+                    (def rest (:read f :all))
+                    (string first-line second-line rest)))
+                (create-dirs path)
+                (spit path contents)
+                (unless is-win (shell "chmod" "+x" path))))
+    (install-rule main binpath))
   # Create a dud batch file when on windows.
   (when is-win
     (def name (last (peg/match path-splitter main)))
@@ -882,7 +972,7 @@ int main(int argc, const char **argv) {
     (def bat (string "@echo off\r\njanet \"" fullname "\" %*"))
     (def newname (string binpath sep name ".bat"))
     (array/push (dyn :installed-files) newname)
-    (add-body "install"
+    (phony "install" []
               (spit newname bat))))
 
 (defn- print-rule-tree
@@ -929,11 +1019,12 @@ int main(int argc, const char **argv) {
 
   (phony "build" [])
 
-  (phony "manifest" []
+  (phony "manifest" [manifest])
+  (rule manifest []
          (print "generating " manifest "...")
          (mkdir manifests)
-         (def sha (pslurp "git rev-parse HEAD"))
+         (def sha (pslurp (string "\"" (git-path) "\" rev-parse HEAD")))
-         (def url (pslurp "git remote get-url origin"))
+         (def url (pslurp (string "\"" (git-path) "\" remote get-url origin")))
          (def man
            {:sha (if-not (empty? sha) sha)
             :repo (if-not (empty? url) url)
@@ -941,7 +1032,7 @@ int main(int argc, const char **argv) {
             :paths installed-files})
          (spit manifest (string/format "%j\n" man)))
 
-  (phony "install" ["uninstall" "build" "manifest"]
+  (phony "install" ["uninstall" "build" manifest]
          (when (dyn :test)
            (do-rule "test"))
          (print "Installed as '" (meta :name) "'."))
@@ -995,37 +1086,52 @@ usage: jpm [--key=value, --flag] ... [subcommand] [args] ...
 
 Run from a directory containing a project.janet file to perform operations
 on a project, or from anywhere to do operations on the global module cache (modpath).
+Commands that need write permission to the modpath are considered privileged commands - in
+some environments they may require super user privileges.
+Other project-level commands need to have a ./project.janet file in the current directory.
 
-Subcommands are:
+Unprivileged global subcommands:
-  build : build all artifacts
   help : show this help text
+  show-paths : prints the paths that will be used to install things.
+  quickbin entry executable : Create an executable from a janet script with a main function.
+
+Privileged global subcommands:
   install (repo or name)... : install artifacts. If a repo is given, install the contents of that
                    git repository, assuming that the repository is a jpm project. If not, build
                    and install the current project.
   uninstall (module)... : uninstall a module. If no module is given, uninstall the module
                        defined by the current directory.
-  show-paths : prints the paths that will be used to install things.
-  clean : remove any generated files or artifacts
-  test : run tests. Tests should be .janet files in the test/ directory relative to project.janet.
-  deps : install dependencies for the current project.
   clear-cache : clear the git cache. Useful for updating dependencies.
-  run rule : run a rule. Can also run custom rules added via (phony "task" [deps...] ...)
+  clear-manifest : clear the manifest. Useful for fixing broken installs.
-             or (rule "ouput.file" [deps...] ...).
-  rules : list rules available with run.
-  rule-tree (root rule) (depth) : Print a nice tree to see what rules depend on other rules.
-                                  Optinally provide a root rule to start printing from, and a
-                                  max depth to print. Without these options, all rules will print
-                                  their full dependency tree.
-  update-pkgs : Update the current package listing from the remote git repository selected.
-  quickbin entry executable : Create an executable from a janet script with a main function.
   make-lockfile (lockfile) : Create a lockfile based on repositories in the cache. The
             lockfile will record the exact versions of dependencies used to ensure a reproducible
             build. Lockfiles are best used with applications, not libraries. The default lockfile
             name is lockfile.jdn.
   load-lockfile (lockfile) : Install modules from a lockfile in a reproducible way. The
                              default lockfile name is lockfile.jdn.
-  repl : Run a repl in the context of the current project.janet file. This lets you run rules and
+  update-pkgs : Update the current package listing from the remote git repository selected.
-         otherwise debug the current project.janet file.
+
+Privileged project subcommands:
+  deps : install dependencies for the current project.
+  install : install artifacts of the current project.
+  uninstall : uninstall the current project's artifacts.
+
+Unprivileged project subcommands:
+  build : build all artifacts
+  clean : remove any generated files or artifacts
+  test : run tests. Tests should be .janet files in the test/ directory relative to project.janet.
+  run rule : run a rule. Can also run custom rules added via (phony "task" [deps...] ...)
+             or (rule "ouput.file" [deps...] ...).
+  rules : list rules available with run.
+  list-installed : list installed packages in the current syspath.
+  list-pkgs (search) : list packages in the package listing that the contain the string search.
+                       If no search pattern is given, prints the entire package listing.
+  rule-tree (root rule) (depth) : Print a nice tree to see what rules depend on other rules.
+                                  Optionally provide a root rule to start printing from, and a
+                                  max depth to print. Without these options, all rules will print
+                                  their full dependency tree.
+  debug-repl : Run a repl in the context of the current project.janet file. This lets you run rules and
+               otherwise debug the current project.janet file.
 
 Keys are:
   --modpath : The directory to install modules to. Defaults to $JANET_MODPATH, $JANET_PATH, or (dyn :syspath)
@@ -1096,10 +1202,33 @@ Flags are:
 
 (defn list-rules
   [&opt ctx]
-  (import-rules "./project.janet" true)
+  (import-rules "./project.janet")
   (def ks (sort (seq [k :keys (dyn :rules)] k)))
   (each k ks (print k)))
 
+(defn list-installed
+  []
+  (def xs
+    (seq [x :in (os/dir (find-manifest-dir))
+          :when (string/has-suffix? ".jdn" x)]
+      (string/slice x 0 -5)))
+  (sort xs)
+  (each x xs (print x)))
+
+(defn list-pkgs
+  [&opt search]
+  (def [ok _] (module/find "pkgs"))
+  (unless ok
+    (eprint "no local package listing found. Run `jpm update-pkgs` to get listing.")
+    (os/exit 1))
+  (def pkgs-mod (require "pkgs"))
+  (def ps
+    (seq [p :keys (get-in pkgs-mod ['packages :value] [])
+          :when (if search (string/find search p) true)]
+      p))
+  (sort ps)
+  (each p ps (print p)))
+
 (defn update-pkgs
   []
   (install-git (dyn :pkglist default-pkglist)))
@@ -1109,7 +1238,7 @@ Flags are:
   (create-executable @{} input output)
   (do-rule output))
 
-(defn jpm-repl
+(defn jpm-debug-repl
   []
   (def env
     (try
@@ -1138,10 +1267,13 @@ Flags are:
    "test" test
    "help" help
    "deps" deps
-   "repl" jpm-repl
+   "debug-repl" jpm-debug-repl
    "rule-tree" show-rule-tree
    "show-paths" show-paths
+   "list-installed" list-installed
+   "list-pkgs" list-pkgs
    "clear-cache" clear-cache
+   "clear-manifest" clear-manifest
    "run" local-rule
    "rules" list-rules
    "update-pkgs" update-pkgs

jpm.1

@@ -26,7 +26,7 @@ More interesting are the local commands. For more information on jpm usage, see
 
 .TP
 .BR \-\-nocolor
-Disable color in the jpm repl.
+Disable color in the jpm debug repl.
 
 .TP
 .BR \-\-verbose
@@ -139,6 +139,23 @@ date or too large, clear-cache will remove the cache and jpm will rebuild it
 when needed. clear-cache is a global command, so a project.janet is not
 required.
 
+.TP
+.BR list-installed
+List all installed packages in the current syspath.
+
+.TP
+.BR list-pkgs [\fBsearch\fR]
+List all package aliases in the current package listing that contain the given search string.
+If no search string is given, prints the entire listing.
+
+.TP
+.BR clear-manifest
+jpm creates a manifest directory that contains a list of all installed files.
+By deleting this directory, jpm will think that nothing is installed and will
+try reinstalling everything on the jpm deps or jpm load-lockfile commands. Be careful with
+this command, as it may leave extra files on your system and shouldn't be needed
+most of the time in a healthy install.
+
 .TP
 .BR run\ [\fBrule\fR]
 Run a given rule defined in project.janet. Project definitions files (project.janet) usually
@@ -172,7 +189,7 @@ as function arguments. The entry file can import other modules, including native
 jpm will attempt to include the dependencies into the generated executable.
 
 .TP
-.BR repl
+.BR debug-repl
 Load the current project.janet file and start a repl in it's environment. This lets a user better
 debug the project file, as well as run rules manually.
 
@@ -237,5 +254,10 @@ The git repository URL that contains a listing of packages. This allows installi
 is mostly a convenience. However, package dependencies can use short names, package listings
 can be used to choose a particular set of dependency versions for a whole project.
 
+.B JANET_GIT
+.RS
+An optional path to a git executable to use to clone git dependencies. By default, uses "git" on the current $PATH. You shouldn't need to set this
+if you have a normal install of git.
+
 .SH AUTHOR
 Written by Calvin Rose <calsrose@gmail.com>

meson.build

@@ -20,7 +20,7 @@
 
 project('janet', 'c',
   default_options : ['c_std=c99', 'b_lundef=false', 'default_library=both'],
-  version : '1.9.1')
+  version : '1.11.4')
 
 # Global settings
 janet_path = join_paths(get_option('prefix'), get_option('libdir'), 'janet')
@@ -63,7 +63,7 @@ conf.set('JANET_NO_NET', not get_option('net'))
 conf.set('JANET_REDUCED_OS', get_option('reduced_os'))
 conf.set('JANET_NO_TYPED_ARRAY', not get_option('typed_array'))
 conf.set('JANET_NO_INT_TYPES', not get_option('int_types'))
-conf.set('JANET_NO_PRF', not get_option('prf'))
+conf.set('JANET_PRF', get_option('prf'))
 conf.set('JANET_RECURSION_GUARD', get_option('recursion_guard'))
 conf.set('JANET_MAX_PROTO_DEPTH', get_option('max_proto_depth'))
 conf.set('JANET_MAX_MACRO_EXPAND', get_option('max_macro_expand'))
@@ -178,30 +178,28 @@ libjanet = library('janet', janetc,
 
 # Extra c flags - adding -fvisibility=hidden matches the Makefile and
 # shaves off about 10k on linux x64, likely similar on other platforms.
-native_cc = meson.get_compiler('c', native: true)
+if cc.has_argument('-fvisibility=hidden')
-cross_cc = meson.get_compiler('c', native: false)
+  extra_cflags = ['-fvisibility=hidden']
-if native_cc.has_argument('-fvisibility=hidden')
-  extra_native_cflags = ['-fvisibility=hidden']
 else
-  extra_native_cflags = []
+  extra_cflags = []
 endif
-if cross_cc.has_argument('-fvisibility=hidden')
-  extra_cross_cflags = ['-fvisibility=hidden']
-else
-  extra_cross_cflags = []
-endif
-
 janet_mainclient = executable('janet', janetc, mainclient_src,
   include_directories : incdir,
   dependencies : [m_dep, dl_dep, thread_dep],
-  c_args : extra_native_cflags,
+  c_args : extra_cflags,
   install : true)
 
 if meson.is_cross_build()
+  native_cc = meson.get_compiler('c', native: true)
+  if native_cc.has_argument('-fvisibility=hidden')
+    extra_native_cflags = ['-fvisibility=hidden']
+  else
+    extra_native_cflags = []
+  endif
   janet_nativeclient = executable('janet-native', janetc, mainclient_src,
     include_directories : incdir,
     dependencies : [m_dep, dl_dep, thread_dep],
-    c_args : extra_cross_cflags,
+    c_args : extra_native_cflags,
     native : true)
 else
   janet_nativeclient = janet_mainclient
@@ -216,16 +214,17 @@ docs = custom_target('docs',
 
 # Tests
 test_files = [
-  'test/suite0.janet',
+  'test/suite0000.janet',
-  'test/suite1.janet',
+  'test/suite0001.janet',
-  'test/suite2.janet',
+  'test/suite0002.janet',
-  'test/suite3.janet',
+  'test/suite0003.janet',
-  'test/suite4.janet',
+  'test/suite0004.janet',
-  'test/suite5.janet',
+  'test/suite0005.janet',
-  'test/suite6.janet',
+  'test/suite0006.janet',
-  'test/suite7.janet',
+  'test/suite0007.janet',
-  'test/suite8.janet',
+  'test/suite0008.janet',
-  'test/suite9.janet'
+  'test/suite0009.janet',
+  'test/suite0010.janet'
 ]
 foreach t : test_files
   test(t, janet_nativeclient, args : files([t]), workdir : meson.current_source_dir())
@@ -245,7 +244,18 @@ pkg.generate(libjanet,
 
 # Installation
 install_man('janet.1')
-install_man('jpm.1')
 install_headers(['src/include/janet.h', jconf], subdir: 'janet')
-install_data(sources : ['jpm'], install_dir : get_option('bindir'))
 install_data(sources : ['tools/.keep'], install_dir : join_paths(get_option('libdir'), 'janet'))
+if get_option('peg') and not get_option('reduced_os') and get_option('processes')
+  install_man('jpm.1')
+  patched_jpm = custom_target('patched-jpm',
+    input : ['tools/patch-jpm.janet', 'jpm'],
+    install : true,
+    install_dir : get_option('bindir'),
+    build_by_default : true,
+    output : ['jpm'],
+    command : [janet_nativeclient, '@INPUT@', '@OUTPUT@',
+      '--binpath=' + join_paths(get_option('prefix'), get_option('bindir')),
+      '--libpath=' + join_paths(get_option('prefix'), get_option('libdir')),
+      '--headerpath=' + join_paths(get_option('prefix'), get_option('includedir'))])
+endif

meson_options.txt

@@ -10,7 +10,7 @@ option('assembler', type : 'boolean', value : true)
 option('peg', type : 'boolean', value : true)
 option('typed_array', type : 'boolean', value : true)
 option('int_types', type : 'boolean', value : true)
-option('prf', type : 'boolean', value : true)
+option('prf', type : 'boolean', value : false)
 option('net', type : 'boolean', value : true)
 option('processes', type : 'boolean', value : true)
 option('umask', type : 'boolean', value : true)

src/boot/boot.janet

@@ -99,7 +99,7 @@
 (defn array? "Check if x is an array." [x] (= (type x) :array))
 (defn tuple? "Check if x is a tuple." [x] (= (type x) :tuple))
 (defn boolean? "Check if x is a boolean." [x] (= (type x) :boolean))
-(defn bytes? "Check if x is a string, symbol, or buffer." [x]
+(defn bytes? "Check if x is a string, symbol, keyword, or buffer." [x]
   (def t (type x))
   (if (= t :string) true (if (= t :symbol) true (if (= t :keyword) true (= t :buffer)))))
 (defn dictionary? "Check if x a table or struct." [x]
@@ -112,7 +112,7 @@
 (defn true? "Check if x is true." [x] (= x true))
 (defn false? "Check if x is false." [x] (= x false))
 (defn nil? "Check if x is nil." [x] (= x nil))
-(defn empty? "Check if xs is empty." [xs] (= 0 (length xs)))
+(defn empty? "Check if xs is empty." [xs] (= (length xs) 0))
 
 (def idempotent?
   "(idempotent? x)\n\nCheck if x is a value that evaluates to itself when compiled."
@@ -379,16 +379,27 @@
      ,(apply defer [(or dtor :close) binding] [truthy])
      ,falsey))
 
-(defn- for-template
+(defn- for-var-template
-  [binding start stop step comparison delta body]
+  [i start stop step comparison delta body]
-  (with-syms [i s]
+  (with-syms [s]
+    (def st (if (idempotent? step) step (gensym)))
+    (def loop-body
+      ~(while (,comparison ,i ,s)
+         ,;body
+         (set ,i (,delta ,i ,st))))
     ~(do
        (var ,i ,start)
        (def ,s ,stop)
-       (while (,comparison ,i ,s)
+       ,;(if (= st step) [] [~(def ,st ,step)])
-         (def ,binding ,i)
+       ,(if (and (number? st) (> st 0))
-         ,;body
+          loop-body
-         (set ,i (,delta ,i ,step))))))
+          ~(if (,> ,st 0) ,loop-body)))))
+
+(defn- for-template
+  [binding start stop step comparison delta body]
+  (def i (gensym))
+  (for-var-template i start stop step comparison delta
+                    [~(def ,binding ,i) ;body]))
 
 (defn- check-indexed [x]
   (if (indexed? x)
@@ -401,26 +412,18 @@
     (for-template binding start stop (or step 1) comparison op [rest])))
 
 (defn- each-template
-  [binding inx body]
+  [binding inx kind body]
   (with-syms [k]
     (def ds (if (idempotent? inx) inx (gensym)))
     ~(do
        ,(unless (= ds inx) ~(def ,ds ,inx))
        (var ,k (,next ,ds nil))
        (while (,not= nil ,k)
-         (def ,binding (,in ,ds ,k))
+         (def ,binding
-         ,;body
+           ,(case kind
-         (set ,k (,next ,ds ,k))))))
+              :each ~(,in ,ds ,k)
-
+              :keys k
-(defn- keys-template
+              :pairs ~(,tuple ,k (,in ,ds ,k))))
-  [binding in pair? body]
-  (with-syms [k]
-    (def ds (if (idempotent? in) in (gensym)))
-    ~(do
-       ,(unless (= ds in) ~(def ,ds ,in))
-       (var ,k (,next ,ds nil))
-       (while (,not= nil ,k)
-         (def ,binding ,(if pair? ~(tuple ,k (in ,ds ,k)) k))
          ,;body
          (set ,k (,next ,ds ,k))))))
 
@@ -433,6 +436,17 @@
          (def ,binding ,i)
          ,body))))
 
+(defn- loop-fiber-template
+  [binding expr body]
+  (with-syms [f s]
+    (def ds (if (idempotent? binding) binding (gensym)))
+    ~(let [,f ,expr]
+       (while true
+         (def ,ds (,resume ,f))
+         (if (= :dead (,fiber/status ,f)) (break))
+         ,;(if (= ds binding) [] [~(def ,binding ,ds)])
+         ,;body))))
+
 (defn- loop1
   [body head i]
 
@@ -466,18 +480,19 @@
       :range-to (range-template binding object rest + <=)
       :down (range-template binding object rest - >)
       :down-to (range-template binding object rest - >=)
-      :keys (keys-template binding object false [rest])
+      :keys (each-template binding object :keys [rest])
-      :pairs (keys-template binding object true [rest])
+      :pairs (each-template binding object :pairs [rest])
-      :in (each-template binding object [rest])
+      :in (each-template binding object :each [rest])
       :iterate (iterate-template binding object rest)
-      :generate (with-syms [f s]
+      :generate (loop-fiber-template binding object [rest])
-                  ~(let [,f ,object]
-                     (while true
-                       (def ,binding (,resume ,f))
-                       (if (= :dead (,fiber/status ,f)) (break))
-                       ,rest)))
       (error (string "unexpected loop verb " verb)))))
 
+(defmacro forv
+  "Do a c style for loop for side effects. The iteration variable i
+  can be mutated in the loop, unlike normal for. Returns nil."
+  [i start stop & body]
+  (for-var-template i start stop 1 < + body))
+
 (defmacro for
   "Do a c style for loop for side effects. Returns nil."
   [i start stop & body]
@@ -486,17 +501,34 @@
 (defmacro eachk
   "Loop over each key in ds. Returns nil."
   [x ds & body]
-  (keys-template x ds false body))
+  (each-template x ds :keys body))
 
 (defmacro eachp
   "Loop over each (key, value) pair in ds. Returns nil."
   [x ds & body]
-  (keys-template x ds true body))
+  (each-template x ds :pairs body))
+
+(defmacro eachy
+  "Resume a fiber in a loop until it has errored or died. Evaluate the body
+  of the loop with binding set to the yielded value."
+  [x fiber & body]
+  (loop-fiber-template x fiber body))
+
+(defmacro repeat
+  "Evaluate body n times. If n is negative, body will be evaluated 0 times. Evaluates to nil."
+  [n & body]
+  (with-syms [iter]
+    ~(do (var ,iter ,n) (while (> ,iter 0) ,;body (-- ,iter)))))
+
+(defmacro forever
+  "Evaluate body forever in a loop, or until a break statement."
+  [& body]
+  ~(while true ,;body))
 
 (defmacro each
   "Loop over each value in ds. Returns nil."
   [x ds & body]
-  (each-template x ds body))
+  (each-template x ds :each body))
 
 (defmacro loop
   "A general purpose loop macro. This macro is similar to the Common Lisp
@@ -538,10 +570,11 @@
 (put _env 'loop1 nil)
 (put _env 'check-indexed nil)
 (put _env 'for-template nil)
+(put _env 'for-var-template nil)
 (put _env 'iterate-template nil)
 (put _env 'each-template nil)
-(put _env 'keys-template nil)
 (put _env 'range-template nil)
+(put _env 'loop-fiber-template nil)
 
 (defmacro seq
   "Similar to loop, but accumulates the loop body into an array and returns that.
@@ -667,6 +700,57 @@
   [xs]
   (get xs (- (length xs) 1)))
 
+## Polymorphic comparisons
+
+(defn compare
+  "Polymorphic compare. Returns -1, 0, 1 for x < y, x = y, x > y respectively.
+   Differs from the primitive comparators in that it first checks to
+   see whether either x or y implement a 'compare' method which can
+   compare x and y. If so it uses that compare method. If not, it
+   delegates to the primitive comparators."
+  [x y]
+  (or
+    (when-let [f (get x :compare)] (f x y))
+    (when-let [f (get y :compare)] (- (f y x)))
+    (cmp x y)))
+
+(defn- compare-reduce [op xs]
+  (var r true)
+  (loop [i :range [0 (- (length xs) 1)]
+         :let [c (compare (xs i) (xs (+ i 1)))
+               ok (op c 0)]
+         :when (not ok)]
+    (set r false)
+    (break))
+  r)
+
+(defn compare=
+  "Equivalent of '=' but using compare function instead of primitive comparator"
+  [& xs]
+  (compare-reduce = xs))
+
+(defn compare<
+  "Equivalent of '<' but using compare function instead of primitive comparator"
+  [& xs]
+  (compare-reduce < xs))
+
+(defn compare<=
+  "Equivalent of '<=' but using compare function instead of primitive comparator"
+  [& xs]
+  (compare-reduce <= xs))
+
+(defn compare>
+  "Equivalent of '>' but using compare function instead of primitive comparator"
+  [& xs]
+  (compare-reduce > xs))
+
+(defn compare>=
+  "Equivalent of '>=' but using compare function instead of primitive comparator"
+  [& xs]
+  (compare-reduce >= xs))
+
+(put _env 'compare-reduce nil)
+
 ###
 ###
 ### Indexed Combinators
@@ -677,7 +761,7 @@
   [a lo hi by]
   (def pivot (in a hi))
   (var i lo)
-  (for j lo hi
+  (forv j lo hi
     (def aj (in a j))
     (when (by aj pivot)
       (def ai (in a i))
@@ -775,19 +859,19 @@
   (def ninds (length inds))
   (if (= 0 ninds) (error "expected at least 1 indexed collection"))
   (var limit (length (in inds 0)))
-  (for i 0 ninds
+  (forv i 0 ninds
     (def l (length (in inds i)))
     (if (< l limit) (set limit l)))
   (def [i1 i2 i3 i4] inds)
   (def res (array/new limit))
   (case ninds
-    1 (for i 0 limit (set (res i) (f (in i1 i))))
+    1 (forv i 0 limit (set (res i) (f (in i1 i))))
-    2 (for i 0 limit (set (res i) (f (in i1 i) (in i2 i))))
+    2 (forv i 0 limit (set (res i) (f (in i1 i) (in i2 i))))
-    3 (for i 0 limit (set (res i) (f (in i1 i) (in i2 i) (in i3 i))))
+    3 (forv i 0 limit (set (res i) (f (in i1 i) (in i2 i) (in i3 i))))
-    4 (for i 0 limit (set (res i) (f (in i1 i) (in i2 i) (in i3 i) (in i4 i))))
+    4 (forv i 0 limit (set (res i) (f (in i1 i) (in i2 i) (in i3 i) (in i4 i))))
-    (for i 0 limit
+    (forv i 0 limit
       (def args (array/new ninds))
-      (for j 0 ninds (set (args j) (in (in inds j) i)))
+      (forv j 0 ninds (set (args j) (in (in inds j) i)))
       (set (res i) (f ;args))))
   res)
 
@@ -839,17 +923,18 @@
     1 (do
         (def [n] args)
         (def arr (array/new n))
-        (for i 0 n (put arr i i))
+        (forv i 0 n (put arr i i))
         arr)
     2 (do
         (def [n m] args)
         (def arr (array/new (- m n)))
-        (for i n m (put arr (- i n) i))
+        (forv i n m (put arr (- i n) i))
         arr)
     3 (do
         (def [n m s] args)
-        (if (neg? s)
+        (cond
-          (seq [i :down [n m (- s)]] i)
+          (zero? s) @[]
+          (neg? s) (seq [i :down [n m (- s)]] i)
           (seq [i :range [n m s]] i)))
     (error "expected 1 to 3 arguments to range")))
 
@@ -872,6 +957,18 @@
   (def i (find-index pred ind))
   (if (= i nil) nil (in ind i)))
 
+(defn index-of
+  "Find the first key associated with a value x in a data structure, acting like a reverse lookup.
+  Will not look at table prototypes.
+  Returns dflt if not found."
+  [x ind &opt dflt]
+  (var k (next ind nil))
+  (var ret dflt)
+  (while (not= nil k)
+    (when (= (in ind k) x) (set ret k) (break))
+    (set k (next ind k)))
+  ret)
+
 (defn take
   "Take first n elements in an indexed type. Returns new indexed instance."
   [n ind]
@@ -945,6 +1042,21 @@
     (array/push parts (tuple apply f $args)))
   (tuple 'fn (tuple '& $args) (tuple/slice parts 0)))
 
+(defmacro tracev
+  "Print a value and a description of the form that produced that value to
+  stderr. Evaluates to x."
+  [x]
+  (def [l c] (tuple/sourcemap (dyn :macro-form ())))
+  (def cf (dyn :current-file))
+  (def fmt-1 (if cf (string/format "trace [%s]" cf) "trace"))
+  (def fmt-2 (if (or (neg? l) (neg? c)) ":" (string/format " on line %d, column %d:" l c)))
+  (def fmt (string fmt-1 fmt-2 " %j is "))
+  (def s (gensym))
+  ~(let [,s ,x]
+     (,eprinf ,fmt ',x)
+     (,eprintf (,dyn :pretty-format "%q") ,s)
+     ,s))
+
 (defmacro ->
   "Threading macro. Inserts x as the second value in the first form
   in forms, and inserts the modified first form into the second form
@@ -1111,19 +1223,43 @@
     (if x nil (set res x)))
   res)
 
+(defn any?
+  "Returns the first truthy value in ind, otherwise nil.
+  falsey value."
+  [ind]
+  (var res nil)
+  (loop [x :in ind :until res]
+    (if x (set res x)))
+  res)
+
+(defn reverse!
+  "Reverses the order of the elements in a given array or buffer and returns it
+  mutated."
+  [t]
+  (def len-1 (- (length t) 1))
+  (def half (/ len-1 2))
+  (forv i 0 half
+    (def j (- len-1 i))
+    (def l (in t i))
+    (def r (in t j))
+    (put t i r)
+    (put t j l))
+  t)
+
 (defn reverse
-  "Reverses the order of the elements in a given array or tuple and returns a new array."
+  "Reverses the order of the elements in a given array or tuple and returns
+  a new array. If string or buffer is provided function returns array of chars reversed."
   [t]
   (def len (length t))
   (var n (- len 1))
-  (def reversed (array/new len))
+  (def ret (array/new len))
   (while (>= n 0)
-    (array/push reversed (in t n))
+    (array/push ret (in t n))
     (-- n))
-  reversed)
+  ret)
 
 (defn invert
-  "Returns a table of where the keys of an associative data structure
+  "Returns a table where the keys of an associative data structure
   are the values, and the values of the keys. If multiple keys have the same
   value, one key will be ignored."
   [ds]
@@ -1137,11 +1273,14 @@
   Returns a new table."
   [ks vs]
   (def res @{})
-  (def lk (length ks))
+  (var kk nil)
-  (def lv (length vs))
+  (var vk nil)
-  (def len (if (< lk lv) lk lv))
+  (while true
-  (for i 0 len
+    (set kk (next ks kk))
-    (put res (in ks i) (in vs i)))
+    (if (= nil kk) (break))
+    (set vk (next vs vk))
+    (if (= nil vk) (break))
+    (put res (in ks kk) (in vs vk)))
   res)
 
 (defn get-in
@@ -1161,7 +1300,7 @@
   (var d ds)
   (def len-1 (- (length ks) 1))
   (if (< len-1 0) (error "expected at least 1 key in ks"))
-  (for i 0 len-1
+  (forv i 0 len-1
     (def k (get ks i))
     (def v (get d k))
     (if (= nil v)
@@ -1183,7 +1322,7 @@
   (var d ds)
   (def len-1 (- (length ks) 1))
   (if (< len-1 0) (error "expected at least 1 key in ks"))
-  (for i 0 len-1
+  (forv i 0 len-1
     (def k (get ks i))
     (def v (get d k))
     (if (= nil v)
@@ -1665,14 +1804,16 @@
   (defn expandqq [t]
     (defn qq [x]
       (case (type x)
-        :tuple (do
+        :tuple (if (= :brackets (tuple/type x))
-                 (def x0 (in x 0))
+                 ~[,;(map qq x)]
-                 (if (or (= 'unquote x0) (= 'unquote-splicing x0))
+                 (do
-                   (tuple x0 (recur (in x 1)))
+                   (def x0 (get x 0))
-                   (tuple/slice (map qq x))))
+                   (if (= 'unquote x0)
+                     (tuple x0 (recur (get x 1)))
+                     (tuple/slice (map qq x)))))
         :array (map qq x)
-        :table (table (map qq (kvs x)))
+        :table (table ;(map qq (kvs x)))
-        :struct (struct (map qq (kvs x)))
+        :struct (struct ;(map qq (kvs x)))
         x))
     (tuple (in t 0) (qq (in t 1))))
 
@@ -1696,7 +1837,7 @@
     (def m? (entry :macro))
     (cond
       s (s t)
-      m? (m ;(tuple/slice t 1))
+      m? (do (setdyn :macro-form t) (m ;(tuple/slice t 1)))
       (tuple/slice (map recur t))))
 
   (def ret
@@ -1856,20 +1997,24 @@
   that should make it easier to write more complex patterns."
   ~@{:d (range "09")
      :a (range "az" "AZ")
-     :s (set " \t\r\n\0\f")
+     :s (set " \t\r\n\0\f\v")
      :w (range "az" "AZ" "09")
+     :h (range "09" "af")
      :S (if-not :s 1)
      :W (if-not :w 1)
      :A (if-not :a 1)
      :D (if-not :d 1)
+     :H (if-not :h 1)
      :d+ (some :d)
      :a+ (some :a)
      :s+ (some :s)
      :w+ (some :w)
+     :h+ (some :h)
      :d* (any :d)
      :a* (any :a)
      :w* (any :w)
-     :s* (any :s)})
+     :s* (any :s)
+     :h* (any :h)})
 
 ###
 ###
@@ -1924,6 +2069,14 @@
       (if ec "\e[0m" "")))
   (eflush))
 
+(defn curenv
+  "Get the current environment table. Same as (fiber/getenv (fiber/current)). If n
+  is provided, gets the nth prototype of the environment table."
+  [&opt n]
+  (var e (fiber/getenv (fiber/current)))
+  (if n (repeat n (if (= nil e) (break)) (set e (table/getproto e))))
+  e)
+
 (defn run-context
   "Run a context. This evaluates expressions in an environment,
   and is encapsulates the parsing, compilation, and evaluation.
@@ -2001,19 +2154,26 @@
   (while going
     (if (env :exit) (break))
     (buffer/clear buf)
-    (chunks buf p)
+    (if (= (chunks buf p)
-    (var pindex 0)
+           :cancel)
-    (var pstatus nil)
+      (do
-    (def len (length buf))
+        # A :cancel chunk represents a cancelled form in the REPL, so reset.
-    (when (= len 0)
+        (parser/flush p)
-      (parser/eof p)
+        (buffer/clear buf))
-      (set going false))
+      (do
-    (while (> len pindex)
+        (var pindex 0)
-      (+= pindex (parser/consume p buf pindex))
+        (var pstatus nil)
-      (while (parser/has-more p)
+        (def len (length buf))
-        (eval1 (parser/produce p)))
+        (when (= len 0)
-      (when (= (parser/status p) :error)
+          (parser/eof p)
-        (parse-err p where))))
+          (set going false))
+        (while (> len pindex)
+          (+= pindex (parser/consume p buf pindex))
+          (while (parser/has-more p)
+            (eval1 (parser/produce p)))
+          (when (= (parser/status p) :error)
+            (parse-err p where))))))
+
   # Check final parser state
   (while (parser/has-more p)
     (eval1 (parser/produce p)))
@@ -2044,12 +2204,12 @@
       (buffer/push-string buf "\n")))
   (var returnval nil)
   (run-context {:chunks chunks
-                :on-compile-error (fn [msg errf &]
+                :on-compile-error (fn compile-error [msg errf &]
                                     (error (string "compile error: " msg)))
-                :on-parse-error (fn [p x]
+                :on-parse-error (fn parse-error [p x]
                                   (error (string "parse error: " (parser/error p))))
                 :fiber-flags :i
-                :on-status (fn [f val]
+                :on-status (fn on-status [f val]
                              (if-not (= (fiber/status f) :dead)
                                (error val))
                              (set returnval val))
@@ -2263,14 +2423,11 @@
               newenv)
     :image (fn [path &] (load-image (slurp path)))})
 
-(defn require
+(defn require-1
-  "Require a module with the given name. Will search all of the paths in
+  [path args kargs]
-  module/paths. Returns the new environment
-  returned from compiling and running the file."
-  [path & args]
   (def [fullpath mod-kind] (module/find path))
   (unless fullpath (error mod-kind))
-  (if-let [check (in module/cache fullpath)]
+  (if-let [check (if-not (kargs :fresh) (in module/cache fullpath))]
     check
     (if (module/loading fullpath)
       (error (string "circular dependency " fullpath " detected"))
@@ -2281,15 +2438,23 @@
         (put module/cache fullpath env)
         env))))
 
+(defn require
+  "Require a module with the given name. Will search all of the paths in
+  module/paths. Returns the new environment
+  returned from compiling and running the file."
+  [path & args]
+  (require-1 path args (struct ;args)))
+
 (defn import*
   "Function form of import. Same parameters, but the path
   and other symbol parameters should be strings instead."
   [path & args]
   (def env (fiber/getenv (fiber/current)))
+  (def kargs (table ;args))
   (def {:as as
         :prefix prefix
-        :export ep} (table ;args))
+        :export ep} kargs)
-  (def newenv (require path ;args))
+  (def newenv (require-1 path args kargs))
   (def prefix (or
                 (and as (string as "/"))
                 prefix
@@ -2298,6 +2463,8 @@
     (def newv (table/setproto @{:private (not ep)} v))
     (put env (symbol prefix k) newv)))
 
+(put _env 'require-1 nil)
+
 (defmacro import
   "Import a module. First requires the module, and then merges its
   symbols into the current environment, prepending a given prefix as needed.
@@ -2305,9 +2472,11 @@
   use the name of the module as a prefix. One can also use :export true
   to re-export the imported symbols. If :exit true is given as an argument,
   any errors encountered at the top level in the module will cause (os/exit 1)
-  to be called. Dynamic bindings will NOT be imported."
+  to be called. Dynamic bindings will NOT be imported. Use :fresh to bypass the
+  module cache."
   [path & args]
-  (def argm (map |(if (keyword? $) $ (string $)) args))
+  (def ps (partition 2 args))
+  (def argm (mapcat (fn [[k v]] [k (if (= k :as) (string v) v)]) ps))
   (tuple import* (string path) ;argm))
 
 (defmacro use
@@ -2373,7 +2542,7 @@
 (defn .bytecode
   "Get the bytecode for the current function."
   [&opt n]
-  ((.disasm n) 'bytecode))
+  ((.disasm n) :bytecode))
 
 (defn .ppasm
   "Pretty prints the assembly for the current function"
@@ -2381,13 +2550,13 @@
   (def frame (.frame n))
   (def func (frame :function))
   (def dasm (disasm-alias func))
-  (def bytecode (in dasm 'bytecode))
+  (def bytecode (in dasm :bytecode))
   (def pc (frame :pc))
-  (def sourcemap (in dasm 'sourcemap))
+  (def sourcemap (in dasm :sourcemap))
   (var last-loc [-2 -2])
   (print "\n  signal: " (.signal))
-  (print "  function:   " (dasm 'name) " [" (in dasm 'source "") "]")
+  (print "  function:   " (dasm :name) " [" (in dasm :source "") "]")
-  (when-let [constants (dasm 'constants)]
+  (when-let [constants (dasm :constants)]
     (printf "  constants:  %.4q" constants))
   (printf "  slots:      %.4q\n" (frame :slots))
   (def padding (string/repeat " " 20))
@@ -2410,7 +2579,7 @@
   [&opt n]
   (def fun (.fn n))
   (def bytecode (.bytecode n))
-  (for i 0 (length bytecode)
+  (forv i 0 (length bytecode)
     (debug/fbreak fun i))
   (print "Set " (length bytecode) " breakpoints in " fun))
 
@@ -2419,7 +2588,7 @@
   [&opt n]
   (def fun (.fn n))
   (def bytecode (.bytecode n))
-  (for i 0 (length bytecode)
+  (forv i 0 (length bytecode)
     (debug/unfbreak fun i))
   (print "Cleared " (length bytecode) " breakpoints in " fun))
 
@@ -2461,7 +2630,7 @@
   "Go to the next breakpoint."
   [&opt n]
   (var res nil)
-  (for i 0 (or n 1)
+  (forv i 0 (or n 1)
     (set res (resume (.fiber))))
   res)
 
@@ -2475,7 +2644,7 @@
   "Execute the next n instructions."
   [&opt n]
   (var res nil)
-  (for i 0 (or n 1)
+  (forv i 0 (or n 1)
     (set res (debug/step (.fiber))))
   res)
 
@@ -2576,7 +2745,9 @@
                   'def is-safe-def 'var is-safe-def 'def- is-safe-def 'var- is-safe-def
                   'defglobal is-safe-def 'varglobal is-safe-def})
 
-(def- importers {'import true 'import* true 'use true 'dofile true 'require true})
+(def- importers {'import true 'import* true 'dofile true 'require true})
+(defn- use-2 [evaluator args]
+  (each a args (import* (string a) :prefix "" :evaluator evaluator)))
 
 # conditional compilation for reduced os
 (def- getenv-alias (if-let [entry (in _env 'os/getenv)] (entry :value) (fn [&])))
@@ -2619,7 +2790,7 @@
   -m syspath : Set system path for loading global modules
   -c source output : Compile janet source code into an image
   -n : Disable ANSI color output in the repl
-  -l path : Execute code in a file before running the main script
+  -l lib : Import a module before processing more arguments
   -- : Stop handling options`)
            (os/exit 0)
            1)
@@ -2631,17 +2802,17 @@
      "k" (fn [&] (set *compile-only* true) (set *exit-on-error* false) 1)
      "n" (fn [&] (set *colorize* false) 1)
      "m" (fn [i &] (setdyn :syspath (in args (+ i 1))) 2)
-     "c" (fn [i &]
+     "c" (fn c-switch [i &]
            (def e (dofile (in args (+ i 1))))
            (spit (in args (+ i 2)) (make-image e))
            (set *no-file* false)
            3)
      "-" (fn [&] (set *handleopts* false) 1)
-     "l" (fn [i &]
+     "l" (fn l-switch [i &]
            (import* (in args (+ i 1))
                     :prefix "" :exit *exit-on-error*)
            2)
-     "e" (fn [i &]
+     "e" (fn e-switch [i &]
            (set *no-file* false)
            (eval-string (in args (+ i 1)))
            2)
@@ -2664,6 +2835,9 @@
           # Always safe form
           safe-check
           (thunk)
+          # Use
+          (= 'use head)
+          (use-2 evaluator (tuple/slice source 1))
           # Import-like form
           (importers head)
           (do
@@ -2685,18 +2859,19 @@
         (def subargs (array/slice args i))
         (put env :args subargs)
         (dofile arg :prefix "" :exit *exit-on-error* :evaluator evaluator :env env)
-        (if-let [main (get (in env 'main) :value)]
+        (unless *compile-only*
-          (let [thunk (compile [main ;(tuple/slice args i)] env arg)]
+          (if-let [main (get (in env 'main) :value)]
-            (if (function? thunk) (thunk) (error (thunk :error)))))
+            (let [thunk (compile [main ;(tuple/slice args i)] env arg)]
+              (if (function? thunk) (thunk) (error (thunk :error))))))
         (set i lenargs))))
 
   (when (and (not *compile-only*) (or *should-repl* *no-file*))
     (if-not *quiet*
-      (print "Janet " janet/version "-" janet/build "  Copyright (C) 2017-2020 Calvin Rose"))
+      (print "Janet " janet/version "-" janet/build " " (os/which) "/" (os/arch)))
     (flush)
     (defn getprompt [p]
       (def [line] (parser/where p))
-      (string "janet:" line ":" (parser/state p :delimiters) "> "))
+      (string "repl:" line ":" (parser/state p :delimiters) "> "))
     (defn getstdin [prompt buf _]
       (file/write stdout prompt)
       (file/flush stdout)
@@ -2714,6 +2889,7 @@
 (put _env 'is-safe-def nil)
 (put _env 'safe-forms nil)
 (put _env 'importers nil)
+(put _env 'use-2 nil)
 (put _env 'getenv-alias nil)
 
 ###
@@ -2843,6 +3019,14 @@
   (each h local-headers
     (do-one-file h))
 
+  # windows.h should not be included in any of the external or internal headers - only in .c files.
+  (print)
+  (print "/* Windows work around - winsock2 must be included before windows.h, especially in amalgamated build */")
+  (print "#if defined(JANET_WINDOWS) && defined(JANET_NET)")
+  (print "#include <winsock2.h>")
+  (print "#endif")
+  (print)
+
   (each s core-sources
     (do-one-file s))
 

src/boot/table_test.c

@@ -61,5 +61,11 @@ int table_test() {
     assert(janet_equals(janet_table_get(t2, janet_csymbolv("t2key1")), janet_wrap_integer(10)));
     assert(janet_equals(janet_table_get(t2, janet_csymbolv("t2key2")), janet_wrap_integer(100)));
 
+    assert(t2->count == 4);
+    assert(janet_equals(janet_table_remove(t2, janet_csymbolv("t2key1")), janet_wrap_integer(10)));
+    assert(t2->count == 3);
+    assert(janet_equals(janet_table_remove(t2, janet_csymbolv("t2key2")), janet_wrap_integer(100)));
+    assert(t2->count == 2);
+
     return 0;
 }

src/conf/janetconf.h

@@ -27,10 +27,10 @@
 #define JANETCONF_H
 
 #define JANET_VERSION_MAJOR 1
-#define JANET_VERSION_MINOR 9
+#define JANET_VERSION_MINOR 11
-#define JANET_VERSION_PATCH 1
+#define JANET_VERSION_PATCH 4
-#define JANET_VERSION_EXTRA ""
+#define JANET_VERSION_EXTRA "-dev"
-#define JANET_VERSION "1.9.1"
+#define JANET_VERSION "1.11.4-dev"
 
 /* #define JANET_BUILD "local" */
 
@@ -41,7 +41,8 @@
 /* #define JANET_API __attribute__((visibility ("default"))) */
 
 /* These settings should be specified before amalgamation is
- * built. */
+ * built. Any build with these set should be considered non-standard, and
+ * certain Janet libraries should be expected not to work. */
 /* #define JANET_NO_DOCSTRINGS */
 /* #define JANET_NO_SOURCEMAPS */
 /* #define JANET_REDUCED_OS */
@@ -51,13 +52,14 @@
 /* #define JANET_NO_NET */
 /* #define JANET_NO_TYPED_ARRAY */
 /* #define JANET_NO_INT_TYPES */
-
-/* Other settings */
-/* #define JANET_NO_PRF */
-/* #define JANET_NO_UTC_MKTIME */
 /* #define JANET_NO_REALPATH */
 /* #define JANET_NO_SYMLINKS */
 /* #define JANET_NO_UMASK */
+
+/* Other settings */
+/* #define JANET_DEBUG */
+/* #define JANET_PRF */
+/* #define JANET_NO_UTC_MKTIME */
 /* #define JANET_OUT_OF_MEMORY do { printf("janet out of memory\n"); exit(1); } while (0) */
 /* #define JANET_EXIT(msg) do { printf("C assert failed executing janet: %s\n", msg); exit(1); } while (0) */
 /* #define JANET_TOP_LEVEL_SIGNAL(msg) call_my_function((msg), stderr) */
@@ -68,4 +70,7 @@
 /* #define JANET_OS_NAME my-custom-os */
 /* #define JANET_ARCH_NAME pdp-8 */
 
+/* Main client settings, does not affect library code */
+/* #define JANET_SIMPLE_GETLINE */
+
 #endif /* end of include guard: JANETCONF_H */

src/core/array.c

@@ -270,6 +270,26 @@ static Janet cfun_array_remove(int32_t argc, Janet *argv) {
     return argv[0];
 }
 
+static Janet cfun_array_trim(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    JanetArray *array = janet_getarray(argv, 0);
+    if (array->count) {
+        if (array->count < array->capacity) {
+            Janet *newData = realloc(array->data, array->count * sizeof(Janet));
+            if (NULL == newData) {
+                JANET_OUT_OF_MEMORY;
+            }
+            array->data = newData;
+            array->capacity = array->count;
+        }
+    } else {
+        array->capacity = 0;
+        free(array->data);
+        array->data = NULL;
+    }
+    return argv[0];
+}
+
 static const JanetReg array_cfuns[] = {
     {
         "array/new", cfun_array_new,
@@ -345,6 +365,11 @@ static const JanetReg array_cfuns[] = {
              "By default, n is 1. "
              "Returns the array.")
     },
+    {
+        "array/trim", cfun_array_trim,
+        JDOC("(array/trim arr)\n\n"
+             "Set the backing capacity of an array to its current length. Returns the modified array.")
+    },
     {NULL, NULL, NULL}
 };
 

src/core/asm.c

@@ -53,7 +53,6 @@ struct JanetAssembler {
 
     Janet name;
     JanetTable labels; /* keyword -> bytecode index */
-    JanetTable constants; /* symbol -> constant index */
     JanetTable slots; /* symbol -> slot index */
     JanetTable envs; /* symbol -> environment index */
     JanetTable defs; /* symbol -> funcdefs index */
@@ -74,6 +73,7 @@ static const JanetInstructionDef janet_ops[] = {
     {"call", JOP_CALL},
     {"clo", JOP_CLOSURE},
     {"cmp", JOP_COMPARE},
+    {"cncl", JOP_CANCEL},
     {"div", JOP_DIVIDE},
     {"divim", JOP_DIVIDE_IMMEDIATE},
     {"eq", JOP_EQUALS},
@@ -113,6 +113,8 @@ static const JanetInstructionDef janet_ops[] = {
     {"movn", JOP_MOVE_NEAR},
     {"mul", JOP_MULTIPLY},
     {"mulim", JOP_MULTIPLY_IMMEDIATE},
+    {"neq", JOP_NOT_EQUALS},
+    {"neqim", JOP_NOT_EQUALS_IMMEDIATE},
     {"next", JOP_NEXT},
     {"noop", JOP_NOOP},
     {"prop", JOP_PROPAGATE},
@@ -172,7 +174,6 @@ static void janet_asm_deinit(JanetAssembler *a) {
     janet_table_deinit(&a->slots);
     janet_table_deinit(&a->labels);
     janet_table_deinit(&a->envs);
-    janet_table_deinit(&a->constants);
     janet_table_deinit(&a->defs);
 }
 
@@ -252,9 +253,6 @@ static int32_t doarg_1(
         case JANET_OAT_ENVIRONMENT:
             c = &a->envs;
             break;
-        case JANET_OAT_CONSTANT:
-            c = &a->constants;
-            break;
         case JANET_OAT_LABEL:
             c = &a->labels;
             break;
@@ -506,7 +504,6 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
     a.defs_capacity = 0;
     a.name = janet_wrap_nil();
     janet_table_init(&a.labels, 0);
-    janet_table_init(&a.constants, 0);
     janet_table_init(&a.slots, 0);
     janet_table_init(&a.envs, 0);
     janet_table_init(&a.defs, 0);
@@ -534,34 +531,34 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
                      "expected struct or table for assembly source");
 
     /* Check for function name */
-    a.name = janet_get1(s, janet_csymbolv("name"));
+    a.name = janet_get1(s, janet_ckeywordv("name"));
     if (!janet_checktype(a.name, JANET_NIL)) {
         def->name = janet_to_string(a.name);
     }
 
     /* Set function arity */
-    x = janet_get1(s, janet_csymbolv("arity"));
+    x = janet_get1(s, janet_ckeywordv("arity"));
     def->arity = janet_checkint(x) ? janet_unwrap_integer(x) : 0;
     janet_asm_assert(&a, def->arity >= 0, "arity must be non-negative");
 
-    x = janet_get1(s, janet_csymbolv("max-arity"));
+    x = janet_get1(s, janet_ckeywordv("max-arity"));
     def->max_arity = janet_checkint(x) ? janet_unwrap_integer(x) : def->arity;
     janet_asm_assert(&a, def->max_arity >= def->arity, "max-arity must be greater than or equal to arity");
 
-    x = janet_get1(s, janet_csymbolv("min-arity"));
+    x = janet_get1(s, janet_ckeywordv("min-arity"));
     def->min_arity = janet_checkint(x) ? janet_unwrap_integer(x) : def->arity;
     janet_asm_assert(&a, def->min_arity <= def->arity, "min-arity must be less than or equal to arity");
 
     /* Check vararg */
-    x = janet_get1(s, janet_csymbolv("vararg"));
+    x = janet_get1(s, janet_ckeywordv("vararg"));
     if (janet_truthy(x)) def->flags |= JANET_FUNCDEF_FLAG_VARARG;
 
     /* Check source */
-    x = janet_get1(s, janet_csymbolv("source"));
+    x = janet_get1(s, janet_ckeywordv("source"));
     if (janet_checktype(x, JANET_STRING)) def->source = janet_unwrap_string(x);
 
     /* Create slot aliases */
-    x = janet_get1(s, janet_csymbolv("slots"));
+    x = janet_get1(s, janet_ckeywordv("slots"));
     if (janet_indexed_view(x, &arr, &count)) {
         for (i = 0; i < count; i++) {
             Janet v = arr[i];
@@ -582,7 +579,7 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
     }
 
     /* Parse constants */
-    x = janet_get1(s, janet_csymbolv("constants"));
+    x = janet_get1(s, janet_ckeywordv("constants"));
     if (janet_indexed_view(x, &arr, &count)) {
         def->constants_length = count;
         def->constants = malloc(sizeof(Janet) * (size_t) count);
@@ -591,25 +588,7 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
         }
         for (i = 0; i < count; i++) {
             Janet ct = arr[i];
-            if (janet_checktype(ct, JANET_TUPLE) &&
+            def->constants[i] = ct;
-                    janet_tuple_length(janet_unwrap_tuple(ct)) > 1 &&
-                    janet_checktype(janet_unwrap_tuple(ct)[0], JANET_SYMBOL)) {
-                const Janet *t = janet_unwrap_tuple(ct);
-                int32_t tcount = janet_tuple_length(t);
-                const uint8_t *macro = janet_unwrap_symbol(t[0]);
-                if (0 == janet_cstrcmp(macro, "quote")) {
-                    def->constants[i] = t[1];
-                } else if (tcount == 3 &&
-                           janet_checktype(t[1], JANET_SYMBOL) &&
-                           0 == janet_cstrcmp(macro, "def")) {
-                    def->constants[i] = t[2];
-                    janet_table_put(&a.constants, t[1], janet_wrap_integer(i));
-                } else {
-                    janet_asm_errorv(&a, janet_formatc("could not parse constant \"%v\"", ct));
-                }
-            } else {
-                def->constants[i] = ct;
-            }
         }
     } else {
         def->constants = NULL;
@@ -617,7 +596,7 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
     }
 
     /* Parse sub funcdefs */
-    x = janet_get1(s, janet_csymbolv("closures"));
+    x = janet_get1(s, janet_ckeywordv("closures"));
     if (janet_indexed_view(x, &arr, &count)) {
         int32_t i;
         for (i = 0; i < count; i++) {
@@ -628,7 +607,7 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
             if (subres.status != JANET_ASSEMBLE_OK) {
                 janet_asm_errorv(&a, subres.error);
             }
-            subname = janet_get1(arr[i], janet_csymbolv("name"));
+            subname = janet_get1(arr[i], janet_ckeywordv("name"));
             if (!janet_checktype(subname, JANET_NIL)) {
                 janet_table_put(&a.defs, subname, janet_wrap_integer(def->defs_length));
             }
@@ -647,7 +626,7 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
     }
 
     /* Parse bytecode and labels */
-    x = janet_get1(s, janet_csymbolv("bytecode"));
+    x = janet_get1(s, janet_ckeywordv("bytecode"));
     if (janet_indexed_view(x, &arr, &count)) {
         /* Do labels and find length */
         int32_t blength = 0;
@@ -703,7 +682,7 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
     a.errindex = -1;
 
     /* Check for source mapping */
-    x = janet_get1(s, janet_csymbolv("sourcemap"));
+    x = janet_get1(s, janet_ckeywordv("sourcemap"));
     if (janet_indexed_view(x, &arr, &count)) {
         janet_asm_assert(&a, count == def->bytecode_length, "sourcemap must have the same length as the bytecode");
         def->sourcemap = malloc(sizeof(JanetSourceMapping) * (size_t) count);
@@ -742,6 +721,9 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
         janet_asm_error(&a, "invalid assembly");
     }
 
+    /* Add final flags */
+    janet_def_addflags(def);
+
     /* Finish everything and return funcdef */
     janet_asm_deinit(&a);
     result.error = NULL;
@@ -859,92 +841,110 @@ Janet janet_asm_decode_instruction(uint32_t instr) {
     return janet_wrap_nil();
 }
 
-Janet janet_disasm(JanetFuncDef *def) {
+/*
-    int32_t i;
+ * Disasm sections
+ */
+
+static Janet janet_disasm_arity(JanetFuncDef *def) {
+    return janet_wrap_integer(def->arity);
+}
+
+static Janet janet_disasm_min_arity(JanetFuncDef *def) {
+    return janet_wrap_integer(def->min_arity);
+}
+
+static Janet janet_disasm_max_arity(JanetFuncDef *def) {
+    return janet_wrap_integer(def->max_arity);
+}
+
+static Janet janet_disasm_slotcount(JanetFuncDef *def) {
+    return janet_wrap_integer(def->slotcount);
+}
+
+static Janet janet_disasm_bytecode(JanetFuncDef *def) {
     JanetArray *bcode = janet_array(def->bytecode_length);
-    JanetArray *constants;
+    for (int32_t i = 0; i < def->bytecode_length; i++) {
-    JanetTable *ret = janet_table(10);
-    janet_table_put(ret, janet_csymbolv("arity"), janet_wrap_integer(def->arity));
-    janet_table_put(ret, janet_csymbolv("min-arity"), janet_wrap_integer(def->min_arity));
-    janet_table_put(ret, janet_csymbolv("max-arity"), janet_wrap_integer(def->max_arity));
-    janet_table_put(ret, janet_csymbolv("bytecode"), janet_wrap_array(bcode));
-    if (NULL != def->source) {
-        janet_table_put(ret, janet_csymbolv("source"), janet_wrap_string(def->source));
-    }
-    if (def->flags & JANET_FUNCDEF_FLAG_VARARG) {
-        janet_table_put(ret, janet_csymbolv("vararg"), janet_wrap_true());
-    }
-    if (NULL != def->name) {
-        janet_table_put(ret, janet_csymbolv("name"), janet_wrap_string(def->name));
-    }
-
-    /* Add constants */
-    if (def->constants_length > 0) {
-        constants = janet_array(def->constants_length);
-        janet_table_put(ret, janet_csymbolv("constants"), janet_wrap_array(constants));
-        for (i = 0; i < def->constants_length; i++) {
-            Janet src = def->constants[i];
-            Janet dest;
-            if (janet_checktype(src, JANET_TUPLE)) {
-                dest = janet_wrap_tuple(tup2(janet_csymbolv("quote"), src));
-            } else {
-                dest = src;
-            }
-            constants->data[i] = dest;
-        }
-        constants->count = def->constants_length;
-    }
-
-    /* Add bytecode */
-    for (i = 0; i < def->bytecode_length; i++) {
         bcode->data[i] = janet_asm_decode_instruction(def->bytecode[i]);
     }
     bcode->count = def->bytecode_length;
+    return janet_wrap_array(bcode);
+}
 
-    /* Add source map */
+static Janet janet_disasm_source(JanetFuncDef *def) {
-    if (NULL != def->sourcemap) {
+    if (def->source != NULL) return janet_wrap_string(def->source);
-        JanetArray *sourcemap = janet_array(def->bytecode_length);
+    return janet_wrap_nil();
-        for (i = 0; i < def->bytecode_length; i++) {
+}
-            Janet *t = janet_tuple_begin(2);
+
-            JanetSourceMapping mapping = def->sourcemap[i];
+static Janet janet_disasm_name(JanetFuncDef *def) {
-            t[0] = janet_wrap_integer(mapping.line);
+    if (def->name != NULL) return janet_wrap_string(def->name);
-            t[1] = janet_wrap_integer(mapping.column);
+    return janet_wrap_nil();
-            sourcemap->data[i] = janet_wrap_tuple(janet_tuple_end(t));
+}
-        }
+
-        sourcemap->count = def->bytecode_length;
+static Janet janet_disasm_vararg(JanetFuncDef *def) {
-        janet_table_put(ret, janet_csymbolv("sourcemap"), janet_wrap_array(sourcemap));
+    return janet_wrap_boolean(def->flags & JANET_FUNCDEF_FLAG_VARARG);
+}
+
+static Janet janet_disasm_constants(JanetFuncDef *def) {
+    JanetArray *constants = janet_array(def->constants_length);
+    for (int32_t i = 0; i < def->constants_length; i++) {
+        constants->data[i] = def->constants[i];
     }
+    constants->count = def->constants_length;
+    return janet_wrap_array(constants);
+}
 
-    /* Add environments */
+static Janet janet_disasm_sourcemap(JanetFuncDef *def) {
-    if (NULL != def->environments) {
+    if (NULL == def->sourcemap) return janet_wrap_nil();
-        JanetArray *envs = janet_array(def->environments_length);
+    JanetArray *sourcemap = janet_array(def->bytecode_length);
-        for (i = 0; i < def->environments_length; i++) {
+    for (int32_t i = 0; i < def->bytecode_length; i++) {
-            envs->data[i] = janet_wrap_integer(def->environments[i]);
+        Janet *t = janet_tuple_begin(2);
-        }
+        JanetSourceMapping mapping = def->sourcemap[i];
-        envs->count = def->environments_length;
+        t[0] = janet_wrap_integer(mapping.line);
-        janet_table_put(ret, janet_csymbolv("environments"), janet_wrap_array(envs));
+        t[1] = janet_wrap_integer(mapping.column);
+        sourcemap->data[i] = janet_wrap_tuple(janet_tuple_end(t));
     }
+    sourcemap->count = def->bytecode_length;
+    return janet_wrap_array(sourcemap);
+}
 
-    /* Add closures */
+static Janet janet_disasm_environments(JanetFuncDef *def) {
-    /* Funcdefs cannot be recursive */
+    JanetArray *envs = janet_array(def->environments_length);
-    if (NULL != def->defs) {
+    for (int32_t i = 0; i < def->environments_length; i++) {
-        JanetArray *defs = janet_array(def->defs_length);
+        envs->data[i] = janet_wrap_integer(def->environments[i]);
-        for (i = 0; i < def->defs_length; i++) {
-            defs->data[i] = janet_disasm(def->defs[i]);
-        }
-        defs->count = def->defs_length;
-        janet_table_put(ret, janet_csymbolv("defs"), janet_wrap_array(defs));
     }
+    envs->count = def->environments_length;
+    return janet_wrap_array(envs);
+}
 
-    /* Add slotcount */
+static Janet janet_disasm_defs(JanetFuncDef *def) {
-    janet_table_put(ret, janet_csymbolv("slotcount"), janet_wrap_integer(def->slotcount));
+    JanetArray *defs = janet_array(def->defs_length);
+    for (int32_t i = 0; i < def->defs_length; i++) {
+        defs->data[i] = janet_disasm(def->defs[i]);
+    }
+    defs->count = def->defs_length;
+    return janet_wrap_array(defs);
+}
 
+Janet janet_disasm(JanetFuncDef *def) {
+    JanetTable *ret = janet_table(10);
+    janet_table_put(ret, janet_ckeywordv("arity"), janet_disasm_arity(def));
+    janet_table_put(ret, janet_ckeywordv("min-arity"), janet_disasm_min_arity(def));
+    janet_table_put(ret, janet_ckeywordv("max-arity"), janet_disasm_max_arity(def));
+    janet_table_put(ret, janet_ckeywordv("bytecode"), janet_disasm_bytecode(def));
+    janet_table_put(ret, janet_ckeywordv("source"), janet_disasm_source(def));
+    janet_table_put(ret, janet_ckeywordv("vararg"), janet_disasm_vararg(def));
+    janet_table_put(ret, janet_ckeywordv("name"), janet_disasm_name(def));
+    janet_table_put(ret, janet_ckeywordv("slotcount"), janet_disasm_slotcount(def));
+    janet_table_put(ret, janet_ckeywordv("constants"), janet_disasm_constants(def));
+    janet_table_put(ret, janet_ckeywordv("sourcemap"), janet_disasm_sourcemap(def));
+    janet_table_put(ret, janet_ckeywordv("environments"), janet_disasm_environments(def));
+    janet_table_put(ret, janet_ckeywordv("defs"), janet_disasm_defs(def));
     return janet_wrap_struct(janet_table_to_struct(ret));
 }
 
 /* C Function for assembly */
 static Janet cfun_asm(int32_t argc, Janet *argv) {
-    janet_arity(argc, 1, 1);
+    janet_fixarity(argc, 1);
     JanetAssembleResult res;
     res = janet_asm(argv[0], 0);
     if (res.status != JANET_ASSEMBLE_OK) {
@@ -954,9 +954,26 @@ static Janet cfun_asm(int32_t argc, Janet *argv) {
 }
 
 static Janet cfun_disasm(int32_t argc, Janet *argv) {
-    janet_arity(argc, 1, 1);
+    janet_arity(argc, 1, 2);
     JanetFunction *f = janet_getfunction(argv, 0);
-    return janet_disasm(f->def);
+    if (argc == 2) {
+        JanetKeyword kw = janet_getkeyword(argv, 1);
+        if (!janet_cstrcmp(kw, "arity")) return janet_disasm_arity(f->def);
+        if (!janet_cstrcmp(kw, "min-arity")) return janet_disasm_min_arity(f->def);
+        if (!janet_cstrcmp(kw, "max-arity")) return janet_disasm_max_arity(f->def);
+        if (!janet_cstrcmp(kw, "bytecode")) return janet_disasm_bytecode(f->def);
+        if (!janet_cstrcmp(kw, "source")) return janet_disasm_source(f->def);
+        if (!janet_cstrcmp(kw, "name")) return janet_disasm_name(f->def);
+        if (!janet_cstrcmp(kw, "vararg")) return janet_disasm_vararg(f->def);
+        if (!janet_cstrcmp(kw, "slotcount")) return janet_disasm_slotcount(f->def);
+        if (!janet_cstrcmp(kw, "constants")) return janet_disasm_constants(f->def);
+        if (!janet_cstrcmp(kw, "sourcemap")) return janet_disasm_sourcemap(f->def);
+        if (!janet_cstrcmp(kw, "environments")) return janet_disasm_environments(f->def);
+        if (!janet_cstrcmp(kw, "defs")) return janet_disasm_defs(f->def);
+        janet_panicf("unknown disasm key %v", argv[1]);
+    } else {
+        return janet_disasm(f->def);
+    }
 }
 
 static const JanetReg asm_cfuns[] = {
@@ -964,15 +981,29 @@ static const JanetReg asm_cfuns[] = {
         "asm", cfun_asm,
         JDOC("(asm assembly)\n\n"
              "Returns a new function that is the compiled result of the assembly.\n"
-             "The syntax for the assembly can be found on the Janet website. Will throw an\n"
+             "The syntax for the assembly can be found on the Janet website, and should correspond\n"
+             "to the return value of disasm. Will throw an\n"
              "error on invalid assembly.")
     },
     {
         "disasm", cfun_disasm,
-        JDOC("(disasm func)\n\n"
+        JDOC("(disasm func &opt field)\n\n"
              "Returns assembly that could be used be compile the given function.\n"
              "func must be a function, not a c function. Will throw on error on a badly\n"
-             "typed argument.")
+             "typed argument. If given a field name, will only return that part of the function assembly.\n"
+             "Possible fields are:\n\n"
+             "\t:arity - number of required and optional arguments.\n"
+             "\t:min-arity - minimum number of arguments function can be called with.\n"
+             "\t:max-arity - maximum number of arguments function can be called with.\n"
+             "\t:vararg - true if function can take a variable number of arguments.\n"
+             "\t:bytecode - array of parsed bytecode instructions. Each instruction is a tuple.\n"
+             "\t:source - name of source file that this function was compiled from.\n"
+             "\t:name - name of function.\n"
+             "\t:slotcount - how many virtual registers, or slots, this function uses. Corresponds to stack space used by function.\n"
+             "\t:constants - an array of constants referenced by this function.\n"
+             "\t:sourcemap - a mapping of each bytecode instruction to a line and column in the source file.\n"
+             "\t:environments - an internal mapping of which enclosing functions are referenced for bindings.\n"
+             "\t:defs - other function definitions that this function may instantiate.\n")
     },
     {NULL, NULL, NULL}
 };

src/core/buffer.c

@@ -197,6 +197,26 @@ static Janet cfun_buffer_fill(int32_t argc, Janet *argv) {
     return argv[0];
 }
 
+static Janet cfun_buffer_trim(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    JanetBuffer *buffer = janet_getbuffer(argv, 0);
+    if (buffer->count) {
+        if (buffer->count < buffer->capacity) {
+            uint8_t *newData = realloc(buffer->data, buffer->count);
+            if (NULL == newData) {
+                JANET_OUT_OF_MEMORY;
+            }
+            buffer->data = newData;
+            buffer->capacity = buffer->count;
+        }
+    } else {
+        buffer->capacity = 0;
+        free(buffer->data);
+        buffer->data = NULL;
+    }
+    return argv[0];
+}
+
 static Janet cfun_buffer_u8(int32_t argc, Janet *argv) {
     int32_t i;
     janet_arity(argc, 1, -1);
@@ -379,6 +399,12 @@ static const JanetReg buffer_cfuns[] = {
              "Fill up a buffer with bytes, defaulting to 0s. Does not change the buffer's length. "
              "Returns the modified buffer.")
     },
+    {
+        "buffer/trim", cfun_buffer_trim,
+        JDOC("(buffer/trim buffer)\n\n"
+             "Set the backing capacity of the buffer to the current length of the buffer. Returns the "
+             "modified buffer.")
+    },
     {
         "buffer/push-byte", cfun_buffer_u8,
         JDOC("(buffer/push-byte buffer x)\n\n"

src/core/bytecode.c

@@ -101,10 +101,13 @@ enum JanetInstructionType janet_instructions[JOP_INSTRUCTION_COUNT] = {
     JINT_SSS, /* JOP_GREATER_THAN_EQUAL */
     JINT_SSS, /* JOP_LESS_THAN_EQUAL */
     JINT_SSS, /* JOP_NEXT */
+    JINT_SSS, /* JOP_NOT_EQUALS, */
+    JINT_SSI, /* JOP_NOT_EQUALS_IMMEDIATE, */
+    JINT_SSS /* JOP_CANCEL, */
 };
 
 /* Verify some bytecode */
-int32_t janet_verify(JanetFuncDef *def) {
+int janet_verify(JanetFuncDef *def) {
     int vargs = !!(def->flags & JANET_FUNCDEF_FLAG_VARARG);
     int32_t i;
     int32_t maxslot = def->arity + vargs;

src/core/capi.c

@@ -27,12 +27,26 @@
 #include "fiber.h"
 #endif
 
+#ifndef JANET_SINGLE_THREADED
+#ifndef JANET_WINDOWS
+#include <pthread.h>
+#else
+#include <windows.h>
+#endif
+#endif
+
 JANET_NO_RETURN static void janet_top_level_signal(const char *msg) {
 #ifdef JANET_TOP_LEVEL_SIGNAL
     JANET_TOP_LEVEL_SIGNAL(msg);
 #else
     fputs(msg, stdout);
-    exit(1);
+# ifdef JANET_SINGLE_THREADED
+    exit(-1);
+# elif defined(JANET_WINDOWS)
+    ExitThread(-1);
+# else
+    pthread_exit(NULL);
+# endif
 #endif
 }
 
@@ -325,7 +339,10 @@ JanetRange janet_getslice(int32_t argc, const Janet *argv) {
 }
 
 Janet janet_dyn(const char *name) {
-    if (!janet_vm_fiber) return janet_wrap_nil();
+    if (!janet_vm_fiber) {
+        if (!janet_vm_top_dyns) return janet_wrap_nil();
+        return janet_table_get(janet_vm_top_dyns, janet_ckeywordv(name));
+    }
     if (janet_vm_fiber->env) {
         return janet_table_get(janet_vm_fiber->env, janet_ckeywordv(name));
     } else {
@@ -334,11 +351,15 @@ Janet janet_dyn(const char *name) {
 }
 
 void janet_setdyn(const char *name, Janet value) {
-    if (!janet_vm_fiber) return;
+    if (!janet_vm_fiber) {
-    if (!janet_vm_fiber->env) {
+        if (!janet_vm_top_dyns) janet_vm_top_dyns = janet_table(10);
-        janet_vm_fiber->env = janet_table(1);
+        janet_table_put(janet_vm_top_dyns, janet_ckeywordv(name), value);
+    } else {
+        if (!janet_vm_fiber->env) {
+            janet_vm_fiber->env = janet_table(1);
+        }
+        janet_table_put(janet_vm_fiber->env, janet_ckeywordv(name), value);
     }
-    janet_table_put(janet_vm_fiber->env, janet_ckeywordv(name), value);
 }
 
 uint64_t janet_getflags(const Janet *argv, int32_t n, const char *flags) {

src/core/cfuns.c

@@ -33,6 +33,11 @@ static int arity1or2(JanetFopts opts, JanetSlot *args) {
     int32_t arity = janet_v_count(args);
     return arity == 1 || arity == 2;
 }
+static int arity2or3(JanetFopts opts, JanetSlot *args) {
+    (void) opts;
+    int32_t arity = janet_v_count(args);
+    return arity == 2 || arity == 3;
+}
 static int fixarity1(JanetFopts opts, JanetSlot *args) {
     (void) opts;
     return janet_v_count(args) == 1;
@@ -90,34 +95,67 @@ static JanetSlot opfunction(
     return t;
 }
 
+/* Check if a value can be coerced to an immediate value */
+static int can_be_imm(Janet x, int8_t *out) {
+    if (!janet_checkint(x)) return 0;
+    int32_t integer = janet_unwrap_integer(x);
+    if (integer > 127 || integer < -127) return 0;
+    *out = (int8_t) integer;
+    return 1;
+}
+
+/* Check if a slot can be coerced to an immediate value */
+static int can_slot_be_imm(JanetSlot s, int8_t *out) {
+    if (!(s.flags & JANET_SLOT_CONSTANT)) return 0;
+    return can_be_imm(s.constant, out);
+}
+
 /* Emit a series of instructions instead of a function call to a math op */
 static JanetSlot opreduce(
     JanetFopts opts,
     JanetSlot *args,
     int op,
+    int opim,
     Janet nullary) {
     JanetCompiler *c = opts.compiler;
     int32_t i, len;
+    int8_t imm = 0;
+    int neg = opim < 0;
+    if (opim < 0) opim = -opim;
     len = janet_v_count(args);
     JanetSlot t;
     if (len == 0) {
         return janetc_cslot(nullary);
     } else if (len == 1) {
         t = janetc_gettarget(opts);
-        janetc_emit_sss(c, op, t, janetc_cslot(nullary), args[0], 1);
+        /* Special case subtract to be times -1 */
+        if (op == JOP_SUBTRACT) {
+            janetc_emit_ssi(c, JOP_MULTIPLY_IMMEDIATE, t, args[0], -1, 1);
+        } else {
+            janetc_emit_sss(c, op, t, janetc_cslot(nullary), args[0], 1);
+        }
         return t;
     }
     t = janetc_gettarget(opts);
-    janetc_emit_sss(c, op, t, args[0], args[1], 1);
+    if (opim && can_slot_be_imm(args[1], &imm)) {
-    for (i = 2; i < len; i++)
+        janetc_emit_ssi(c, opim, t, args[0], neg ? -imm : imm, 1);
-        janetc_emit_sss(c, op, t, t, args[i], 1);
+    } else {
+        janetc_emit_sss(c, op, t, args[0], args[1], 1);
+    }
+    for (i = 2; i < len; i++) {
+        if (opim && can_slot_be_imm(args[i], &imm)) {
+            janetc_emit_ssi(c, opim, t, t, neg ? -imm : imm, 1);
+        } else {
+            janetc_emit_sss(c, op, t, t, args[i], 1);
+        }
+    }
     return t;
 }
 
 /* Function optimizers */
 
 static JanetSlot do_propagate(JanetFopts opts, JanetSlot *args) {
-    return opreduce(opts, args, JOP_PROPAGATE, janet_wrap_nil());
+    return opreduce(opts, args, JOP_PROPAGATE, 0, janet_wrap_nil());
 }
 static JanetSlot do_error(JanetFopts opts, JanetSlot *args) {
     janetc_emit_s(opts.compiler, JOP_ERROR, args[0], 0);
@@ -134,19 +172,40 @@ static JanetSlot do_debug(JanetFopts opts, JanetSlot *args) {
     return t;
 }
 static JanetSlot do_in(JanetFopts opts, JanetSlot *args) {
-    return opreduce(opts, args, JOP_IN, janet_wrap_nil());
+    return opreduce(opts, args, JOP_IN, 0, janet_wrap_nil());
 }
 static JanetSlot do_get(JanetFopts opts, JanetSlot *args) {
-    return opreduce(opts, args, JOP_GET, janet_wrap_nil());
+    if (janet_v_count(args) == 3) {
+        JanetCompiler *c = opts.compiler;
+        JanetSlot t = janetc_gettarget(opts);
+        int target_is_default = janetc_sequal(t, args[2]);
+        JanetSlot dflt_slot = args[2];
+        if (target_is_default) {
+            dflt_slot = janetc_farslot(c);
+            janetc_copy(c, dflt_slot, t);
+        }
+        janetc_emit_sss(c, JOP_GET, t, args[0], args[1], 1);
+        int32_t label = janetc_emit_si(c, JOP_JUMP_IF_NOT_NIL, t, 0, 0);
+        janetc_copy(c, t, dflt_slot);
+        if (target_is_default) janetc_freeslot(c, dflt_slot);
+        int32_t current = janet_v_count(c->buffer);
+        c->buffer[label] |= (current - label) << 16;
+        return t;
+    } else {
+        return opreduce(opts, args, JOP_GET, 0, janet_wrap_nil());
+    }
 }
 static JanetSlot do_next(JanetFopts opts, JanetSlot *args) {
     return opfunction(opts, args, JOP_NEXT, janet_wrap_nil());
 }
 static JanetSlot do_modulo(JanetFopts opts, JanetSlot *args) {
-    return opreduce(opts, args, JOP_MODULO, janet_wrap_nil());
+    return opreduce(opts, args, JOP_MODULO, 0, janet_wrap_nil());
 }
 static JanetSlot do_remainder(JanetFopts opts, JanetSlot *args) {
-    return opreduce(opts, args, JOP_REMAINDER, janet_wrap_nil());
+    return opreduce(opts, args, JOP_REMAINDER, 0, janet_wrap_nil());
+}
+static JanetSlot do_cmp(JanetFopts opts, JanetSlot *args) {
+    return opreduce(opts, args, JOP_COMPARE, 0, janet_wrap_nil());
 }
 static JanetSlot do_put(JanetFopts opts, JanetSlot *args) {
     if (opts.flags & JANET_FOPTS_DROP) {
@@ -172,6 +231,9 @@ static JanetSlot do_yield(JanetFopts opts, JanetSlot *args) {
 static JanetSlot do_resume(JanetFopts opts, JanetSlot *args) {
     return opfunction(opts, args, JOP_RESUME, janet_wrap_nil());
 }
+static JanetSlot do_cancel(JanetFopts opts, JanetSlot *args) {
+    return opfunction(opts, args, JOP_CANCEL, janet_wrap_nil());
+}
 static JanetSlot do_apply(JanetFopts opts, JanetSlot *args) {
     /* Push phase */
     JanetCompiler *c = opts.compiler;
@@ -200,34 +262,34 @@ static JanetSlot do_apply(JanetFopts opts, JanetSlot *args) {
 /* Variadic operators specialization */
 
 static JanetSlot do_add(JanetFopts opts, JanetSlot *args) {
-    return opreduce(opts, args, JOP_ADD, janet_wrap_integer(0));
+    return opreduce(opts, args, JOP_ADD, JOP_ADD_IMMEDIATE, janet_wrap_integer(0));
 }
 static JanetSlot do_sub(JanetFopts opts, JanetSlot *args) {
-    return opreduce(opts, args, JOP_SUBTRACT, janet_wrap_integer(0));
+    return opreduce(opts, args, JOP_SUBTRACT, -JOP_ADD_IMMEDIATE, janet_wrap_integer(0));
 }
 static JanetSlot do_mul(JanetFopts opts, JanetSlot *args) {
-    return opreduce(opts, args, JOP_MULTIPLY, janet_wrap_integer(1));
+    return opreduce(opts, args, JOP_MULTIPLY, JOP_MULTIPLY_IMMEDIATE, janet_wrap_integer(1));
 }
 static JanetSlot do_div(JanetFopts opts, JanetSlot *args) {
-    return opreduce(opts, args, JOP_DIVIDE, janet_wrap_integer(1));
+    return opreduce(opts, args, JOP_DIVIDE, JOP_DIVIDE_IMMEDIATE, janet_wrap_integer(1));
 }
 static JanetSlot do_band(JanetFopts opts, JanetSlot *args) {
-    return opreduce(opts, args, JOP_BAND, janet_wrap_integer(-1));
+    return opreduce(opts, args, JOP_BAND, 0, janet_wrap_integer(-1));
 }
 static JanetSlot do_bor(JanetFopts opts, JanetSlot *args) {
-    return opreduce(opts, args, JOP_BOR, janet_wrap_integer(0));
+    return opreduce(opts, args, JOP_BOR, 0, janet_wrap_integer(0));
 }
 static JanetSlot do_bxor(JanetFopts opts, JanetSlot *args) {
-    return opreduce(opts, args, JOP_BXOR, janet_wrap_integer(0));
+    return opreduce(opts, args, JOP_BXOR, 0, janet_wrap_integer(0));
 }
 static JanetSlot do_lshift(JanetFopts opts, JanetSlot *args) {
-    return opreduce(opts, args, JOP_SHIFT_LEFT, janet_wrap_integer(1));
+    return opreduce(opts, args, JOP_SHIFT_LEFT, JOP_SHIFT_LEFT_IMMEDIATE, janet_wrap_integer(1));
 }
 static JanetSlot do_rshift(JanetFopts opts, JanetSlot *args) {
-    return opreduce(opts, args, JOP_SHIFT_RIGHT, janet_wrap_integer(1));
+    return opreduce(opts, args, JOP_SHIFT_RIGHT, JOP_SHIFT_RIGHT_IMMEDIATE, janet_wrap_integer(1));
 }
 static JanetSlot do_rshiftu(JanetFopts opts, JanetSlot *args) {
-    return opreduce(opts, args, JOP_SHIFT_RIGHT, janet_wrap_integer(1));
+    return opreduce(opts, args, JOP_SHIFT_RIGHT_UNSIGNED, JOP_SHIFT_RIGHT_UNSIGNED_IMMEDIATE, janet_wrap_integer(1));
 }
 static JanetSlot do_bnot(JanetFopts opts, JanetSlot *args) {
     return genericSS(opts, JOP_BNOT, args[0]);
@@ -238,9 +300,11 @@ static JanetSlot compreduce(
     JanetFopts opts,
     JanetSlot *args,
     int op,
+    int opim,
     int invert) {
     JanetCompiler *c = opts.compiler;
     int32_t i, len;
+    int8_t imm = 0;
     len = janet_v_count(args);
     int32_t *labels = NULL;
     JanetSlot t;
@@ -251,19 +315,17 @@ static JanetSlot compreduce(
     }
     t = janetc_gettarget(opts);
     for (i = 1; i < len; i++) {
-        janetc_emit_sss(c, op, t, args[i - 1], args[i], 1);
+        if (opim && can_slot_be_imm(args[i], &imm)) {
+            janetc_emit_ssi(c, opim, t, args[i - 1], imm, 1);
+        } else {
+            janetc_emit_sss(c, op, t, args[i - 1], args[i], 1);
+        }
         if (i != (len - 1)) {
-            int32_t label = janetc_emit_si(c, JOP_JUMP_IF_NOT, t, 0, 1);
+            int32_t label = janetc_emit_si(c, invert ? JOP_JUMP_IF : JOP_JUMP_IF_NOT, t, 0, 1);
             janet_v_push(labels, label);
         }
     }
     int32_t end = janet_v_count(c->buffer);
-    if (invert) {
-        janetc_emit_si(c, JOP_JUMP_IF, t, 3, 0);
-        janetc_emit_s(c, JOP_LOAD_TRUE, t, 1);
-        janetc_emit(c, JOP_JUMP | (2 << 8));
-        janetc_emit_s(c, JOP_LOAD_FALSE, t, 1);
-    }
     for (i = 0; i < janet_v_count(labels); i++) {
         int32_t label = labels[i];
         c->buffer[label] |= ((end - label) << 16);
@@ -273,22 +335,22 @@ static JanetSlot compreduce(
 }
 
 static JanetSlot do_gt(JanetFopts opts, JanetSlot *args) {
-    return compreduce(opts, args, JOP_GREATER_THAN, 0);
+    return compreduce(opts, args, JOP_GREATER_THAN, JOP_GREATER_THAN_IMMEDIATE, 0);
 }
 static JanetSlot do_lt(JanetFopts opts, JanetSlot *args) {
-    return compreduce(opts, args, JOP_LESS_THAN, 0);
+    return compreduce(opts, args, JOP_LESS_THAN, JOP_LESS_THAN_IMMEDIATE, 0);
 }
 static JanetSlot do_gte(JanetFopts opts, JanetSlot *args) {
-    return compreduce(opts, args, JOP_GREATER_THAN_EQUAL, 0);
+    return compreduce(opts, args, JOP_GREATER_THAN_EQUAL, 0, 0);
 }
 static JanetSlot do_lte(JanetFopts opts, JanetSlot *args) {
-    return compreduce(opts, args, JOP_LESS_THAN_EQUAL, 0);
+    return compreduce(opts, args, JOP_LESS_THAN_EQUAL, 0, 0);
 }
 static JanetSlot do_eq(JanetFopts opts, JanetSlot *args) {
-    return compreduce(opts, args, JOP_EQUALS, 0);
+    return compreduce(opts, args, JOP_EQUALS, JOP_EQUALS_IMMEDIATE, 0);
 }
 static JanetSlot do_neq(JanetFopts opts, JanetSlot *args) {
-    return compreduce(opts, args, JOP_EQUALS, 1);
+    return compreduce(opts, args, JOP_NOT_EQUALS, JOP_NOT_EQUALS_IMMEDIATE, 1);
 }
 
 /* Arranged by tag */
@@ -319,10 +381,12 @@ static const JanetFunOptimizer optimizers[] = {
     {NULL, do_eq},
     {NULL, do_neq},
     {fixarity2, do_propagate},
-    {fixarity2, do_get},
+    {arity2or3, do_get},
     {arity1or2, do_next},
     {fixarity2, do_modulo},
     {fixarity2, do_remainder},
+    {fixarity2, do_cmp},
+    {fixarity2, do_cancel},
 };
 
 const JanetFunOptimizer *janetc_funopt(uint32_t flags) {

src/core/compile.c

@@ -596,8 +596,11 @@ static int macroexpand1(
     /* Set env */
     fiberp->env = c->env;
     int lock = janet_gclock();
+    Janet mf_kw = janet_ckeywordv("macro-form");
+    janet_table_put(c->env, mf_kw, x);
     Janet tempOut;
     JanetSignal status = janet_continue(fiberp, janet_wrap_nil(), &tempOut);
+    janet_table_put(c->env, mf_kw, janet_wrap_nil());
     janet_gcunlock(lock);
     if (status != JANET_SIGNAL_OK) {
         const uint8_t *es = janet_formatc("(macro) %V", tempOut);
@@ -695,7 +698,32 @@ JanetSlot janetc_value(JanetFopts opts, Janet x) {
     return ret;
 }
 
+/* Add function flags to janet functions */
+void janet_def_addflags(JanetFuncDef *def) {
+    int32_t set_flags = 0;
+    int32_t unset_flags = 0;
+    /* pos checks */
+    if (def->name)            set_flags |= JANET_FUNCDEF_FLAG_HASNAME;
+    if (def->source)          set_flags |= JANET_FUNCDEF_FLAG_HASSOURCE;
+    if (def->defs)            set_flags |= JANET_FUNCDEF_FLAG_HASDEFS;
+    if (def->environments)    set_flags |= JANET_FUNCDEF_FLAG_HASENVS;
+    if (def->sourcemap)       set_flags |= JANET_FUNCDEF_FLAG_HASSOURCEMAP;
+    if (def->closure_bitset)  set_flags |= JANET_FUNCDEF_FLAG_HASCLOBITSET;
+    /* negative checks */
+    if (!def->name)           unset_flags |= JANET_FUNCDEF_FLAG_HASNAME;
+    if (!def->source)         unset_flags |= JANET_FUNCDEF_FLAG_HASSOURCE;
+    if (!def->defs)           unset_flags |= JANET_FUNCDEF_FLAG_HASDEFS;
+    if (!def->environments)   unset_flags |= JANET_FUNCDEF_FLAG_HASENVS;
+    if (!def->sourcemap)      unset_flags |= JANET_FUNCDEF_FLAG_HASSOURCEMAP;
+    if (!def->closure_bitset) unset_flags |= JANET_FUNCDEF_FLAG_HASCLOBITSET;
+    /* Update flags */
+    def->flags |= set_flags;
+    def->flags &= ~unset_flags;
+}
+
 /* Compile a funcdef */
+/* Once the various other settings of the FuncDef have been tweaked,
+ * call janet_def_addflags to set the proper flags for the funcdef */
 JanetFuncDef *janetc_pop_funcdef(JanetCompiler *c) {
     JanetScope *scope = c->scope;
     JanetFuncDef *def = janet_funcdef_alloc();
@@ -747,8 +775,10 @@ JanetFuncDef *janetc_pop_funcdef(JanetCompiler *c) {
     /* Copy upvalue bitset */
     if (scope->ua.count) {
         /* Number of u32s we need to create a bitmask for all slots */
-        int32_t numchunks = (def->slotcount + 31) >> 5;
+        int32_t slotchunks = (def->slotcount + 31) >> 5;
-        uint32_t *chunks = malloc(sizeof(uint32_t) * numchunks);
+        /* numchunks is min of slotchunks and scope->ua.count */
+        int32_t numchunks = slotchunks > scope->ua.count ? scope->ua.count : slotchunks;
+        uint32_t *chunks = calloc(sizeof(uint32_t), slotchunks);
         if (NULL == chunks) {
             JANET_OUT_OF_MEMORY;
         }
@@ -756,7 +786,6 @@ JanetFuncDef *janetc_pop_funcdef(JanetCompiler *c) {
         /* Register allocator preallocates some registers [240-255, high 16 bits of chunk index 7], we can ignore those. */
         if (scope->ua.count > 7) chunks[7] &= 0xFFFFU;
         def->closure_bitset = chunks;
-        def->flags |= JANET_FUNCDEF_FLAG_HASCLOBITSET;
     }
 
     /* Pop the scope */
@@ -813,6 +842,7 @@ JanetCompileResult janet_compile(Janet source, JanetTable *env, const uint8_t *w
     if (c.result.status == JANET_COMPILE_OK) {
         JanetFuncDef *def = janetc_pop_funcdef(&c);
         def->name = janet_cstring("_thunk");
+        janet_def_addflags(def);
         c.result.funcdef = def;
     } else {
         c.result.error_mapping = c.current_mapping;

src/core/compile.h

@@ -60,6 +60,8 @@
 #define JANET_FUN_NEXT 28
 #define JANET_FUN_MODULO 29
 #define JANET_FUN_REMAINDER 30
+#define JANET_FUN_CMP 31
+#define JANET_FUN_CANCEL 32
 
 /* Compiler typedefs */
 typedef struct JanetCompiler JanetCompiler;

src/core/corelib.c

@@ -63,10 +63,29 @@ typedef void *Clib;
 #define error_clib() dlerror()
 #endif
 
+static char *get_processed_name(const char *name) {
+    if (name[0] == '.') return (char *) name;
+    const char *c;
+    for (c = name; *c; c++) {
+        if (*c == '/') return (char *) name;
+    }
+    size_t l = (size_t)(c - name);
+    char *ret = malloc(l + 3);
+    if (NULL == ret) {
+        JANET_OUT_OF_MEMORY;
+    }
+    ret[0] = '.';
+    ret[1] = '/';
+    memcpy(ret + 2, name, l + 1);
+    return ret;
+}
+
 JanetModule janet_native(const char *name, const uint8_t **error) {
-    Clib lib = load_clib(name);
+    char *processed_name = get_processed_name(name);
+    Clib lib = load_clib(processed_name);
     JanetModule init;
     JanetModconf getter;
+    if (name != processed_name) free(processed_name);
     if (!lib) {
         *error = janet_cstring(error_clib());
         return NULL;
@@ -404,9 +423,11 @@ static Janet janet_core_gcsetinterval(int32_t argc, Janet *argv) {
     janet_fixarity(argc, 1);
     size_t s = janet_getsize(argv, 0);
     /* limit interval to 48 bits */
-    if (s > 0xFFFFFFFFFFFFUl) {
+#ifdef JANET_64
+    if (s >> 48) {
         janet_panic("interval too large");
     }
+#endif
     janet_vm_gc_interval = s;
     return janet_wrap_nil();
 }
@@ -739,6 +760,7 @@ static void janet_quick_asm(
         JANET_OUT_OF_MEMORY;
     }
     memcpy(def->bytecode, bytecode, bytecode_size);
+    janet_def_addflags(def);
     janet_def(env, name, janet_wrap_function(janet_thunk(def)), doc);
 }
 
@@ -924,6 +946,10 @@ static const uint32_t resume_asm[] = {
     JOP_RESUME | (1 << 24),
     JOP_RETURN
 };
+static const uint32_t cancel_asm[] = {
+    JOP_CANCEL | (1 << 24),
+    JOP_RETURN
+};
 static const uint32_t in_asm[] = {
     JOP_IN | (1 << 24),
     JOP_LOAD_NIL | (3 << 8),
@@ -968,6 +994,10 @@ static const uint32_t remainder_asm[] = {
     JOP_REMAINDER | (1 << 24),
     JOP_RETURN
 };
+static const uint32_t cmp_asm[] = {
+    JOP_COMPARE | (1 << 24),
+    JOP_RETURN
+};
 #endif /* ifdef JANET_BOOTSTRAP */
 
 /*
@@ -1021,6 +1051,11 @@ JanetTable *janet_core_env(JanetTable *replacements) {
                     "%", 2, 2, 2, 2, remainder_asm, sizeof(remainder_asm),
                     JDOC("(% dividend divisor)\n\n"
                          "Returns the remainder of dividend / divisor."));
+    janet_quick_asm(env, JANET_FUN_CMP,
+                    "cmp", 2, 2, 2, 2, cmp_asm, sizeof(cmp_asm),
+                    JDOC("(cmp x y)\n\n"
+                         "Returns -1 if x is strictly less than y, 1 if y is strictly greater "
+                         "than x, and 0 otherwise. To return 0, x and y must be the exact same type."));
     janet_quick_asm(env, JANET_FUN_NEXT,
                     "next", 2, 1, 2, 2, next_asm, sizeof(next_asm),
                     JDOC("(next ds &opt key)\n\n"
@@ -1052,6 +1087,11 @@ JanetTable *janet_core_env(JanetTable *replacements) {
                          "Yield a value to a parent fiber. When a fiber yields, its execution is paused until "
                          "another thread resumes it. The fiber will then resume, and the last yield call will "
                          "return the value that was passed to resume."));
+    janet_quick_asm(env, JANET_FUN_CANCEL,
+                    "cancel", 2, 2, 2, 2, cancel_asm, sizeof(cancel_asm),
+                    JDOC("(cancel fiber err)\n\n"
+                         "Resume a fiber but have it immediately raise an error. This lets a programmer unwind a pending fiber. "
+                         "Returns the same result as resume."));
     janet_quick_asm(env, JANET_FUN_RESUME,
                     "resume", 2, 1, 2, 2, resume_asm, sizeof(resume_asm),
                     JDOC("(resume fiber &opt x)\n\n"

src/core/emit.c

@@ -37,7 +37,7 @@ int32_t janetc_allocfar(JanetCompiler *c) {
     return reg;
 }
 
-/* Get a register less than 256 */
+/* Get a register less than 256 for temporary use. */
 int32_t janetc_allocnear(JanetCompiler *c, JanetcRegisterTemp tag) {
     return janetc_regalloc_temp(&c->scope->ra, tag);
 }
@@ -205,7 +205,7 @@ static int32_t janetc_regnear(JanetCompiler *c, JanetSlot s, JanetcRegisterTemp
 }
 
 /* Check if two slots are equal */
-static int janetc_sequal(JanetSlot lhs, JanetSlot rhs) {
+int janetc_sequal(JanetSlot lhs, JanetSlot rhs) {
     if ((lhs.flags & ~JANET_SLOTTYPE_ANY) == (rhs.flags & ~JANET_SLOTTYPE_ANY) &&
             lhs.index == rhs.index &&
             lhs.envindex == rhs.envindex) {
@@ -245,8 +245,8 @@ void janetc_copy(
     janetc_moveback(c, dest, nearreg);
     /* Cleanup */
     janetc_regalloc_freetemp(&c->scope->ra, nearreg, JANETC_REGTEMP_3);
-
 }
+
 /* Instruction templated emitters */
 
 static int32_t emit1s(JanetCompiler *c, uint8_t op, JanetSlot s, int32_t rest, int wr) {

src/core/emit.h

@@ -42,6 +42,9 @@ int32_t janetc_emit_ssi(JanetCompiler *c, uint8_t op, JanetSlot s1, JanetSlot s2
 int32_t janetc_emit_ssu(JanetCompiler *c, uint8_t op, JanetSlot s1, JanetSlot s2, uint8_t immediate, int wr);
 int32_t janetc_emit_sss(JanetCompiler *c, uint8_t op, JanetSlot s1, JanetSlot s2, JanetSlot s3, int wr);
 
+/* Check if two slots are equivalent */
+int janetc_sequal(JanetSlot x, JanetSlot y);
+
 /* Move value from one slot to another. Cannot copy to constant slots. */
 void janetc_copy(JanetCompiler *c, JanetSlot dest, JanetSlot src);
 

src/core/features.h

@@ -25,8 +25,15 @@
 #ifndef JANET_FEATURES_H_defined
 #define JANET_FEATURES_H_defined
 
-#ifndef _POSIX_C_SOURCE
+#if defined(__NetBSD__) || defined(__APPLE__) || defined(__OpenBSD__) \
-#define _POSIX_C_SOURCE 200809L
+    || defined(__bsdi__) || defined(__DragonFly__)
+/* Use BSD soucre on any BSD systems, include OSX */
+# define _BSD_SOURCE
+#else
+/* Use POSIX feature flags */
+# ifndef _POSIX_C_SOURCE
+# define _POSIX_C_SOURCE 200809L
+# endif
 #endif
 
 #if defined(WIN32) || defined(_WIN32)
@@ -38,4 +45,11 @@
 #define _XOPEN_SOURCE 500
 #endif
 
+/* Needed for timegm and other extensions when building with -std=c99.
+ * It also defines realpath, etc, which would normally require
+ * _XOPEN_SOURCE >= 500. */
+#if !defined(_NETBSD_SOURCE) && defined(__NetBSD__)
+#define _NETBSD_SOURCE
+#endif
+
 #endif

src/core/fiber.c

@@ -85,6 +85,22 @@ JanetFiber *janet_fiber(JanetFunction *callee, int32_t capacity, int32_t argc, c
     return janet_fiber_reset(fiber_alloc(capacity), callee, argc, argv);
 }
 
+#ifdef JANET_DEBUG
+/* Test for memory issues by reallocating fiber every time we push a stack frame */
+static void janet_fiber_refresh_memory(JanetFiber *fiber) {
+    int32_t n = fiber->capacity;
+    if (n) {
+        Janet *newData = malloc(sizeof(Janet) * n);
+        if (NULL == newData) {
+            JANET_OUT_OF_MEMORY;
+        }
+        memcpy(newData, fiber->data, fiber->capacity * sizeof(Janet));
+        free(fiber->data);
+        fiber->data = newData;
+    }
+}
+#endif
+
 /* Ensure that the fiber has enough extra capacity */
 void janet_fiber_setcapacity(JanetFiber *fiber, int32_t n) {
     Janet *newData = realloc(fiber->data, sizeof(Janet) * n);
@@ -173,6 +189,10 @@ int janet_fiber_funcframe(JanetFiber *fiber, JanetFunction *func) {
 
     if (fiber->capacity < nextstacktop) {
         janet_fiber_setcapacity(fiber, 2 * nextstacktop);
+#ifdef JANET_DEBUG
+    } else {
+        janet_fiber_refresh_memory(fiber);
+#endif
     }
 
     /* Nil unset stack arguments (Needed for gc correctness) */
@@ -305,6 +325,10 @@ int janet_fiber_funcframe_tail(JanetFiber *fiber, JanetFunction *func) {
 
     if (fiber->capacity < nextstacktop) {
         janet_fiber_setcapacity(fiber, 2 * nextstacktop);
+#ifdef JANET_DEBUG
+    } else {
+        janet_fiber_refresh_memory(fiber);
+#endif
     }
 
     Janet *stack = fiber->data + fiber->frame;
@@ -367,6 +391,10 @@ void janet_fiber_cframe(JanetFiber *fiber, JanetCFunction cfun) {
 
     if (fiber->capacity < nextstacktop) {
         janet_fiber_setcapacity(fiber, 2 * nextstacktop);
+#ifdef JANET_DEBUG
+    } else {
+        janet_fiber_refresh_memory(fiber);
+#endif
     }
 
     /* Set the next frame */

src/core/fiber.h

@@ -46,7 +46,8 @@
 #define JANET_FIBER_MASK_USERN(N) (16 << (N))
 #define JANET_FIBER_MASK_USER 0x3FF0
 
-#define JANET_FIBER_STATUS_MASK 0xFF0000
+#define JANET_FIBER_RESUME_SIGNAL 0x800000
+#define JANET_FIBER_STATUS_MASK 0x7F0000
 #define JANET_FIBER_STATUS_OFFSET 16
 
 #define JANET_FIBER_BREAKPOINT       0x1000000

src/core/gc.c

@@ -39,6 +39,7 @@ struct JanetScratch {
 JANET_THREAD_LOCAL void *janet_vm_blocks;
 JANET_THREAD_LOCAL size_t janet_vm_gc_interval;
 JANET_THREAD_LOCAL size_t janet_vm_next_collection;
+JANET_THREAD_LOCAL size_t janet_vm_block_count;
 JANET_THREAD_LOCAL int janet_vm_gc_suspend = 0;
 
 /* Roots */
@@ -327,6 +328,7 @@ void janet_sweep() {
             previous = current;
             current->flags &= ~JANET_MEM_REACHABLE;
         } else {
+            janet_vm_block_count--;
             janet_deinit_block(current);
             if (NULL != previous) {
                 previous->next = next;
@@ -359,6 +361,7 @@ void *janet_gcalloc(enum JanetMemoryType type, size_t size) {
     janet_vm_next_collection += size;
     mem->next = janet_vm_blocks;
     janet_vm_blocks = mem;
+    janet_vm_block_count++;
 
     return (void *)mem;
 }
@@ -388,6 +391,14 @@ void janet_collect(void) {
     uint32_t i;
     if (janet_vm_gc_suspend) return;
     depth = JANET_RECURSION_GUARD;
+    /* Try and prevent many major collections back to back.
+     * A full collection will take O(janet_vm_block_count) time.
+     * If we have a large heap, make sure our interval is not too
+     * small so we won't make many collections over it. This is just a
+     * heuristic for automatically changing the gc interval */
+    if (janet_vm_block_count * 8 > janet_vm_gc_interval) {
+        janet_vm_gc_interval = janet_vm_block_count * sizeof(JanetGCObject);
+    }
     orig_rootcount = janet_vm_root_count;
 #ifdef JANET_NET
     janet_net_markloop();

src/core/inttypes.c

@@ -20,18 +20,18 @@
 * IN THE SOFTWARE.
 */
 
-#include <errno.h>
-#include <stdlib.h>
-#include <limits.h>
-#include <inttypes.h>
-#include <math.h>
-
 #ifndef JANET_AMALG
 #include "features.h"
 #include <janet.h>
 #include "util.h"
 #endif
 
+#include <errno.h>
+#include <stdlib.h>
+#include <limits.h>
+#include <inttypes.h>
+#include <math.h>
+
 /* Conditional compilation */
 #ifdef JANET_INT_TYPES
 
@@ -197,6 +197,122 @@ static Janet cfun_it_u64_new(int32_t argc, Janet *argv) {
     return janet_wrap_u64(janet_unwrap_u64(argv[0]));
 }
 
+// Code to support polymorphic comparison.
+//
+// int/u64 and int/s64 support a "compare" method that allows
+// comparison to each other, and to Janet numbers, using the
+// "compare" "compare<" ... functions.
+//
+// In the following code explicit casts are sometimes used to help
+// make it clear when int/float conversions are happening.
+//
+static int compare_double_double(double x, double y) {
+    return (x < y) ? -1 : ((x > y) ? 1 : 0);
+}
+
+static int compare_int64_double(int64_t x, double y) {
+    if (isnan(y)) {
+        return 0; // clojure and python do this
+    } else if ((y > (- ((double) MAX_INT_IN_DBL))) && (y < ((double) MAX_INT_IN_DBL))) {
+        double dx = (double) x;
+        return compare_double_double(dx, y);
+    } else if (y > ((double) INT64_MAX)) {
+        return -1;
+    } else if (y < ((double) INT64_MIN)) {
+        return 1;
+    } else {
+        int64_t yi = (int64_t) y;
+        return (x < yi) ? -1 : ((x > yi) ? 1 : 0);
+    }
+}
+
+static int compare_uint64_double(uint64_t x, double y) {
+    if (isnan(y)) {
+        return 0; // clojure and python do this
+    } else if (y < 0) {
+        return 1;
+    } else if ((y >= 0) && (y < ((double) MAX_INT_IN_DBL))) {
+        double dx = (double) x;
+        return compare_double_double(dx, y);
+    } else if (y > ((double) UINT64_MAX)) {
+        return -1;
+    } else {
+        uint64_t yi = (uint64_t) y;
+        return (x < yi) ? -1 : ((x > yi) ? 1 : 0);
+    }
+}
+
+
+static Janet cfun_it_s64_compare(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 2);
+    if (janet_is_int(argv[0]) != JANET_INT_S64)
+        janet_panic("compare method requires int/s64 as first argument");
+    int64_t x = janet_unwrap_s64(argv[0]);
+    switch (janet_type(argv[1])) {
+        default:
+            break;
+        case JANET_NUMBER : {
+            double y = janet_unwrap_number(argv[1]);
+            return janet_wrap_number(compare_int64_double(x, y));
+        }
+        case JANET_ABSTRACT: {
+            void *abst = janet_unwrap_abstract(argv[1]);
+            if (janet_abstract_type(abst) == &janet_s64_type) {
+                int64_t y = *(int64_t *)abst;
+                return janet_wrap_number((x < y) ? -1 : (x > y ? 1 : 0));
+            } else if (janet_abstract_type(abst) == &janet_u64_type) {
+                // comparing signed to unsigned -- be careful!
+                uint64_t y = *(uint64_t *)abst;
+                if (x < 0) {
+                    return janet_wrap_number(-1);
+                } else if (y > INT64_MAX) {
+                    return janet_wrap_number(-1);
+                } else {
+                    int64_t y2 = (int64_t) y;
+                    return janet_wrap_number((x < y2) ? -1 : (x > y2 ? 1 : 0));
+                }
+            }
+            break;
+        }
+    }
+    return janet_wrap_nil();
+}
+
+static Janet cfun_it_u64_compare(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 2);
+    if (janet_is_int(argv[0]) != JANET_INT_U64)  // is this needed?
+        janet_panic("compare method requires int/u64 as first argument");
+    uint64_t x = janet_unwrap_u64(argv[0]);
+    switch (janet_type(argv[1])) {
+        default:
+            break;
+        case JANET_NUMBER : {
+            double y = janet_unwrap_number(argv[1]);
+            return janet_wrap_number(compare_uint64_double(x, y));
+        }
+        case JANET_ABSTRACT: {
+            void *abst = janet_unwrap_abstract(argv[1]);
+            if (janet_abstract_type(abst) == &janet_u64_type) {
+                uint64_t y = *(uint64_t *)abst;
+                return janet_wrap_number((x < y) ? -1 : (x > y ? 1 : 0));
+            } else if (janet_abstract_type(abst) == &janet_s64_type) {
+                // comparing unsigned to signed -- be careful!
+                int64_t y = *(int64_t *)abst;
+                if (y < 0) {
+                    return janet_wrap_number(1);
+                } else if (x > INT64_MAX) {
+                    return janet_wrap_number(1);
+                } else {
+                    int64_t x2 = (int64_t) x;
+                    return janet_wrap_number((x2 < y) ? -1 : (x2 > y ? 1 : 0));
+                }
+            }
+            break;
+        }
+    }
+    return janet_wrap_nil();
+}
+
 #define OPMETHOD(T, type, name, oper) \
 static Janet cfun_it_##type##_##name(int32_t argc, Janet *argv) { \
     janet_arity(argc, 2, -1); \
@@ -266,14 +382,6 @@ static Janet cfun_it_##type##_##name(int32_t argc, Janet *argv) { \
     return janet_wrap_abstract(box); \
 } \
 
-#define COMPMETHOD(T, type, name, oper) \
-static Janet cfun_it_##type##_##name(int32_t argc, Janet *argv) { \
-    janet_fixarity(argc, 2); \
-    T v1 = janet_unwrap_##type(argv[0]); \
-    T v2 = janet_unwrap_##type(argv[1]); \
-    return janet_wrap_boolean(v1 oper v2); \
-}
-
 static Janet cfun_it_s64_mod(int32_t argc, Janet *argv) {
     janet_arity(argc, 2, -1);
     int64_t *box = janet_abstract(&janet_s64_type, sizeof(int64_t));
@@ -316,13 +424,6 @@ OPMETHOD(int64_t, s64, or, |)
 OPMETHOD(int64_t, s64, xor, ^)
 OPMETHOD(int64_t, s64, lshift, <<)
 OPMETHOD(int64_t, s64, rshift, >>)
-COMPMETHOD(int64_t, s64, lt, <)
-COMPMETHOD(int64_t, s64, gt, >)
-COMPMETHOD(int64_t, s64, le, <=)
-COMPMETHOD(int64_t, s64, ge, >=)
-COMPMETHOD(int64_t, s64, eq, ==)
-COMPMETHOD(int64_t, s64, ne, !=)
-
 OPMETHOD(uint64_t, u64, add, +)
 OPMETHOD(uint64_t, u64, sub, -)
 OPMETHODINVERT(uint64_t, u64, subi, -)
@@ -336,18 +437,13 @@ OPMETHOD(uint64_t, u64, or, |)
 OPMETHOD(uint64_t, u64, xor, ^)
 OPMETHOD(uint64_t, u64, lshift, <<)
 OPMETHOD(uint64_t, u64, rshift, >>)
-COMPMETHOD(uint64_t, u64, lt, <)
-COMPMETHOD(uint64_t, u64, gt, >)
-COMPMETHOD(uint64_t, u64, le, <=)
-COMPMETHOD(uint64_t, u64, ge, >=)
-COMPMETHOD(uint64_t, u64, eq, ==)
-COMPMETHOD(uint64_t, u64, ne, !=)
 
 #undef OPMETHOD
 #undef DIVMETHOD
 #undef DIVMETHOD_SIGNED
 #undef COMPMETHOD
 
+
 static JanetMethod it_s64_methods[] = {
     {"+", cfun_it_s64_add},
     {"r+", cfun_it_s64_add},
@@ -361,12 +457,6 @@ static JanetMethod it_s64_methods[] = {
     {"rmod", cfun_it_s64_modi},
     {"%", cfun_it_s64_rem},
     {"r%", cfun_it_s64_remi},
-    {"<", cfun_it_s64_lt},
-    {">", cfun_it_s64_gt},
-    {"<=", cfun_it_s64_le},
-    {">=", cfun_it_s64_ge},
-    {"=", cfun_it_s64_eq},
-    {"!=", cfun_it_s64_ne},
     {"&", cfun_it_s64_and},
     {"r&", cfun_it_s64_and},
     {"|", cfun_it_s64_or},
@@ -375,6 +465,7 @@ static JanetMethod it_s64_methods[] = {
     {"r^", cfun_it_s64_xor},
     {"<<", cfun_it_s64_lshift},
     {">>", cfun_it_s64_rshift},
+    {"compare", cfun_it_s64_compare},
 
     {NULL, NULL}
 };
@@ -392,12 +483,6 @@ static JanetMethod it_u64_methods[] = {
     {"rmod", cfun_it_u64_modi},
     {"%", cfun_it_u64_mod},
     {"r%", cfun_it_u64_modi},
-    {"<", cfun_it_u64_lt},
-    {">", cfun_it_u64_gt},
-    {"<=", cfun_it_u64_le},
-    {">=", cfun_it_u64_ge},
-    {"=", cfun_it_u64_eq},
-    {"!=", cfun_it_u64_ne},
     {"&", cfun_it_u64_and},
     {"r&", cfun_it_u64_and},
     {"|", cfun_it_u64_or},
@@ -406,6 +491,7 @@ static JanetMethod it_u64_methods[] = {
     {"r^", cfun_it_u64_xor},
     {"<<", cfun_it_u64_lshift},
     {">>", cfun_it_u64_rshift},
+    {"compare", cfun_it_u64_compare},
 
     {NULL, NULL}
 };

src/core/io.c

@@ -37,22 +37,27 @@
 
 static int cfun_io_gc(void *p, size_t len);
 static int io_file_get(void *p, Janet key, Janet *out);
+static void io_file_marshal(void *p, JanetMarshalContext *ctx);
+static void *io_file_unmarshal(JanetMarshalContext *ctx);
 
 const JanetAbstractType janet_file_type = {
     "core/file",
     cfun_io_gc,
     NULL,
     io_file_get,
-    JANET_ATEND_GET
+    NULL,
+    io_file_marshal,
+    io_file_unmarshal,
+    JANET_ATEND_UNMARSHAL
 };
 
 /* Check arguments to fopen */
-static int checkflags(const uint8_t *str) {
+static int32_t checkflags(const uint8_t *str) {
-    int flags = 0;
+    int32_t flags = 0;
     int32_t i;
     int32_t len = janet_string_length(str);
-    if (!len || len > 3)
+    if (!len || len > 10)
-        janet_panic("file mode must have a length between 1 and 3");
+        janet_panic("file mode must have a length between 1 and 10");
     switch (*str) {
         default:
             janet_panicf("invalid flag %c, expected w, a, or r", *str);
@@ -70,7 +75,7 @@ static int checkflags(const uint8_t *str) {
     for (i = 1; i < len; i++) {
         switch (str[i]) {
             default:
-                janet_panicf("invalid flag %c, expected + or b", str[i]);
+                janet_panicf("invalid flag %c, expected +, b, or n", str[i]);
                 break;
             case '+':
                 if (flags & JANET_FILE_UPDATE) return -1;
@@ -80,12 +85,16 @@ static int checkflags(const uint8_t *str) {
                 if (flags & JANET_FILE_BINARY) return -1;
                 flags |= JANET_FILE_BINARY;
                 break;
+            case 'n':
+                if (flags & JANET_FILE_NONIL) return -1;
+                flags |= JANET_FILE_NONIL;
+                break;
         }
     }
     return flags;
 }
 
-static Janet makef(FILE *f, int flags) {
+static void *makef(FILE *f, int32_t flags) {
     JanetFile *iof = (JanetFile *) janet_abstract(&janet_file_type, sizeof(JanetFile));
     iof->file = f;
     iof->flags = flags;
@@ -95,7 +104,7 @@ static Janet makef(FILE *f, int flags) {
     if (!(flags & JANET_FILE_NOT_CLOSEABLE))
         fcntl(fileno(f), F_SETFD, FD_CLOEXEC);
 #endif
-    return janet_wrap_abstract(iof);
+    return iof;
 }
 
 /* Open a process */
@@ -104,14 +113,14 @@ static Janet cfun_io_popen(int32_t argc, Janet *argv) {
     janet_arity(argc, 1, 2);
     const uint8_t *fname = janet_getstring(argv, 0);
     const uint8_t *fmode = NULL;
-    int flags;
+    int32_t flags;
     if (argc == 2) {
         fmode = janet_getkeyword(argv, 1);
-        if (janet_string_length(fmode) != 1 ||
+        flags = JANET_FILE_PIPED | checkflags(fmode);
-                !(fmode[0] == 'r' || fmode[0] == 'w')) {
+        if (flags & (JANET_FILE_UPDATE | JANET_FILE_BINARY | JANET_FILE_APPEND)) {
-            janet_panicf("invalid file mode :%S, expected :r or :w", fmode);
+            janet_panicf("invalid popen file mode :%S, expected :r or :w", fmode);
         }
-        flags = JANET_FILE_PIPED | (fmode[0] == 'r' ? JANET_FILE_READ : JANET_FILE_WRITE);
+        fmode = (const uint8_t *)((fmode[0] == 'r') ? "r" : "w");
     } else {
         fmode = (const uint8_t *)"r";
         flags = JANET_FILE_PIPED | JANET_FILE_READ;
@@ -121,9 +130,11 @@ static Janet cfun_io_popen(int32_t argc, Janet *argv) {
 #endif
     FILE *f = popen((const char *)fname, (const char *)fmode);
     if (!f) {
+        if (flags & JANET_FILE_NONIL)
+            janet_panicf("failed to popen %s: %s", fname, strerror(errno));
         return janet_wrap_nil();
     }
-    return makef(f, flags);
+    return janet_makefile(f, flags);
 }
 #endif
 
@@ -141,7 +152,7 @@ static Janet cfun_io_fopen(int32_t argc, Janet *argv) {
     janet_arity(argc, 1, 2);
     const uint8_t *fname = janet_getstring(argv, 0);
     const uint8_t *fmode;
-    int flags;
+    int32_t flags;
     if (argc == 2) {
         fmode = janet_getkeyword(argv, 1);
         flags = checkflags(fmode);
@@ -150,7 +161,9 @@ static Janet cfun_io_fopen(int32_t argc, Janet *argv) {
         flags = JANET_FILE_READ;
     }
     FILE *f = fopen((const char *)fname, (const char *)fmode);
-    return f ? makef(f, flags) : janet_wrap_nil();
+    return f ? janet_makefile(f, flags)
+           : (flags & JANET_FILE_NONIL) ? (janet_panicf("failed to open file %s: %s", fname, strerror(errno)), janet_wrap_nil())
+           : janet_wrap_nil();
 }
 
 /* Read up to n bytes into buffer. */
@@ -277,9 +290,14 @@ static Janet cfun_io_fclose(int32_t argc, Janet *argv) {
         iof->flags |= JANET_FILE_CLOSED;
         if (status == -1) janet_panic("could not close file");
         return janet_wrap_integer(WEXITSTATUS(status));
+#else
+        return janet_wrap_nil();
 #endif
     } else {
-        if (fclose(iof->file)) janet_panic("could not close file");
+        if (fclose(iof->file)) {
+            iof->flags |= JANET_FILE_NOT_CLOSEABLE;
+            janet_panic("could not close file");
+        }
         iof->flags |= JANET_FILE_CLOSED;
         return janet_wrap_nil();
     }
@@ -328,6 +346,50 @@ static int io_file_get(void *p, Janet key, Janet *out) {
     return janet_getmethod(janet_unwrap_keyword(key), io_file_methods, out);
 }
 
+static void io_file_marshal(void *p, JanetMarshalContext *ctx) {
+    JanetFile *iof = (JanetFile *)p;
+    if (ctx->flags & JANET_MARSHAL_UNSAFE) {
+        janet_marshal_abstract(ctx, p);
+#ifdef JANET_WINDOWS
+        janet_marshal_int(ctx, _fileno(iof->file));
+#else
+        janet_marshal_int(ctx, fileno(iof->file));
+#endif
+        janet_marshal_int(ctx, iof->flags);
+    } else {
+        janet_panic("cannot marshal file in safe mode");
+    }
+}
+
+static void *io_file_unmarshal(JanetMarshalContext *ctx) {
+    if (ctx->flags & JANET_MARSHAL_UNSAFE) {
+        JanetFile *iof = janet_unmarshal_abstract(ctx, sizeof(JanetFile));
+        int32_t fd = janet_unmarshal_int(ctx);
+        int32_t flags = janet_unmarshal_int(ctx);
+        char fmt[4] = {0};
+        int index = 0;
+        if (flags & JANET_FILE_READ) fmt[index++] = 'r';
+        if (flags & JANET_FILE_APPEND) {
+            fmt[index++] = 'a';
+        } else if (flags & JANET_FILE_WRITE) {
+            fmt[index++] = 'w';
+        }
+#ifdef JANET_WINDOWS
+        iof->file = _fdopen(fd, fmt);
+#else
+        iof->file = fdopen(fd, fmt);
+#endif
+        if (iof->file == NULL) {
+            iof->flags = JANET_FILE_CLOSED;
+        } else {
+            iof->flags = flags;
+        }
+        return iof;
+    } else {
+        janet_panic("cannot unmarshal file in safe mode");
+    }
+}
+
 FILE *janet_dynfile(const char *name, FILE *def) {
     Janet x = janet_dyn(name);
     if (!janet_checktype(x, JANET_ABSTRACT)) return def;
@@ -337,18 +399,16 @@ FILE *janet_dynfile(const char *name, FILE *def) {
     return iofile->file;
 }
 
-static Janet cfun_io_print_impl(int32_t argc, Janet *argv,
+static Janet cfun_io_print_impl_x(int32_t argc, Janet *argv, int newline,
-                                int newline, const char *name, FILE *dflt_file) {
+                                  FILE *dflt_file, int32_t offset, Janet x) {
     FILE *f;
-    Janet x = janet_dyn(name);
     switch (janet_type(x)) {
         default:
-            /* Other values simply do nothing */
+            janet_panicf("cannot print to %v", x);
-            return janet_wrap_nil();
         case JANET_BUFFER: {
             /* Special case buffer */
             JanetBuffer *buf = janet_unwrap_buffer(x);
-            for (int32_t i = 0; i < argc; ++i) {
+            for (int32_t i = offset; i < argc; ++i) {
                 janet_to_string_b(buf, argv[i]);
             }
             if (newline)
@@ -357,6 +417,7 @@ static Janet cfun_io_print_impl(int32_t argc, Janet *argv,
         }
         case JANET_NIL:
             f = dflt_file;
+            if (f == NULL) janet_panic("cannot print to nil");
             break;
         case JANET_ABSTRACT: {
             void *abstract = janet_unwrap_abstract(x);
@@ -367,7 +428,7 @@ static Janet cfun_io_print_impl(int32_t argc, Janet *argv,
             break;
         }
     }
-    for (int32_t i = 0; i < argc; ++i) {
+    for (int32_t i = offset; i < argc; ++i) {
         int32_t len;
         const uint8_t *vstr;
         if (janet_checktype(argv[i], JANET_BUFFER)) {
@@ -380,7 +441,11 @@ static Janet cfun_io_print_impl(int32_t argc, Janet *argv,
         }
         if (len) {
             if (1 != fwrite(vstr, len, 1, f)) {
-                janet_panicf("could not print %d bytes to (dyn :%s)", len, name);
+                if (f == dflt_file) {
+                    janet_panicf("cannot print %d bytes", len);
+                } else {
+                    janet_panicf("cannot print %d bytes to %v", len, x);
+                }
             }
         }
     }
@@ -389,6 +454,13 @@ static Janet cfun_io_print_impl(int32_t argc, Janet *argv,
     return janet_wrap_nil();
 }
 
+
+static Janet cfun_io_print_impl(int32_t argc, Janet *argv,
+                                int newline, const char *name, FILE *dflt_file) {
+    Janet x = janet_dyn(name);
+    return cfun_io_print_impl_x(argc, argv, newline, dflt_file, 0, x);
+}
+
 static Janet cfun_io_print(int32_t argc, Janet *argv) {
     return cfun_io_print_impl(argc, argv, 1, "out", stdout);
 }
@@ -405,25 +477,33 @@ static Janet cfun_io_eprin(int32_t argc, Janet *argv) {
     return cfun_io_print_impl(argc, argv, 0, "err", stderr);
 }
 
-static Janet cfun_io_printf_impl(int32_t argc, Janet *argv, int newline,
+static Janet cfun_io_xprint(int32_t argc, Janet *argv) {
-                                 const char *name, FILE *dflt_file) {
-    FILE *f;
     janet_arity(argc, 1, -1);
-    const char *fmt = janet_getcstring(argv, 0);
+    return cfun_io_print_impl_x(argc, argv, 1, NULL, 1, argv[0]);
-    Janet x = janet_dyn(name);
+}
+
+static Janet cfun_io_xprin(int32_t argc, Janet *argv) {
+    janet_arity(argc, 1, -1);
+    return cfun_io_print_impl_x(argc, argv, 0, NULL, 1, argv[0]);
+}
+
+static Janet cfun_io_printf_impl_x(int32_t argc, Janet *argv, int newline,
+                                   FILE *dflt_file, int32_t offset, Janet x) {
+    FILE *f;
+    const char *fmt = janet_getcstring(argv, offset);
     switch (janet_type(x)) {
         default:
-            /* Other values simply do nothing */
+            janet_panicf("cannot print to %v", x);
-            return janet_wrap_nil();
         case JANET_BUFFER: {
             /* Special case buffer */
             JanetBuffer *buf = janet_unwrap_buffer(x);
-            janet_buffer_format(buf, fmt, 0, argc, argv);
+            janet_buffer_format(buf, fmt, offset, argc, argv);
             if (newline) janet_buffer_push_u8(buf, '\n');
             return janet_wrap_nil();
         }
         case JANET_NIL:
             f = dflt_file;
+            if (f == NULL) janet_panic("cannot print to nil");
             break;
         case JANET_ABSTRACT: {
             void *abstract = janet_unwrap_abstract(x);
@@ -435,11 +515,11 @@ static Janet cfun_io_printf_impl(int32_t argc, Janet *argv, int newline,
         }
     }
     JanetBuffer *buf = janet_buffer(10);
-    janet_buffer_format(buf, fmt, 0, argc, argv);
+    janet_buffer_format(buf, fmt, offset, argc, argv);
     if (newline) janet_buffer_push_u8(buf, '\n');
     if (buf->count) {
         if (1 != fwrite(buf->data, buf->count, 1, f)) {
-            janet_panicf("could not print %d bytes to file", buf->count, name);
+            janet_panicf("could not print %d bytes to file", buf->count);
         }
     }
     /* Clear buffer to make things easier for GC */
@@ -450,6 +530,14 @@ static Janet cfun_io_printf_impl(int32_t argc, Janet *argv, int newline,
     return janet_wrap_nil();
 }
 
+static Janet cfun_io_printf_impl(int32_t argc, Janet *argv, int newline,
+                                 const char *name, FILE *dflt_file) {
+    janet_arity(argc, 1, -1);
+    Janet x = janet_dyn(name);
+    return cfun_io_printf_impl_x(argc, argv, newline, dflt_file, 0, x);
+
+}
+
 static Janet cfun_io_printf(int32_t argc, Janet *argv) {
     return cfun_io_printf_impl(argc, argv, 1, "out", stdout);
 }
@@ -466,6 +554,16 @@ static Janet cfun_io_eprinf(int32_t argc, Janet *argv) {
     return cfun_io_printf_impl(argc, argv, 0, "err", stderr);
 }
 
+static Janet cfun_io_xprintf(int32_t argc, Janet *argv) {
+    janet_arity(argc, 2, -1);
+    return cfun_io_printf_impl_x(argc, argv, 1, NULL, 1, argv[0]);
+}
+
+static Janet cfun_io_xprinf(int32_t argc, Janet *argv) {
+    janet_arity(argc, 2, -1);
+    return cfun_io_printf_impl_x(argc, argv, 0, NULL, 1, argv[0]);
+}
+
 static void janet_flusher(const char *name, FILE *dflt_file) {
     Janet x = janet_dyn(name);
     switch (janet_type(x)) {
@@ -579,6 +677,29 @@ static const JanetReg io_cfuns[] = {
         JDOC("(eprinf fmt & xs)\n\n"
              "Like eprintf but with no trailing newline.")
     },
+    {
+        "xprint", cfun_io_xprint,
+        JDOC("(xprint to & xs)\n\n"
+             "Print to a file or other value explicitly (no dynamic bindings) with a trailing "
+             "newline character. The value to print "
+             "to is the first argument, and is otherwise the same as print. Returns nil.")
+    },
+    {
+        "xprin", cfun_io_xprin,
+        JDOC("(xprin to & xs)\n\n"
+             "Print to a file or other value explicitly (no dynamic bindings). The value to print "
+             "to is the first argument, and is otherwise the same as prin. Returns nil.")
+    },
+    {
+        "xprintf", cfun_io_xprintf,
+        JDOC("(xprint to fmt & xs)\n\n"
+             "Like printf but prints to an explicit file or value to. Returns nil.")
+    },
+    {
+        "xprinf", cfun_io_xprinf,
+        JDOC("(xprin to fmt & xs)\n\n"
+             "Like prinf but prints to an explicit file or value to. Returns nil.")
+    },
     {
         "flush", cfun_io_flush,
         JDOC("(flush)\n\n"
@@ -607,7 +728,8 @@ static const JanetReg io_cfuns[] = {
              "\tw - allow writing to the file\n"
              "\ta - append to the file\n"
              "\tb - open the file in binary mode (rather than text mode)\n"
-             "\t+ - append to the file instead of overwriting it")
+             "\t+ - append to the file instead of overwriting it\n"
+             "\tn - error if the file cannot be opened instead of returning nil")
     },
     {
         "file/close", cfun_io_fclose,
@@ -667,6 +789,10 @@ static const JanetReg io_cfuns[] = {
 
 /* C API */
 
+JanetFile *janet_getjfile(const Janet *argv, int32_t n) {
+    return janet_getabstract(argv, n, &janet_file_type);
+}
+
 FILE *janet_getfile(const Janet *argv, int32_t n, int *flags) {
     JanetFile *iof = janet_getabstract(argv, n, &janet_file_type);
     if (NULL != flags) *flags = iof->flags;
@@ -674,7 +800,7 @@ FILE *janet_getfile(const Janet *argv, int32_t n, int *flags) {
 }
 
 Janet janet_makefile(FILE *f, int flags) {
-    return makef(f, flags);
+    return janet_wrap_abstract(makef(f, flags));
 }
 
 JanetAbstract janet_checkfile(Janet j) {
@@ -690,18 +816,19 @@ FILE *janet_unwrapfile(Janet j, int *flags) {
 /* Module entry point */
 void janet_lib_io(JanetTable *env) {
     janet_core_cfuns(env, NULL, io_cfuns);
-
+    janet_register_abstract_type(&janet_file_type);
+    int default_flags = JANET_FILE_NOT_CLOSEABLE | JANET_FILE_SERIALIZABLE;
     /* stdout */
     janet_core_def(env, "stdout",
-                   makef(stdout, JANET_FILE_APPEND | JANET_FILE_NOT_CLOSEABLE | JANET_FILE_SERIALIZABLE),
+                   janet_makefile(stdout, JANET_FILE_APPEND | default_flags),
                    JDOC("The standard output file."));
     /* stderr */
     janet_core_def(env, "stderr",
-                   makef(stderr, JANET_FILE_APPEND | JANET_FILE_NOT_CLOSEABLE | JANET_FILE_SERIALIZABLE),
+                   janet_makefile(stderr, JANET_FILE_APPEND | default_flags),
                    JDOC("The standard error file."));
     /* stdin */
     janet_core_def(env, "stdin",
-                   makef(stdin, JANET_FILE_READ | JANET_FILE_NOT_CLOSEABLE | JANET_FILE_SERIALIZABLE),
+                   janet_makefile(stdin, JANET_FILE_READ | default_flags),
                    JDOC("The standard input file."));
 
 }

src/core/marsh.c

@@ -214,15 +214,6 @@ static void marshal_one_env(MarshalState *st, JanetFuncEnv *env, int flags) {
     }
 }
 
-/* Add function flags to janet functions */
-static void janet_func_addflags(JanetFuncDef *def) {
-    if (def->name) def->flags |= JANET_FUNCDEF_FLAG_HASNAME;
-    if (def->source) def->flags |= JANET_FUNCDEF_FLAG_HASSOURCE;
-    if (def->defs) def->flags |= JANET_FUNCDEF_FLAG_HASDEFS;
-    if (def->environments) def->flags |= JANET_FUNCDEF_FLAG_HASENVS;
-    if (def->sourcemap) def->flags |= JANET_FUNCDEF_FLAG_HASSOURCEMAP;
-}
-
 /* Marshal a sequence of u32s */
 static void janet_marshal_u32s(MarshalState *st, const uint32_t *u32s, int32_t n) {
     for (int32_t i = 0; i < n; i++) {
@@ -243,7 +234,6 @@ static void marshal_one_def(MarshalState *st, JanetFuncDef *def, int flags) {
             return;
         }
     }
-    janet_func_addflags(def);
     /* Add to lookup */
     janet_v_push(st->seen_defs, def);
     pushint(st, def->flags);
@@ -900,7 +890,7 @@ static const uint8_t *unmarshal_one_def(
             for (int32_t i = 0; i < bytecode_length; i++) {
                 current += readint(st, &data);
                 def->sourcemap[i].line = current;
-                def->sourcemap[i].column = readnat(st, &data);
+                def->sourcemap[i].column = readint(st, &data);
             }
         } else {
             def->sourcemap = NULL;
@@ -908,11 +898,12 @@ static const uint8_t *unmarshal_one_def(
 
         /* Unmarshal closure bitset if needed */
         if (def->flags & JANET_FUNCDEF_FLAG_HASCLOBITSET) {
-            def->closure_bitset = malloc(sizeof(uint32_t) * def->slotcount);
+            int32_t n = (def->slotcount + 31) >> 5;
+            def->closure_bitset = malloc(sizeof(uint32_t) * (size_t) n);
             if (NULL == def->closure_bitset) {
                 JANET_OUT_OF_MEMORY;
             }
-            data = janet_unmarshal_u32s(st, data, def->closure_bitset, (def->slotcount + 31) >> 5);
+            data = janet_unmarshal_u32s(st, data, def->closure_bitset, n);
         }
 
         /* Validate */

src/core/math.c

@@ -499,5 +499,19 @@ void janet_lib_math(JanetTable *env) {
               JDOC("The number representing positive infinity"));
     janet_def(env, "math/-inf", janet_wrap_number(-INFINITY),
               JDOC("The number representing negative infinity"));
+    janet_def(env, "math/int32-min", janet_wrap_number(INT32_MIN),
+              JDOC("The maximum contiguous integer representable by a 32 bit signed integer"));
+    janet_def(env, "math/int32-max", janet_wrap_number(INT32_MAX),
+              JDOC("The minimum contiguous integer represtenable by a 32 bit signed integer"));
+    janet_def(env, "math/int-min", janet_wrap_number(JANET_INTMIN_DOUBLE),
+              JDOC("The maximum contiguous integer representable by a double (2^53)"));
+    janet_def(env, "math/int-max", janet_wrap_number(JANET_INTMAX_DOUBLE),
+              JDOC("The minimum contiguous integer represtenable by a double (-(2^53))"));
+#ifdef NAN
+    janet_def(env, "math/nan", janet_wrap_number(NAN),
+#else
+    janet_def(env, "math/nan", janet_wrap_number(0.0 / 0.0),
+#endif
+              JDOC("Not a number (IEEE-754 NaN)"));
 #endif
 }

src/core/net.c

@@ -26,9 +26,11 @@
 #include "util.h"
 #endif
 
+#ifdef JANET_NET
+
 #ifdef JANET_WINDOWS
-#include <windows.h>
 #include <winsock2.h>
+#include <windows.h>
 #include <ws2tcpip.h>
 #pragma comment (lib, "Ws2_32.lib")
 #pragma comment (lib, "Mswsock.lib")
@@ -110,7 +112,7 @@ typedef struct {
 #endif
 static JanetStream *make_stream(int fd, int flags) {
     JanetStream *stream = janet_abstract(&StreamAT, sizeof(JanetStream));
-#ifndef SOCK_CLOEXEC
+#if !defined(SOCK_CLOEXEC) && defined(O_CLOEXEC)
     int extra = O_CLOEXEC;
 #else
     int extra = 0;
@@ -137,6 +139,19 @@ static int janet_stream_close(void *p, size_t s) {
     return 0;
 }
 
+
+static void nosigpipe(JSock s) {
+#ifdef SO_NOSIGPIPE
+    int enable = 1;
+    if (setsockopt(s, SOL_SOCKET, SO_NOSIGPIPE, &enable, sizeof(int)) < 0) {
+        JSOCKCLOSE(s);
+        janet_panic("setsockopt(SO_NOSIGPIPE) failed");
+    }
+#else
+    (void) s;
+#endif
+}
+
 /*
  * Event loop
  */
@@ -260,8 +275,11 @@ static size_t janet_loop_event(size_t index) {
     int ret = 1;
     int should_resume = 0;
     Janet resumeval = janet_wrap_nil();
+    JanetSignal resumesignal = JANET_SIGNAL_OK;
     if (stream->flags & JANET_STREAM_CLOSED) {
         should_resume = 1;
+        resumeval = janet_cstringv("stream is closed");
+        resumesignal = JANET_SIGNAL_ERROR;
         ret = 0;
     } else {
         switch (jlfd->event_type) {
@@ -273,6 +291,8 @@ static size_t janet_loop_event(size_t index) {
                 if (!(stream->flags & JANET_STREAM_READABLE)) {
                     should_resume = 1;
                     ret = 0;
+                    resumesignal = JANET_SIGNAL_ERROR;
+                    resumeval = janet_cstringv("stream not readable");
                     break;
                 }
                 JReadInt nread;
@@ -294,6 +314,13 @@ static size_t janet_loop_event(size_t index) {
                     should_resume = 1;
                     if (nread > 0) {
                         resumeval = janet_wrap_buffer(buffer);
+                    } else {
+                        if (nread == 0) {
+                            resumeval = janet_cstringv("could not read from stream");
+                        } else {
+                            resumeval = janet_cstringv(strerror(JLASTERR));
+                        }
+                        resumesignal = JANET_SIGNAL_ERROR;
                     }
                     ret = 0;
                 } else {
@@ -306,6 +333,7 @@ static size_t janet_loop_event(size_t index) {
                 JSock connfd = accept(fd, NULL, NULL);
                 if (JSOCKVALID(connfd)) {
                     /* Made a new connection socket */
+                    nosigpipe(connfd);
                     JanetStream *stream = make_stream(connfd, JANET_STREAM_READABLE | JANET_STREAM_WRITABLE);
                     Janet streamv = janet_wrap_abstract(stream);
                     JanetFunction *handler = jlfd->data.read_accept.handler;
@@ -326,6 +354,8 @@ static size_t janet_loop_event(size_t index) {
                 const uint8_t *bytes;
                 if (!(stream->flags & JANET_STREAM_WRITABLE)) {
                     should_resume = 1;
+                    resumesignal = JANET_SIGNAL_ERROR;
+                    resumeval = janet_cstringv("stream not writeable");
                     ret = 0;
                     break;
                 }
@@ -348,6 +378,12 @@ static size_t janet_loop_event(size_t index) {
                     if (nwrote > 0) {
                         start += nwrote;
                     } else {
+                        resumesignal = JANET_SIGNAL_ERROR;
+                        if (nwrote == -1) {
+                            resumeval = janet_cstringv(strerror(JLASTERR));
+                        } else {
+                            resumeval = janet_cstringv("could not write");
+                        }
                         start = len;
                     }
                 }
@@ -374,7 +410,7 @@ static size_t janet_loop_event(size_t index) {
     if (NULL != jlfd->fiber && should_resume) {
         /* Resume the fiber */
         Janet out;
-        JanetSignal sig = janet_continue(jlfd->fiber, resumeval, &out);
+        JanetSignal sig = janet_continue_signal(jlfd->fiber, resumeval, &out, resumesignal);
         if (sig != JANET_SIGNAL_OK && sig != JANET_SIGNAL_EVENT) {
             janet_stacktrace(jlfd->fiber, out);
         }
@@ -509,6 +545,8 @@ static Janet cfun_net_connect(int32_t argc, Janet *argv) {
         janet_panic("could not connect to socket");
     }
 
+    nosigpipe(sock);
+
     /* Wrap socket in abstract type JanetStream */
     JanetStream *stream = make_stream(sock, JANET_STREAM_READABLE | JANET_STREAM_WRITABLE);
     return janet_wrap_abstract(stream);
@@ -534,12 +572,7 @@ static Janet cfun_net_server(int32_t argc, Janet *argv) {
             JSOCKCLOSE(sfd);
             janet_panic("setsockopt(SO_REUSEADDR) failed");
         }
-#ifdef SO_NOSIGPIPE
+        nosigpipe(sfd);
-        if (setsockopt(sfd, SOL_SOCKET, SO_NOSIGPIPE, &enable, sizeof(int)) < 0) {
-            JSOCKCLOSE(sfd);
-            janet_panic("setsockopt(SO_NOSIGPIPE) failed");
-        }
-#endif
 #ifdef SO_REUSEPORT
         if (setsockopt(sfd, SOL_SOCKET, SO_REUSEPORT, &enable, sizeof(int)) < 0) {
             JSOCKCLOSE(sfd);
@@ -564,7 +597,8 @@ static Janet cfun_net_server(int32_t argc, Janet *argv) {
     }
 
     /* Put sfd on our loop */
-    JanetLoopFD lfd = {0};
+    JanetLoopFD lfd;
+    memset(&lfd, 0, sizeof(lfd));
     lfd.stream = make_stream(sfd, 0);
     lfd.event_type = JLE_READ_ACCEPT;
     lfd.data.read_accept.handler = fun;
@@ -634,7 +668,7 @@ static const JanetReg net_cfuns[] = {
         JDOC("(net/read stream nbytes &opt buf)\n\n"
              "Read up to n bytes from a stream, suspending the current fiber until the bytes are available. "
              "If less than n bytes are available (and more than 0), will push those bytes and return early. "
-             "Returns a buffer with up to n more bytes in it.")
+             "Returns a buffer with up to n more bytes in it, or raises an error if the read failed.")
     },
     {
         "net/chunk", cfun_stream_chunk,
@@ -645,7 +679,7 @@ static const JanetReg net_cfuns[] = {
         "net/write", cfun_stream_write,
         JDOC("(net/write stream data)\n\n"
              "Write data to a stream, suspending the current fiber until the write "
-             "completes. Returns stream.")
+             "completes. Returns nil, or raises an error if the write failed.")
     },
     {
         "net/close", cfun_stream_close,
@@ -675,3 +709,5 @@ void janet_net_deinit(void) {
     WSACleanup();
 #endif
 }
+
+#endif

src/core/os.c

@@ -24,6 +24,7 @@
 #include "features.h"
 #include <janet.h>
 #include "util.h"
+#include "gc.h"
 #endif
 
 #ifndef JANET_REDUCED_OS
@@ -36,8 +37,11 @@
 #include <stdio.h>
 #include <string.h>
 #include <sys/stat.h>
+#include <signal.h>
 
-#define RETRY_EINTR(RC, CALL) do { (RC) = CALL; } while((RC) < 0 && errno == EINTR)
+#ifdef JANET_APPLE
+#include <AvailabilityMacros.h>
+#endif
 
 #ifdef JANET_WINDOWS
 #include <windows.h>
@@ -64,12 +68,6 @@ extern char **environ;
 #include <mach/mach.h>
 #endif
 
-/* Setting C99 standard makes this not available, but it should
- * work/link properly if we detect a BSD */
-#if defined(JANET_BSD) || defined(JANET_APPLE)
-void arc4random_buf(void *buf, size_t nbytes);
-#endif
-
 /* Not POSIX, but all Unixes but Solaris have this function. */
 #if defined(JANET_POSIX) && !defined(__sun)
 time_t timegm(struct tm *tm);
@@ -159,6 +157,8 @@ static Janet os_arch(int32_t argc, Janet *argv) {
     return janet_ckeywordv("arm");
 #elif (defined(__sparc__))
     return janet_ckeywordv("sparc");
+#elif (defined(__ppc__))
+    return janet_ckeywordv("ppc");
 #else
     return janet_ckeywordv("unknown");
 #endif
@@ -314,13 +314,127 @@ static JanetBuffer *os_exec_escape(JanetView args) {
 }
 #endif
 
+/* Process type for when running a subprocess and not immediately waiting */
+static const JanetAbstractType ProcAT;
+#define JANET_PROC_CLOSED 1
+#define JANET_PROC_WAITED 2
+typedef struct {
+    int flags;
+#ifdef JANET_WINDOWS
+    HANDLE pHandle;
+	HANDLE tHandle;
+#else
+    int pid;
+#endif
+    int return_code;
+    JanetFile *in;
+    JanetFile *out;
+    JanetFile *err;
+} JanetProc;
+
+static int janet_proc_mark(void *p, size_t s) {
+    (void) s;
+    JanetProc *proc = (JanetProc *)p;
+    if (NULL != proc->in) janet_mark(janet_wrap_abstract(proc->in));
+    if (NULL != proc->out) janet_mark(janet_wrap_abstract(proc->out));
+    if (NULL != proc->err) janet_mark(janet_wrap_abstract(proc->err));
+    return 0;
+}
+
+static Janet os_proc_wait_impl(JanetProc *proc) {
+    if (proc->flags & JANET_PROC_WAITED) {
+        janet_panicf("cannot wait on process that has already finished");
+    }
+    proc->flags |= JANET_PROC_WAITED;
+    int status = 0;
+#ifdef JANET_WINDOWS
+    WaitForSingleObject(proc->pHandle, INFINITE);
+    GetExitCodeProcess(proc->pHandle, &status);
+	if (!(proc->flags & JANET_PROC_CLOSED)) {
+		proc->flags |= JANET_PROC_CLOSED;
+		CloseHandle(proc->pHandle);
+		CloseHandle(proc->tHandle);
+    }
+#else
+    waitpid(proc->pid, &status, 0);
+#endif
+    proc->return_code = (int32_t) status;
+    return janet_wrap_integer(proc->return_code);
+}
+
+static Janet os_proc_wait(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    JanetProc *proc = janet_getabstract(argv, 0, &ProcAT);
+    return os_proc_wait_impl(proc);
+}
+
+static Janet os_proc_kill(int32_t argc, Janet *argv) {
+    janet_arity(argc, 1, 2);
+    JanetProc *proc = janet_getabstract(argv, 0, &ProcAT);
+#ifdef JANET_WINDOWS
+    if (proc->flags & JANET_PROC_CLOSED) {
+        janet_panicf("cannot close process handle that is already closed");
+    }
+    proc->flags |= JANET_PROC_CLOSED;
+	CloseHandle(proc->pHandle);
+	CloseHandle(proc->tHandle);
+#else
+    int status = kill(proc->pid, SIGKILL);
+    if (status) {
+        janet_panic(strerror(errno));
+    }
+#endif
+    /* After killing process we wait on it. */
+    if (argc > 1 && janet_truthy(argv[1])) {
+        return os_proc_wait_impl(proc);
+    } else {
+        return argv[0];
+    }
+}
+
+static const JanetMethod proc_methods[] = {
+    {"wait", os_proc_wait},
+    {"kill", os_proc_kill},
+    {NULL, NULL}
+};
+
+static int janet_proc_get(void *p, Janet key, Janet *out) {
+    JanetProc *proc = (JanetProc *)p;
+    if (janet_keyeq(key, "in")) {
+        *out = (NULL == proc->in) ? janet_wrap_nil() : janet_wrap_abstract(proc->in);
+        return 1;
+    }
+    if (janet_keyeq(key, "out")) {
+        *out = (NULL == proc->out) ? janet_wrap_nil() : janet_wrap_abstract(proc->out);
+        return 1;
+    }
+    if (janet_keyeq(key, "err")) {
+        *out = (NULL == proc->out) ? janet_wrap_nil() : janet_wrap_abstract(proc->err);
+        return 1;
+    }
+    if ((-1 != proc->return_code) && janet_keyeq(key, "return-code")) {
+        *out = janet_wrap_integer(proc->return_code);
+        return 1;
+    }
+    if (!janet_checktype(key, JANET_KEYWORD)) return 0;
+    return janet_getmethod(janet_unwrap_keyword(key), proc_methods, out);
+}
+
+static const JanetAbstractType ProcAT = {
+    "core/process",
+    NULL,
+    janet_proc_mark,
+    janet_proc_get,
+    JANET_ATEND_GET
+};
+
 static Janet os_execute(int32_t argc, Janet *argv) {
     janet_arity(argc, 1, 3);
 
     /* Get flags */
     uint64_t flags = 0;
     if (argc > 1) {
-        flags = janet_getflags(argv, 1, "ep");
+        flags = janet_getflags(argv, 1, "epxa");
     }
 
     /* Get environment */
@@ -332,43 +446,77 @@ static Janet os_execute(int32_t argc, Janet *argv) {
         janet_panic("expected at least 1 command line argument");
     }
 
+    /* Optional stdio redirections */
+    JanetFile *new_in = NULL, *new_out = NULL, *new_err = NULL;
+
+    /* Get optional redirections */
+    if (argc > 2) {
+        JanetDictView tab = janet_getdictionary(argv, 2);
+        Janet maybe_stdin = janet_dictionary_get(tab.kvs, tab.cap, janet_ckeywordv("in"));
+        Janet maybe_stdout = janet_dictionary_get(tab.kvs, tab.cap, janet_ckeywordv("out"));
+        Janet maybe_stderr = janet_dictionary_get(tab.kvs, tab.cap, janet_ckeywordv("err"));
+        if (!janet_checktype(maybe_stdin, JANET_NIL)) new_in = janet_getjfile(&maybe_stdin, 0);
+        if (!janet_checktype(maybe_stdout, JANET_NIL)) new_out = janet_getjfile(&maybe_stdout, 0);
+        if (!janet_checktype(maybe_stderr, JANET_NIL)) new_err = janet_getjfile(&maybe_stderr, 0);
+    }
+
     /* Result */
     int status = 0;
+    int is_async = janet_flag_at(flags, 3);
 
 #ifdef JANET_WINDOWS
 
+    HANDLE pHandle, tHandle;
+	PROCESS_INFORMATION processInfo;
+	STARTUPINFO startupInfo;
+	memset(&processInfo, 0, sizeof(processInfo));
+	memset(&startupInfo, 0, sizeof(startupInfo));
+	startupInfo.cb = sizeof(startupInfo);
+	startupInfo.dwFlags |= STARTF_USESTDHANDLES;
+
     JanetBuffer *buf = os_exec_escape(exargs);
     if (buf->count > 8191) {
-        janet_panic("command line string too long");
+        janet_panic("command line string too long (max 8191 characters)");
     }
     const char *path = (const char *) janet_unwrap_string(exargs.items[0]);
-    char *cargv[2] = {(char *) buf->data, NULL};
+
+	/* Do IO redirection */
+	startupInfo.hStdInput = (HANDLE) _get_osfhandle((new_in == NULL) ? 0 : _fileno(new_in->file));
+	startupInfo.hStdOutput = (HANDLE) _get_osfhandle((new_out == NULL) ? 1 : _fileno(new_out->file));
+	startupInfo.hStdError = (HANDLE) _get_osfhandle((new_err == NULL) ? 2 : _fileno(new_err->file));
 
     /* Use _spawn family of functions. */
     /* Windows docs say do this before any spawns. */
     _flushall();
 
-    /* Use an empty env instead when envp is NULL to be consistent with other implementation. */
+	/* TODO - redirection, :p flag */
-    char *empty_env[1] = {NULL};
+	if(!CreateProcess(janet_flag_at(flags, 1) ? NULL : path, /* NULL? */
-    char **envp1 = (NULL == envp) ? empty_env : envp;
+		(char *) buf->data, /* Single CLI argument */
-
+        NULL, /* no proc inheritance */
-    if (janet_flag_at(flags, 1) && janet_flag_at(flags, 0)) {
+        NULL, /* no thread inheritance */
-        status = (int) _spawnvpe(_P_WAIT, path, cargv, envp1);
+        TRUE, /* handle inheritance */
-    } else if (janet_flag_at(flags, 1)) {
+        0, /* flags */
-        status = (int) _spawnvp(_P_WAIT, path, cargv);
+        envp, /* pass in environment */
-    } else if (janet_flag_at(flags, 0)) {
+        NULL, /* use parents starting directory */
-        status = (int) _spawnve(_P_WAIT, path, cargv, envp1);
+        &startupInfo,
-    } else {
+        &processInfo))  {
-        status = (int) _spawnv(_P_WAIT, path, cargv);
+        janet_panic("failed to create process");
     }
+	
+	pHandle = processInfo.hProcess;
+	tHandle = processInfo.hThread;
+
     os_execute_cleanup(envp, NULL);
 
-    /* Check error */
+	/* Wait and cleanup immedaitely */
-    if (-1 == status) {
+    if (!is_async) {
-        janet_panicf("%p: %s", argv[0], strerror(errno));
+		DWORD code;
+		WaitForSingleObject(pHandle, INFINITE);
+		GetExitCodeProcess(pHandle, &code);
+        status = (int) code;
+		CloseHandle(pHandle);
+		CloseHandle(tHandle);
     }
-
-    return janet_wrap_integer(status);
 #else
 
     const char **child_argv = janet_smalloc(sizeof(char *) * ((size_t) exargs.len + 1));
@@ -387,17 +535,32 @@ static Janet os_execute(int32_t argc, Janet *argv) {
         janet_lock_environ();
     }
 
+    /* Posix spawn setup */
+    posix_spawn_file_actions_t actions;
+    posix_spawn_file_actions_init(&actions);
+    if (new_in != NULL) {
+        posix_spawn_file_actions_adddup2(&actions, fileno(new_in->file), 0);
+    }
+    if (new_out != NULL) {
+        posix_spawn_file_actions_adddup2(&actions, fileno(new_out->file), 1);
+    }
+    if (new_err != NULL) {
+        posix_spawn_file_actions_adddup2(&actions, fileno(new_err->file), 2);
+    }
+
     pid_t pid;
     if (janet_flag_at(flags, 1)) {
         status = posix_spawnp(&pid,
-                              child_argv[0], NULL, NULL, cargv,
+                              child_argv[0], &actions, NULL, cargv,
                               use_environ ? environ : envp);
     } else {
         status = posix_spawn(&pid,
-                             child_argv[0], NULL, NULL, cargv,
+                             child_argv[0], &actions, NULL, cargv,
                              use_environ ? environ : envp);
     }
 
+    posix_spawn_file_actions_destroy(&actions);
+
     if (use_environ) {
         janet_unlock_environ();
     }
@@ -406,22 +569,43 @@ static Janet os_execute(int32_t argc, Janet *argv) {
     if (status) {
         os_execute_cleanup(envp, child_argv);
         janet_panicf("%p: %s", argv[0], strerror(errno));
+    } else if (janet_flag_at(flags, 3)) {
+        /* Get process handle */
+        os_execute_cleanup(envp, child_argv);
     } else {
+        /* Wait to complete */
         waitpid(pid, &status, 0);
+        os_execute_cleanup(envp, child_argv);
+        /* Use POSIX shell semantics for interpreting signals */
+        if (WIFEXITED(status)) {
+            status = WEXITSTATUS(status);
+        } else if (WIFSTOPPED(status)) {
+            status = WSTOPSIG(status) + 128;
+        } else {
+            status = WTERMSIG(status) + 128;
+        }
     }
 
-    os_execute_cleanup(envp, child_argv);
-    /* Use POSIX shell semantics for interpreting signals */
-    int ret;
-    if (WIFEXITED(status)) {
-        ret = WEXITSTATUS(status);
-    } else if (WIFSTOPPED(status)) {
-        ret = WSTOPSIG(status) + 128;
-    } else {
-        ret = WTERMSIG(status) + 128;
-    }
-    return janet_wrap_integer(ret);
 #endif
+    if (is_async) {
+        JanetProc *proc = janet_abstract(&ProcAT, sizeof(JanetProc));
+        proc->return_code = -1;
+#ifdef JANET_WINDOWS
+		proc->pHandle = pHandle;
+		proc->tHandle = tHandle;
+#else
+	    proc->pid = pid;
+#endif
+        proc->in = new_in;
+        proc->out = new_out;
+        proc->err = new_err;
+        proc->flags = 0;
+        return janet_wrap_abstract(proc);
+    } else if (janet_flag_at(flags, 2) && status) {
+        janet_panicf("command failed with non-zero exit code %d", status);
+    } else {
+        return janet_wrap_integer(status);
+    }
 }
 
 static Janet os_shell(int32_t argc, Janet *argv) {
@@ -508,39 +692,11 @@ static Janet os_time(int32_t argc, Janet *argv) {
     return janet_wrap_number(dtime);
 }
 
-/* Clock shims */
-#ifdef JANET_WINDOWS
-static int gettime(struct timespec *spec) {
-    FILETIME ftime;
-    GetSystemTimeAsFileTime(&ftime);
-    int64_t wintime = (int64_t)(ftime.dwLowDateTime) | ((int64_t)(ftime.dwHighDateTime) << 32);
-    /* Windows epoch is January 1, 1601 apparently */
-    wintime -= 116444736000000000LL;
-    spec->tv_sec  = wintime / 10000000LL;
-    /* Resolution is 100 nanoseconds. */
-    spec->tv_nsec = wintime % 10000000LL * 100;
-    return 0;
-}
-#elif defined(__MACH__)
-static int gettime(struct timespec *spec) {
-    clock_serv_t cclock;
-    mach_timespec_t mts;
-    host_get_clock_service(mach_host_self(), CALENDAR_CLOCK, &cclock);
-    clock_get_time(cclock, &mts);
-    mach_port_deallocate(mach_task_self(), cclock);
-    spec->tv_sec = mts.tv_sec;
-    spec->tv_nsec = mts.tv_nsec;
-    return 0;
-}
-#else
-#define gettime(TV) clock_gettime(CLOCK_MONOTONIC, (TV))
-#endif
-
 static Janet os_clock(int32_t argc, Janet *argv) {
     janet_fixarity(argc, 0);
     (void) argv;
     struct timespec tv;
-    if (gettime(&tv)) janet_panic("could not get time");
+    if (janet_gettime(&tv)) janet_panic("could not get time");
     double dtime = tv.tv_sec + (tv.tv_nsec / 1E9);
     return janet_wrap_number(dtime);
 }
@@ -579,7 +735,6 @@ static Janet os_cwd(int32_t argc, Janet *argv) {
 
 static Janet os_cryptorand(int32_t argc, Janet *argv) {
     JanetBuffer *buffer;
-    const char *genericerr = "unable to get sufficient random data";
     janet_arity(argc, 1, 2);
     int32_t offset;
     int32_t n = janet_getinteger(argv, 0);
@@ -594,43 +749,9 @@ static Janet os_cryptorand(int32_t argc, Janet *argv) {
     /* We could optimize here by adding setcount_uninit */
     janet_buffer_setcount(buffer, offset + n);
 
-#ifdef JANET_WINDOWS
+    if (janet_cryptorand(buffer->data + offset, n) != 0)
-    for (int32_t i = offset; i < buffer->count; i += sizeof(unsigned int)) {
+        janet_panic("unable to get sufficient random data");
-        unsigned int v;
+
-        if (rand_s(&v))
-            janet_panic(genericerr);
-        for (int32_t j = 0; (j < sizeof(unsigned int)) && (i + j < buffer->count); j++) {
-            buffer->data[i + j] = v & 0xff;
-            v = v >> 8;
-        }
-    }
-#elif defined(JANET_LINUX)
-    /* We should be able to call getrandom on linux, but it doesn't seem
-       to be uniformly supported on linux distros.
-       In both cases, use this fallback path for now... */
-    int rc;
-    int randfd;
-    RETRY_EINTR(randfd, open("/dev/urandom", O_RDONLY | O_CLOEXEC));
-    if (randfd < 0)
-        janet_panic(genericerr);
-    while (n > 0) {
-        ssize_t nread;
-        RETRY_EINTR(nread, read(randfd, buffer->data + offset, n));
-        if (nread <= 0) {
-            RETRY_EINTR(rc, close(randfd));
-            janet_panic(genericerr);
-        }
-        offset += nread;
-        n -= nread;
-    }
-    RETRY_EINTR(rc, close(randfd));
-#elif defined(JANET_BSD) || defined(JANET_APPLE)
-    (void) genericerr;
-    arc4random_buf(buffer->data + offset, n);
-#else
-    (void) genericerr;
-    janet_panic("cryptorand currently unsupported on this platform");
-#endif
     return janet_wrap_buffer(buffer);
 }
 
@@ -1224,12 +1345,15 @@ static Janet os_rename(int32_t argc, Janet *argv) {
 
 static Janet os_realpath(int32_t argc, Janet *argv) {
     janet_fixarity(argc, 1);
-#ifdef JANET_NO_REALPATH
-    (void) argv;
-    janet_panic("os/realpath not supported on this platform");
-#else
     const char *src = janet_getcstring(argv, 0);
+#ifdef JANET_NO_REALPATH
+    janet_panic("os/realpath not enabled for this platform");
+#else
+#ifdef JANET_WINDOWS
+    char *dest = _fullpath(NULL, src, _MAX_PATH);
+#else
     char *dest = realpath(src, NULL);
+#endif
     if (NULL == dest) janet_panicf("%s: %s", strerror(errno), src);
     Janet ret = janet_cstringv(dest);
     free(dest);
@@ -1396,9 +1520,13 @@ static const JanetReg os_cfuns[] = {
              "\t:e - enables passing an environment to the program. Without :e, the "
              "current environment is inherited.\n"
              "\t:p - allows searching the current PATH for the binary to execute. "
-             "Without this flag, binaries must use absolute paths.\n\n"
+             "Without this flag, binaries must use absolute paths.\n"
-             "env is a table or struct mapping environment variables to values. "
+             "\t:x - raise error if exit code is non-zero.\n"
-             "Returns the exit status of the program.")
+             "\t:a - Runs the process asynchronously and returns a core/process.\n\n"
+             "env is a table or struct mapping environment variables to values. It can also "
+             "contain the keys :in, :out, and :err, which allow redirecting stdio in the subprocess. "
+             "These arguments should be core/file values. "
+             "Returns the exit status of the program, or a core/process object if the :a flag is given.")
     },
     {
         "os/shell", os_shell,
@@ -1490,6 +1618,18 @@ static const JanetReg os_cfuns[] = {
         JDOC("(os/perm-int bytes)\n\n"
              "Parse a 9 character permission string and return an integer that can be used by chmod.")
     },
+    {
+        "os/proc-wait", os_proc_wait,
+        JDOC("(os/proc-wait proc)\n\n"
+             "Block until the subprocess completes. Returns the subprocess return code.")
+    },
+    {
+        "os/proc-kill", os_proc_kill,
+        JDOC("(os/proc-kill proc &opt wait)\n\n"
+             "Kill a subprocess by sending SIGKILL to it on posix systems, or by closing the process "
+             "handle on windows. If wait is truthy, will wait for the process to finsih and "
+             "returns the exit code. Otherwise, returns proc.")
+    },
 #endif
     {NULL, NULL, NULL}
 };

src/core/parse.c

@@ -167,12 +167,12 @@ static void popstate(JanetParser *p, Janet val) {
     for (;;) {
         JanetParseState top = p->states[--p->statecount];
         JanetParseState *newtop = p->states + p->statecount - 1;
+        /* Source mapping info */
+        if (janet_checktype(val, JANET_TUPLE)) {
+            janet_tuple_sm_line(janet_unwrap_tuple(val)) = (int32_t) top.line;
+            janet_tuple_sm_column(janet_unwrap_tuple(val)) = (int32_t) top.column;
+        }
         if (newtop->flags & PFLAG_CONTAINER) {
-            /* Source mapping info */
-            if (janet_checktype(val, JANET_TUPLE)) {
-                janet_tuple_sm_line(janet_unwrap_tuple(val)) = (int32_t) top.line;
-                janet_tuple_sm_column(janet_unwrap_tuple(val)) = (int32_t) top.column;
-            }
             newtop->argn++;
             /* Keep track of number of values in the root state */
             if (p->statecount == 1) p->pending++;

src/core/peg.c

@@ -150,6 +150,7 @@ tail:
             down1(s);
             const uint8_t *result = peg_rule(s, s->bytecode + rule[2], text);
             up1(s);
+            text -= ((int32_t *)rule)[1];
             return result ? text : NULL;
         }
 
@@ -205,6 +206,29 @@ tail:
             return (result) ? NULL : text;
         }
 
+        case RULE_THRU:
+        case RULE_TO: {
+            const uint32_t *rule_a = s->bytecode + rule[1];
+            const uint8_t *next_text;
+            CapState cs = cap_save(s);
+            down1(s);
+            while (text < s->text_end) {
+                CapState cs2 = cap_save(s);
+                next_text = peg_rule(s, rule_a, text);
+                if (next_text) {
+                    if (rule[0] == RULE_TO) cap_load(s, cs2);
+                    break;
+                }
+                text++;
+            }
+            up1(s);
+            if (text >= s->text_end) {
+                cap_load(s, cs);
+                return NULL;
+            }
+            return rule[0] == RULE_TO ? text : next_text;
+        }
+
         case RULE_BETWEEN: {
             uint32_t lo = rule[1];
             uint32_t hi = rule[2];
@@ -764,6 +788,12 @@ static void spec_error(Builder *b, int32_t argc, const Janet *argv) {
 static void spec_drop(Builder *b, int32_t argc, const Janet *argv) {
     spec_onerule(b, argc, argv, RULE_DROP);
 }
+static void spec_to(Builder *b, int32_t argc, const Janet *argv) {
+    spec_onerule(b, argc, argv, RULE_TO);
+}
+static void spec_thru(Builder *b, int32_t argc, const Janet *argv) {
+    spec_onerule(b, argc, argv, RULE_THRU);
+}
 
 /* Rule of the form [rule, tag] */
 static void spec_cap1(Builder *b, int32_t argc, const Janet *argv, uint32_t op) {
@@ -894,6 +924,8 @@ static const SpecialPair peg_specials[] = {
     {"sequence", spec_sequence},
     {"set", spec_set},
     {"some", spec_some},
+    {"thru", spec_thru},
+    {"to", spec_to},
 };
 
 /* Compile a janet value into a rule and return the rule index. */
@@ -996,6 +1028,14 @@ static uint32_t peg_compile1(Builder *b, Janet peg) {
             const Janet *tup = janet_unwrap_tuple(peg);
             int32_t len = janet_tuple_length(tup);
             if (len == 0) peg_panic(b, "tuple in grammar must have non-zero length");
+            if (janet_checkint(tup[0])) {
+                int32_t n = janet_unwrap_integer(tup[0]);
+                if (n < 0) {
+                    peg_panicf(b, "expected non-negative integer, got %d", n);
+                }
+                spec_repeat(b, len, tup);
+                break;
+            }
             if (!janet_checktype(tup[0], JANET_SYMBOL))
                 peg_panicf(b, "expected grammar command, found %v", tup[0]);
             const uint8_t *sym = janet_unwrap_symbol(tup[0]);
@@ -1179,6 +1219,8 @@ static void *peg_unmarshal(JanetMarshalContext *ctx) {
             case RULE_ERROR:
             case RULE_DROP:
             case RULE_NOT:
+            case RULE_TO:
+            case RULE_THRU:
                 /* [rule] */
                 if (rule[1] >= blen) goto bad;
                 op_flags[rule[1]] |= 0x01;
@@ -1266,47 +1308,136 @@ static Janet cfun_peg_compile(int32_t argc, Janet *argv) {
     return janet_wrap_abstract(peg);
 }
 
-static Janet cfun_peg_match(int32_t argc, Janet *argv) {
+/* Common data for peg cfunctions */
-    janet_arity(argc, 2, -1);
+typedef struct {
     JanetPeg *peg;
+    PegState s;
+    JanetByteView bytes;
+    JanetByteView repl;
+    int32_t start;
+} PegCall;
+
+/* Initialize state for peg cfunctions */
+static PegCall peg_cfun_init(int32_t argc, Janet *argv, int get_replace) {
+    PegCall ret;
+    int32_t min = get_replace ? 3 : 2;
+    janet_arity(argc, get_replace, -1);
     if (janet_checktype(argv[0], JANET_ABSTRACT) &&
             janet_abstract_type(janet_unwrap_abstract(argv[0])) == &janet_peg_type) {
-        peg = janet_unwrap_abstract(argv[0]);
+        ret.peg = janet_unwrap_abstract(argv[0]);
     } else {
-        peg = compile_peg(argv[0]);
+        ret.peg = compile_peg(argv[0]);
     }
-    JanetByteView bytes = janet_getbytes(argv, 1);
+    if (get_replace) {
-    int32_t start;
+        ret.repl = janet_getbytes(argv, 1);
-    PegState s;
+        ret.bytes = janet_getbytes(argv, 2);
-    if (argc > 2) {
-        start = janet_gethalfrange(argv, 2, bytes.len, "offset");
-        s.extrac = argc - 3;
-        s.extrav = janet_tuple_n(argv + 3, argc - 3);
     } else {
-        start = 0;
+        ret.bytes = janet_getbytes(argv, 1);
-        s.extrac = 0;
-        s.extrav = NULL;
     }
-    s.mode = PEG_MODE_NORMAL;
+    if (argc > min) {
-    s.text_start = bytes.bytes;
+        ret.start = janet_gethalfrange(argv, min, ret.bytes.len, "offset");
-    s.text_end = bytes.bytes + bytes.len;
+        ret.s.extrac = argc - min - 1;
-    s.depth = JANET_RECURSION_GUARD;
+        ret.s.extrav = janet_tuple_n(argv + min + 1, argc - min - 1);
-    s.captures = janet_array(0);
+    } else {
-    s.scratch = janet_buffer(10);
+        ret.start = 0;
-    s.tags = janet_buffer(10);
+        ret.s.extrac = 0;
-    s.constants = peg->constants;
+        ret.s.extrav = NULL;
-    s.bytecode = peg->bytecode;
+    }
-    const uint8_t *result = peg_rule(&s, s.bytecode, bytes.bytes + start);
+    ret.s.mode = PEG_MODE_NORMAL;
-    return result ? janet_wrap_array(s.captures) : janet_wrap_nil();
+    ret.s.text_start = ret.bytes.bytes;
+    ret.s.text_end = ret.bytes.bytes + ret.bytes.len;
+    ret.s.depth = JANET_RECURSION_GUARD;
+    ret.s.captures = janet_array(0);
+    ret.s.scratch = janet_buffer(10);
+    ret.s.tags = janet_buffer(10);
+    ret.s.constants = ret.peg->constants;
+    ret.s.bytecode = ret.peg->bytecode;
+    return ret;
 }
 
+static void peg_call_reset(PegCall *c) {
+    c->s.captures->count = 0;
+    c->s.scratch->count = 0;
+    c->s.tags->count = 0;
+}
+
+static Janet cfun_peg_match(int32_t argc, Janet *argv) {
+    PegCall c = peg_cfun_init(argc, argv, 0);
+    const uint8_t *result = peg_rule(&c.s, c.s.bytecode, c.bytes.bytes + c.start);
+    return result ? janet_wrap_array(c.s.captures) : janet_wrap_nil();
+}
+
+static Janet cfun_peg_find(int32_t argc, Janet *argv) {
+    PegCall c = peg_cfun_init(argc, argv, 0);
+    for (int32_t i = c.start; i < c.bytes.len; i++) {
+        peg_call_reset(&c);
+        if (peg_rule(&c.s, c.s.bytecode, c.bytes.bytes + i))
+            return janet_wrap_integer(i);
+    }
+    return janet_wrap_nil();
+}
+
+static Janet cfun_peg_find_all(int32_t argc, Janet *argv) {
+    PegCall c = peg_cfun_init(argc, argv, 0);
+    JanetArray *ret = janet_array(0);
+    for (int32_t i = c.start; i < c.bytes.len; i++) {
+        peg_call_reset(&c);
+        if (peg_rule(&c.s, c.s.bytecode, c.bytes.bytes + i))
+            janet_array_push(ret, janet_wrap_integer(i));
+    }
+    return janet_wrap_array(ret);
+}
+
+static Janet cfun_peg_replace_generic(int32_t argc, Janet *argv, int only_one) {
+    PegCall c = peg_cfun_init(argc, argv, 1);
+    JanetBuffer *ret = janet_buffer(0);
+    int32_t trail = 0;
+    for (int32_t i = c.start; i < c.bytes.len;) {
+        peg_call_reset(&c);
+        const uint8_t *result = peg_rule(&c.s, c.s.bytecode, c.bytes.bytes + i);
+        if (NULL != result) {
+            if (trail < i) {
+                janet_buffer_push_bytes(ret, c.bytes.bytes + trail, (i - trail));
+                trail = i;
+            }
+            int32_t nexti = (int32_t)(result - c.bytes.bytes);
+            janet_buffer_push_bytes(ret, c.repl.bytes, c.repl.len);
+            trail = nexti;
+            if (nexti == i) nexti++;
+            i = nexti;
+            if (only_one) break;
+        } else {
+            i++;
+        }
+    }
+    if (trail < c.bytes.len) {
+        janet_buffer_push_bytes(ret, c.bytes.bytes + trail, (c.bytes.len - trail));
+    }
+    return janet_wrap_buffer(ret);
+}
+
+static Janet cfun_peg_replace_all(int32_t argc, Janet *argv) {
+    return cfun_peg_replace_generic(argc, argv, 0);
+}
+
+static Janet cfun_peg_replace(int32_t argc, Janet *argv) {
+    return cfun_peg_replace_generic(argc, argv, 1);
+}
+
+static JanetMethod peg_methods[] = {
+    {"match", cfun_peg_match},
+    {"find", cfun_peg_find},
+    {"find-all", cfun_peg_find_all},
+    {"replace", cfun_peg_replace},
+    {"replace-all", cfun_peg_replace_all},
+    {NULL, NULL}
+};
+
 static int cfun_peg_getter(JanetAbstract a, Janet key, Janet *out) {
     (void) a;
-    if (janet_keyeq(key, "match")) {
+    if (!janet_checktype(key, JANET_KEYWORD))
-        *out = janet_wrap_cfunction(cfun_peg_match);
+        return 0;
-        return 1;
+    return janet_getmethod(janet_unwrap_keyword(key), peg_methods, out);
-    }
-    return 0;
 }
 
 static const JanetReg peg_cfuns[] = {
@@ -1322,6 +1453,27 @@ static const JanetReg peg_cfuns[] = {
              "Match a Parsing Expression Grammar to a byte string and return an array of captured values. "
              "Returns nil if text does not match the language defined by peg. The syntax of PEGs is documented on the Janet website.")
     },
+    {
+        "peg/find", cfun_peg_find,
+        JDOC("(peg/find peg text &opt start & args)\n\n"
+             "Find first index where the peg matches in text. Returns an integer, or nil if not found.")
+    },
+    {
+        "peg/find-all", cfun_peg_find_all,
+        JDOC("(peg/find-all peg text &opt start & args)\n\n"
+             "Find all indexes where the peg matches in text. Returns an array of integers.")
+    },
+    {
+        "peg/replace", cfun_peg_replace,
+        JDOC("(peg/replace peg repl text &opt start & args)\n\n"
+             "Replace first match of peg in text with repl, returning a new buffer. The peg does not need to make captures to do replacement. "
+             "If no matches are found, returns the input string in a new buffer.")
+    },
+    {
+        "peg/replace-all", cfun_peg_replace_all,
+        JDOC("(peg/replace-all peg repl text &opt start & args)\n\n"
+             "Replace all matches of peg in text with repl, returning a new buffer. The peg does not need to make captures to do replacement.")
+    },
     {NULL, NULL, NULL}
 };
 

src/core/pp.c

@@ -39,11 +39,9 @@
 
 static void number_to_string_b(JanetBuffer *buffer, double x) {
     janet_buffer_ensure(buffer, buffer->count + BUFSIZE, 2);
-    /* Use int32_t range for valid integers because that is the
-     * range most integer-expecting functions in the C api use. */
     const char *fmt = (x == floor(x) &&
-                       x <= ((double) INT32_MAX) &&
+                       x <= JANET_INTMAX_DOUBLE &&
-                       x >= ((double) INT32_MIN)) ? "%.0f" : "%g";
+                       x >= JANET_INTMIN_DOUBLE) ? "%.0f" : "%g";
     int count = snprintf((char *) buffer->data + buffer->count, BUFSIZE, fmt, x);
     buffer->count += count;
 }
@@ -123,9 +121,6 @@ static void string_description_b(JanetBuffer *buffer, const char *title, void *p
 #undef POINTSIZE
 }
 
-#undef HEX
-#undef BUFSIZE
-
 static void janet_escape_string_impl(JanetBuffer *buffer, const uint8_t *str, int32_t len) {
     janet_buffer_push_u8(buffer, '"');
     for (int32_t i = 0; i < len; ++i) {
@@ -191,7 +186,7 @@ static void janet_escape_buffer_b(JanetBuffer *buffer, JanetBuffer *bx) {
 void janet_to_string_b(JanetBuffer *buffer, Janet x) {
     switch (janet_type(x)) {
         case JANET_NIL:
-            janet_buffer_push_cstring(buffer, "nil");
+            janet_buffer_push_cstring(buffer, "");
             break;
         case JANET_BOOLEAN:
             janet_buffer_push_cstring(buffer,
@@ -280,6 +275,9 @@ void janet_description_b(JanetBuffer *buffer, Janet x) {
     switch (janet_type(x)) {
         default:
             break;
+        case JANET_NIL:
+            janet_buffer_push_cstring(buffer, "nil");
+            return;
         case JANET_KEYWORD:
             janet_buffer_push_u8(buffer, ':');
             break;
@@ -354,12 +352,16 @@ static int print_jdn_one(struct pretty *S, Janet x, int depth) {
     if (depth == 0) return 1;
     switch (janet_type(x)) {
         case JANET_NIL:
-        case JANET_NUMBER:
         case JANET_BOOLEAN:
         case JANET_BUFFER:
         case JANET_STRING:
             janet_description_b(S->buffer, x);
             break;
+        case JANET_NUMBER:
+            janet_buffer_ensure(S->buffer, S->buffer->count + BUFSIZE, 2);
+            int count = snprintf((char *) S->buffer->data + S->buffer->count, BUFSIZE, "%.17g", janet_unwrap_number(x));
+            S->buffer->count += count;
+            break;
         case JANET_SYMBOL:
         case JANET_KEYWORD:
             if (contains_bad_chars(janet_unwrap_keyword(x), janet_type(x) == JANET_SYMBOL)) return 1;
@@ -994,3 +996,6 @@ void janet_buffer_format(
         }
     }
 }
+
+#undef HEX
+#undef BUFSIZE

src/core/run.c

@@ -23,7 +23,6 @@
 #ifndef JANET_AMALG
 #include "features.h"
 #include <janet.h>
-#include "state.h"
 #endif
 
 /* Run a string */
@@ -56,9 +55,10 @@ int janet_dobytes(JanetTable *env, const uint8_t *bytes, int32_t len, const char
                     done = 1;
                 }
             } else {
+                ret = janet_wrap_string(cres.error);
                 if (cres.macrofiber) {
                     janet_eprintf("compile error in %s: ", sourcePath);
-                    janet_stacktrace(cres.macrofiber, janet_wrap_string(cres.error));
+                    janet_stacktrace(cres.macrofiber, ret);
                 } else {
                     janet_eprintf("compile error in %s: %s\n", sourcePath,
                                   (const char *)cres.error);
@@ -68,25 +68,23 @@ int janet_dobytes(JanetTable *env, const uint8_t *bytes, int32_t len, const char
             }
         }
 
+        if (done) break;
+
         /* Dispatch based on parse state */
         switch (janet_parser_status(&parser)) {
             case JANET_PARSE_DEAD:
                 done = 1;
                 break;
-            case JANET_PARSE_ERROR:
+            case JANET_PARSE_ERROR: {
+                const char *e = janet_parser_error(&parser);
                 errflags |= 0x04;
-                janet_eprintf("parse error in %s: %s\n",
+                ret = janet_cstringv(e);
-                              sourcePath, janet_parser_error(&parser));
+                janet_eprintf("parse error in %s: %s\n", sourcePath, e);
                 done = 1;
                 break;
-            case JANET_PARSE_PENDING:
+            }
-                if (index == len) {
-                    janet_parser_eof(&parser);
-                } else {
-                    janet_parser_consume(&parser, bytes[index++]);
-                }
-                break;
             case JANET_PARSE_ROOT:
+            case JANET_PARSE_PENDING:
                 if (index >= len) {
                     janet_parser_eof(&parser);
                 } else {

src/core/specials.c

@@ -622,10 +622,11 @@ static JanetSlot janetc_while(JanetFopts opts, int32_t argn, const Janet *argv)
     /* Check if closure created in while scope. If so,
      * recompile in a function scope. */
     if (tempscope.flags & JANET_SCOPE_CLOSURE) {
+        subopts = janetc_fopts_default(c);
         tempscope.flags |= JANET_SCOPE_UNUSED;
         janetc_popscope(c);
-        janet_v__cnt(c->buffer) = labelwt;
+        if (c->buffer) janet_v__cnt(c->buffer) = labelwt;
-        janet_v__cnt(c->mapbuffer) = labelwt;
+        if (c->mapbuffer) janet_v__cnt(c->mapbuffer) = labelwt;
 
         janetc_scope(&tempscope, c, JANET_SCOPE_FUNCTION, "while-iife");
 
@@ -648,6 +649,7 @@ static JanetSlot janetc_while(JanetFopts opts, int32_t argn, const Janet *argv)
         /* Compile function */
         JanetFuncDef *def = janetc_pop_funcdef(c);
         def->name = janet_cstring("_while");
+        janet_def_addflags(def);
         int32_t defindex = janetc_addfuncdef(c, def);
         /* And then load the closure and call it. */
         int32_t cloreg = janetc_regalloc_temp(&c->scope->ra, JANETC_REGTEMP_0);
@@ -822,6 +824,7 @@ static JanetSlot janetc_fn(JanetFopts opts, int32_t argn, const Janet *argv) {
     if (structarg) def->flags |= JANET_FUNCDEF_FLAG_STRUCTARG;
 
     if (selfref) def->name = janet_unwrap_symbol(head);
+    janet_def_addflags(def);
     defindex = janetc_addfuncdef(c, def);
 
     /* Ensure enough slots for vararg function. */

src/core/state.h

@@ -34,6 +34,9 @@
 
 typedef struct JanetScratch JanetScratch;
 
+/* Top level dynamic bindings */
+extern JANET_THREAD_LOCAL JanetTable *janet_vm_top_dyns;
+
 /* Cache the core environment */
 extern JANET_THREAD_LOCAL JanetTable *janet_vm_core_env;
 
@@ -68,6 +71,7 @@ extern JANET_THREAD_LOCAL uint32_t janet_vm_cache_deleted;
 extern JANET_THREAD_LOCAL void *janet_vm_blocks;
 extern JANET_THREAD_LOCAL size_t janet_vm_gc_interval;
 extern JANET_THREAD_LOCAL size_t janet_vm_next_collection;
+extern JANET_THREAD_LOCAL size_t janet_vm_block_count;
 extern JANET_THREAD_LOCAL int janet_vm_gc_suspend;
 
 /* GC roots */

src/core/string.c

@@ -62,7 +62,7 @@ int janet_string_compare(const uint8_t *lhs, const uint8_t *rhs) {
     int32_t ylen = janet_string_length(rhs);
     int32_t len = xlen > ylen ? ylen : xlen;
     int res = memcmp(lhs, rhs, len);
-    if (res) return res;
+    if (res) return res > 0 ? 1 : -1;
     if (xlen == ylen) return 0;
     return xlen < ylen ? -1 : 1;
 }
@@ -176,6 +176,18 @@ static Janet cfun_string_slice(int32_t argc, Janet *argv) {
     return janet_stringv(view.bytes + range.start, range.end - range.start);
 }
 
+static Janet cfun_symbol_slice(int32_t argc, Janet *argv) {
+    JanetByteView view = janet_getbytes(argv, 0);
+    JanetRange range = janet_getslice(argc, argv);
+    return janet_symbolv(view.bytes + range.start, range.end - range.start);
+}
+
+static Janet cfun_keyword_slice(int32_t argc, Janet *argv) {
+    JanetByteView view = janet_getbytes(argv, 0);
+    JanetRange range = janet_getslice(argc, argv);
+    return janet_keywordv(view.bytes + range.start, range.end - range.start);
+}
+
 static Janet cfun_string_repeat(int32_t argc, Janet *argv) {
     janet_fixarity(argc, 2);
     JanetByteView view = janet_getbytes(argv, 0);
@@ -529,6 +541,16 @@ static const JanetReg string_cfuns[] = {
              "from the end of the string. Note that index -1 is synonymous with "
              "index (length bytes) to allow a full negative slice range. ")
     },
+    {
+        "keyword/slice", cfun_keyword_slice,
+        JDOC("(keyword/slice bytes &opt start end)\n\n"
+             "Same a string/slice, but returns a keyword.")
+    },
+    {
+        "symbol/slice", cfun_symbol_slice,
+        JDOC("(symbol/slice bytes &opt start end)\n\n"
+             "Same a string/slice, but returns a symbol.")
+    },
     {
         "string/repeat", cfun_string_repeat,
         JDOC("(string/repeat bytes n)\n\n"

src/core/table.c

@@ -173,7 +173,7 @@ Janet janet_table_rawget(JanetTable *t, Janet key) {
 Janet janet_table_remove(JanetTable *t, Janet key) {
     JanetKV *bucket = janet_table_find(t, key);
     if (NULL != bucket && !janet_checktype(bucket->key, JANET_NIL)) {
-        Janet ret = bucket->key;
+        Janet ret = bucket->value;
         t->count--;
         t->deleted++;
         bucket->key = janet_wrap_nil();

src/core/thread.c

@@ -66,9 +66,15 @@ struct JanetMailbox {
     JanetBuffer messages[];
 };
 
+#define JANET_THREAD_HEAVYWEIGHT 0x1
+#define JANET_THREAD_ABSTRACTS 0x2
+#define JANET_THREAD_CFUNCTIONS 0x4
+static const char janet_thread_flags[] = "hac";
+
 typedef struct {
     JanetMailbox *original;
     JanetMailbox *newbox;
+    uint64_t flags;
 } JanetMailboxPair;
 
 static JANET_THREAD_LOCAL JanetMailbox *janet_vm_mailbox = NULL;
@@ -175,7 +181,7 @@ static int thread_mark(void *p, size_t size) {
     return 0;
 }
 
-static JanetMailboxPair *make_mailbox_pair(JanetMailbox *original) {
+static JanetMailboxPair *make_mailbox_pair(JanetMailbox *original, uint64_t flags) {
     JanetMailboxPair *pair = malloc(sizeof(JanetMailboxPair));
     if (NULL == pair) {
         JANET_OUT_OF_MEMORY;
@@ -183,6 +189,7 @@ static JanetMailboxPair *make_mailbox_pair(JanetMailbox *original) {
     pair->original = original;
     janet_mailbox_ref(original, 1);
     pair->newbox = janet_mailbox_create(1, 16);
+    pair->flags = flags;
     return pair;
 }
 
@@ -227,7 +234,7 @@ static void janet_waiter_init(JanetWaiter *waiter, double sec) {
     if (waiter->timedwait) {
         /* N seconds -> timespec of (now + sec) */
         struct timespec now;
-        clock_gettime(CLOCK_REALTIME, &now);
+        janet_gettime(&now);
         time_t tvsec = (time_t) floor(sec);
         long tvnsec = (long) floor(1000000000.0 * (sec - ((double) tvsec)));
         tvsec += now.tv_sec;
@@ -368,8 +375,12 @@ int janet_thread_receive(Janet *msg_out, double timeout) {
 
             /* Handle errors */
             if (setjmp(buf)) {
-                /* Cleanup jmp_buf, keep lock */
+                /* Cleanup jmp_buf, return error.
+                 * Do not ignore bad messages as before. */
                 janet_vm_jmp_buf = old_buf;
+                *msg_out = *janet_vm_return_reg;
+                janet_mailbox_unlock(mailbox);
+                return 2;
             } else {
                 JanetBuffer *msgbuf = mailbox->messages + mailbox->messageFirst;
                 mailbox->messageCount--;
@@ -404,7 +415,6 @@ int janet_thread_receive(Janet *msg_out, double timeout) {
             return 1;
         }
     }
-
 }
 
 static int janet_thread_getter(void *p, Janet key, Janet *out);
@@ -442,16 +452,44 @@ static int thread_worker(JanetMailboxPair *pair) {
     janet_init();
 
     /* Get dictionaries for default encode/decode */
-    JanetTable *encode = janet_get_core_table("make-image-dict");
+    JanetTable *encode;
+    if (pair->flags & JANET_THREAD_HEAVYWEIGHT) {
+        encode = janet_get_core_table("make-image-dict");
+    } else {
+        encode = NULL;
+        janet_vm_thread_decode = janet_table(0);
+        janet_gcroot(janet_wrap_table(janet_vm_thread_decode));
+    }
 
     /* Create parent thread */
     JanetThread *parent = janet_make_thread(pair->original, encode);
     Janet parentv = janet_wrap_abstract(parent);
 
+    /* Unmarshal the abstract registry */
+    if (pair->flags & JANET_THREAD_ABSTRACTS) {
+        Janet reg;
+        int status = janet_thread_receive(&reg, INFINITY);
+        if (status) goto error;
+        if (!janet_checktype(reg, JANET_TABLE)) goto error;
+        janet_gcunroot(janet_wrap_table(janet_vm_abstract_registry));
+        janet_vm_abstract_registry = janet_unwrap_table(reg);
+        janet_gcroot(janet_wrap_table(janet_vm_abstract_registry));
+    }
+
+    /* Unmarshal the normal registry */
+    if (pair->flags & JANET_THREAD_CFUNCTIONS) {
+        Janet reg;
+        int status = janet_thread_receive(&reg, INFINITY);
+        if (status) goto error;
+        if (!janet_checktype(reg, JANET_TABLE)) goto error;
+        janet_gcunroot(janet_wrap_table(janet_vm_registry));
+        janet_vm_registry = janet_unwrap_table(reg);
+        janet_gcroot(janet_wrap_table(janet_vm_registry));
+    }
+
     /* Unmarshal the function */
     Janet funcv;
     int status = janet_thread_receive(&funcv, INFINITY);
-
     if (status) goto error;
     if (!janet_checktype(funcv, JANET_FUNCTION)) goto error;
     JanetFunction *func = janet_unwrap_function(funcv);
@@ -464,6 +502,10 @@ static int thread_worker(JanetMailboxPair *pair) {
     /* Call function */
     Janet argv[1] = { parentv };
     fiber = janet_fiber(func, 64, 1, argv);
+    if (pair->flags & JANET_THREAD_HEAVYWEIGHT) {
+        fiber->env = janet_table(0);
+        fiber->env->proto = janet_core_env(NULL);
+    }
     JanetSignal sig = janet_continue(fiber, janet_wrap_nil(), &out);
     if (sig != JANET_SIGNAL_OK && sig < JANET_SIGNAL_USER0) {
         janet_eprintf("in thread %v: ", janet_wrap_abstract(janet_make_thread(pair->newbox, encode)));
@@ -558,22 +600,40 @@ static Janet cfun_thread_current(int32_t argc, Janet *argv) {
 }
 
 static Janet cfun_thread_new(int32_t argc, Janet *argv) {
-    janet_arity(argc, 1, 2);
+    janet_arity(argc, 1, 3);
     /* Just type checking */
     janet_getfunction(argv, 0);
     int32_t cap = janet_optinteger(argv, argc, 1, 10);
     if (cap < 1 || cap > UINT16_MAX) {
         janet_panicf("bad slot #1, expected integer in range [1, 65535], got %d", cap);
     }
-    JanetTable *encode = janet_get_core_table("make-image-dict");
+    uint64_t flags = argc >= 3 ? janet_getflags(argv, 2, janet_thread_flags) : JANET_THREAD_ABSTRACTS;
+    JanetTable *encode;
+    if (flags & JANET_THREAD_HEAVYWEIGHT) {
+        encode = janet_get_core_table("make-image-dict");
+    } else {
+        encode = NULL;
+    }
 
-    JanetMailboxPair *pair = make_mailbox_pair(janet_vm_mailbox);
+    JanetMailboxPair *pair = make_mailbox_pair(janet_vm_mailbox, flags);
     JanetThread *thread = janet_make_thread(pair->newbox, encode);
     if (janet_thread_start_child(pair)) {
         destroy_mailbox_pair(pair);
         janet_panic("could not start thread");
     }
 
+    if (flags & JANET_THREAD_ABSTRACTS) {
+        if (janet_thread_send(thread, janet_wrap_table(janet_vm_abstract_registry), INFINITY)) {
+            janet_panic("could not send abstract registry to thread");
+        }
+    }
+
+    if (flags & JANET_THREAD_CFUNCTIONS) {
+        if (janet_thread_send(thread, janet_wrap_table(janet_vm_registry), INFINITY)) {
+            janet_panic("could not send registry to thread");
+        }
+    }
+
     /* If thread started, send the worker function. */
     if (janet_thread_send(thread, argv[0], INFINITY)) {
         janet_panicf("could not send worker function %v to thread", argv[0]);
@@ -607,6 +667,8 @@ static Janet cfun_thread_receive(int32_t argc, Janet *argv) {
             break;
         case 1:
             janet_panicf("timeout after %f seconds", wait);
+        case 2:
+            janet_panicf("failed to receive message: %v", out);
     }
     return out;
 }
@@ -618,6 +680,18 @@ static Janet cfun_thread_close(int32_t argc, Janet *argv) {
     return janet_wrap_nil();
 }
 
+static Janet cfun_thread_exit(int32_t argc, Janet *argv) {
+    (void) argv;
+    janet_arity(argc, 0, 1);
+#if defined(JANET_WINDOWS)
+    int32_t flag = janet_optinteger(argv, argc, 0, 0);
+    ExitThread(flag);
+#else
+    pthread_exit(NULL);
+#endif
+    return janet_wrap_nil();
+}
+
 static const JanetMethod janet_thread_methods[] = {
     {"send", cfun_thread_send},
     {"close", cfun_thread_close},
@@ -638,10 +712,14 @@ static const JanetReg threadlib_cfuns[] = {
     },
     {
         "thread/new", cfun_thread_new,
-        JDOC("(thread/new func &opt capacity)\n\n"
+        JDOC("(thread/new func &opt capacity flags)\n\n"
              "Start a new thread that will start immediately. "
              "If capacity is provided, that is how many messages can be stored in the thread's mailbox before blocking senders. "
              "The capacity must be between 1 and 65535 inclusive, and defaults to 10. "
+             "Can optionally provide flags to the new thread - supported flags are:\n"
+             "\t:h - Start a heavyweight thread. This loads the core environment by default, so may use more memory initially. Messages may compress better, though.\n"
+             "\t:a - Allow sending over registered abstract types to the new thread\n"
+             "\t:c - Send over cfunction information to the new thread.\n"
              "Returns a handle to the new thread.")
     },
     {
@@ -662,6 +740,12 @@ static const JanetReg threadlib_cfuns[] = {
              "Close a thread, unblocking it and ending communication with it. Note that closing "
              "a thread is idempotent and does not cancel the thread's operation. Returns nil.")
     },
+    {
+        "thread/exit", cfun_thread_exit,
+        JDOC("(thread/exit &opt code)\n\n"
+             "Exit from the current thread. If no more threads are running, ends the process, but otherwise does "
+             "not end the current process.")
+    },
     {NULL, NULL, NULL}
 };
 

src/core/typedarray.c

@@ -275,6 +275,24 @@ static void ta_setter(void *p, Janet key, Janet value) {
     }
 }
 
+static Janet ta_view_next(void *p, Janet key) {
+    JanetTArrayView *view = p;
+    if (janet_checktype(key, JANET_NIL)) {
+        if (view->size > 0) {
+            return janet_wrap_number(0);
+        } else {
+            return janet_wrap_nil();
+        }
+    }
+    if (!janet_checksize(key)) janet_panic("expected size as key");
+    size_t index = (size_t) janet_unwrap_number(key);
+    index++;
+    if (index < view->size) {
+        return janet_wrap_number((double) index);
+    }
+    return janet_wrap_nil();
+}
+
 const JanetAbstractType janet_ta_view_type = {
     "ta/view",
     NULL,
@@ -283,7 +301,11 @@ const JanetAbstractType janet_ta_view_type = {
     ta_setter,
     ta_view_marshal,
     ta_view_unmarshal,
-    JANET_ATEND_UNMARSHAL
+    NULL,
+    NULL,
+    NULL,
+    ta_view_next,
+    JANET_ATEND_NEXT
 };
 
 JanetTArrayBuffer *janet_tarray_buffer(size_t size) {
@@ -354,18 +376,29 @@ static Janet cfun_typed_array_new(int32_t argc, Janet *argv) {
     if (argc > 3)
         offset = janet_getsize(argv, 3);
     if (argc > 4) {
-        if (!janet_checktype(argv[4], JANET_ABSTRACT)) {
+        int32_t blen;
-            janet_panicf("bad slot #%d, expected ta/view|ta/buffer, got %v",
+        const uint8_t *bytes;
-                         4, argv[4]);
+        if (janet_bytes_view(argv[4], &bytes, &blen)) {
-        }
+            buffer = janet_abstract(&janet_ta_buffer_type, sizeof(JanetTArrayBuffer));
-        void *p = janet_unwrap_abstract(argv[4]);
+            ta_buffer_init(buffer, (size_t) blen);
-        if (janet_abstract_type(p) == &janet_ta_view_type) {
+            memcpy(buffer->data, bytes, blen);
-            JanetTArrayView *view = (JanetTArrayView *)p;
-            offset = (view->buffer->data - view->as.u8) + offset * ta_type_sizes[view->type];
-            stride *= view->stride;
-            buffer = view->buffer;
         } else {
-            buffer = p;
+            if (!janet_checktype(argv[4], JANET_ABSTRACT)) {
+                janet_panicf("bad slot #%d, expected ta/view|ta/buffer, got %v",
+                             4, argv[4]);
+            }
+            void *p = janet_unwrap_abstract(argv[4]);
+            if (janet_abstract_type(p) == &janet_ta_view_type) {
+                JanetTArrayView *view = (JanetTArrayView *)p;
+                offset = (view->buffer->data - view->as.u8) + offset * ta_type_sizes[view->type];
+                stride *= view->stride;
+                buffer = view->buffer;
+            } else if (janet_abstract_type(p) == &janet_ta_buffer_type) {
+                buffer = p;
+            } else {
+                janet_panicf("bad slot #%d, expected ta/view|ta/buffer, got %v",
+                             4, argv[4]);
+            }
         }
     }
     JanetTArrayView *view = janet_tarray_view(type, size, stride, offset, buffer);

src/core/util.c

@@ -26,6 +26,14 @@
 #include "util.h"
 #include "state.h"
 #include "gc.h"
+#ifdef JANET_WINDOWS
+#include <windows.h>
+#else
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#endif
 #endif
 
 #include <inttypes.h>
@@ -94,7 +102,7 @@ const char *const janet_status_names[16] = {
     "alive"
 };
 
-#ifdef JANET_NO_PRF
+#ifndef JANET_PRF
 
 int32_t janet_string_calchash(const uint8_t *str, int32_t len) {
     const uint8_t *end = str + len;
@@ -574,8 +582,12 @@ int janet_checksize(Janet x) {
     if (!janet_checktype(x, JANET_NUMBER))
         return 0;
     double dval = janet_unwrap_number(x);
-    return dval == (double)((size_t) dval) &&
+    if (dval != (double)((size_t) dval)) return 0;
-           dval <= SIZE_MAX;
+    if (SIZE_MAX > JANET_INTMAX_INT64) {
+        return dval <= JANET_INTMAX_INT64;
+    } else {
+        return dval <= SIZE_MAX;
+    }
 }
 
 JanetTable *janet_get_core_table(const char *name) {
@@ -586,3 +598,88 @@ JanetTable *janet_get_core_table(const char *name) {
     if (!janet_checktype(out, JANET_TABLE)) return NULL;
     return janet_unwrap_table(out);
 }
+
+/* Clock shims for various platforms */
+#ifdef JANET_GETTIME
+/* For macos */
+#ifdef __MACH__
+#include <mach/clock.h>
+#include <mach/mach.h>
+#endif
+#ifdef JANET_WINDOWS
+int janet_gettime(struct timespec *spec) {
+    FILETIME ftime;
+    GetSystemTimeAsFileTime(&ftime);
+    int64_t wintime = (int64_t)(ftime.dwLowDateTime) | ((int64_t)(ftime.dwHighDateTime) << 32);
+    /* Windows epoch is January 1, 1601 apparently */
+    wintime -= 116444736000000000LL;
+    spec->tv_sec  = wintime / 10000000LL;
+    /* Resolution is 100 nanoseconds. */
+    spec->tv_nsec = wintime % 10000000LL * 100;
+    return 0;
+}
+#elif defined(__MACH__)
+int janet_gettime(struct timespec *spec) {
+    clock_serv_t cclock;
+    mach_timespec_t mts;
+    host_get_clock_service(mach_host_self(), CALENDAR_CLOCK, &cclock);
+    clock_get_time(cclock, &mts);
+    mach_port_deallocate(mach_task_self(), cclock);
+    spec->tv_sec = mts.tv_sec;
+    spec->tv_nsec = mts.tv_nsec;
+    return 0;
+}
+#else
+int janet_gettime(struct timespec *spec) {
+    return clock_gettime(CLOCK_REALTIME, spec);
+}
+#endif
+#endif
+
+/* Setting C99 standard makes this not available, but it should
+ * work/link properly if we detect a BSD */
+#if defined(JANET_BSD) || defined(MAC_OS_X_VERSION_10_7)
+void arc4random_buf(void *buf, size_t nbytes);
+#endif
+
+int janet_cryptorand(uint8_t *out, size_t n) {
+#ifdef JANET_WINDOWS
+    for (size_t i = 0; i < n; i += sizeof(unsigned int)) {
+        unsigned int v;
+        if (rand_s(&v))
+            return -1;
+        for (int32_t j = 0; (j < sizeof(unsigned int)) && (i + j < n); j++) {
+            out[i + j] = v & 0xff;
+            v = v >> 8;
+        }
+    }
+    return 0;
+#elif defined(JANET_LINUX) || ( defined(JANET_APPLE) && !defined(MAC_OS_X_VERSION_10_7) )
+    /* We should be able to call getrandom on linux, but it doesn't seem
+       to be uniformly supported on linux distros.
+       On Mac, arc4random_buf wasn't available on until 10.7.
+       In these cases, use this fallback path for now... */
+    int rc;
+    int randfd;
+    RETRY_EINTR(randfd, open("/dev/urandom", O_RDONLY | O_CLOEXEC));
+    if (randfd < 0)
+        return -1;
+    while (n > 0) {
+        ssize_t nread;
+        RETRY_EINTR(nread, read(randfd, out, n));
+        if (nread <= 0) {
+            RETRY_EINTR(rc, close(randfd));
+            return -1;
+        }
+        out += nread;
+        n -= nread;
+    }
+    RETRY_EINTR(rc, close(randfd));
+    return 0;
+#elif defined(JANET_BSD) || defined(MAC_OS_X_VERSION_10_7)
+    arc4random_buf(out, n);
+    return 0;
+#else
+    return -1;
+#endif
+}

src/core/util.h

@@ -31,6 +31,11 @@
 #include <stdio.h>
 #include <errno.h>
 
+#if !defined(JANET_REDUCED_OS) || !defined(JANET_SINGLE_THREADED)
+#include <time.h>
+#define JANET_GETTIME
+#endif
+
 /* Handle runtime errors */
 #ifndef JANET_EXIT
 #include <stdio.h>
@@ -71,10 +76,10 @@ int32_t janet_tablen(int32_t n);
 void safe_memcpy(void *dest, const void *src, size_t len);
 void janet_buffer_push_types(JanetBuffer *buffer, int types);
 const JanetKV *janet_dict_find(const JanetKV *buckets, int32_t cap, Janet key);
-Janet janet_dict_get(const JanetKV *buckets, int32_t cap, Janet key);
 void janet_memempty(JanetKV *mem, int32_t count);
 void *janet_memalloc_empty(int32_t count);
 JanetTable *janet_get_core_table(const char *name);
+void janet_def_addflags(JanetFuncDef *def);
 const void *janet_strbinsearch(
     const void *tab,
     size_t tabcount,
@@ -97,6 +102,13 @@ void janet_core_def(JanetTable *env, const char *name, Janet x, const void *p);
 void janet_core_cfuns(JanetTable *env, const char *regprefix, const JanetReg *cfuns);
 #endif
 
+/* Clock gettime */
+#ifdef JANET_GETTIME
+int janet_gettime(struct timespec *spec);
+#endif
+
+#define RETRY_EINTR(RC, CALL) do { (RC) = CALL; } while((RC) < 0 && errno == EINTR)
+
 /* Initialize builtin libraries */
 void janet_lib_io(JanetTable *env);
 void janet_lib_math(JanetTable *env);

src/core/vm.c

@@ -33,6 +33,7 @@
 #include <math.h>
 
 /* VM state */
+JANET_THREAD_LOCAL JanetTable *janet_vm_top_dyns;
 JANET_THREAD_LOCAL JanetTable *janet_vm_core_env;
 JANET_THREAD_LOCAL JanetTable *janet_vm_registry;
 JANET_THREAD_LOCAL JanetTable *janet_vm_abstract_registry;
@@ -94,6 +95,10 @@ JANET_THREAD_LOCAL jmp_buf *janet_vm_jmp_buf = NULL;
     vm_commit(); \
     return (sig); \
 } while (0)
+#define vm_return_no_restore(sig, val) do { \
+    janet_vm_return_reg[0] = (val); \
+    return (sig); \
+} while (0)
 
 /* Next instruction variations */
 #define maybe_collect() do {\
@@ -373,9 +378,9 @@ static JanetSignal run_vm(JanetFiber *fiber, Janet in) {
         &&label_JOP_GREATER_THAN_EQUAL,
         &&label_JOP_LESS_THAN_EQUAL,
         &&label_JOP_NEXT,
-        &&label_unknown_op,
+        &&label_JOP_NOT_EQUALS,
-        &&label_unknown_op,
+        &&label_JOP_NOT_EQUALS_IMMEDIATE,
-        &&label_unknown_op,
+        &&label_JOP_CANCEL,
         &&label_unknown_op,
         &&label_unknown_op,
         &&label_unknown_op,
@@ -563,6 +568,15 @@ static JanetSignal run_vm(JanetFiber *fiber, Janet in) {
     register Janet *stack;
     register uint32_t *pc;
     register JanetFunction *func;
+
+    if (fiber->flags & JANET_FIBER_RESUME_SIGNAL) {
+        JanetSignal sig = (fiber->gc.flags & JANET_FIBER_STATUS_MASK) >> JANET_FIBER_STATUS_OFFSET;
+        fiber->gc.flags &= ~JANET_FIBER_STATUS_MASK;
+        fiber->flags &= ~(JANET_FIBER_RESUME_SIGNAL | JANET_FIBER_FLAG_MASK);
+        janet_vm_return_reg[0] = in;
+        return sig;
+    }
+
     vm_restore();
 
     if (fiber->flags & JANET_FIBER_DID_LONGJUMP) {
@@ -613,7 +627,7 @@ static JanetSignal run_vm(JanetFiber *fiber, Janet in) {
         Janet retval = stack[D];
         int entrance_frame = janet_stack_frame(stack)->flags & JANET_STACKFRAME_ENTRANCE;
         janet_fiber_popframe(fiber);
-        if (entrance_frame) vm_return(JANET_SIGNAL_OK, retval);
+        if (entrance_frame) vm_return_no_restore(JANET_SIGNAL_OK, retval);
         vm_restore();
         stack[A] = retval;
         vm_checkgc_pcnext();
@@ -623,7 +637,7 @@ static JanetSignal run_vm(JanetFiber *fiber, Janet in) {
         Janet retval = janet_wrap_nil();
         int entrance_frame = janet_stack_frame(stack)->flags & JANET_STACKFRAME_ENTRANCE;
         janet_fiber_popframe(fiber);
-        if (entrance_frame) vm_return(JANET_SIGNAL_OK, retval);
+        if (entrance_frame) vm_return_no_restore(JANET_SIGNAL_OK, retval);
         vm_restore();
         stack[A] = retval;
         vm_checkgc_pcnext();
@@ -787,11 +801,20 @@ static JanetSignal run_vm(JanetFiber *fiber, Janet in) {
     stack[A] = janet_wrap_boolean(janet_unwrap_integer(stack[B]) == CS);
     vm_pcnext();
 
+    VM_OP(JOP_NOT_EQUALS)
+    stack[A] = janet_wrap_boolean(!janet_equals(stack[B], stack[C]));
+    vm_pcnext();
+
+    VM_OP(JOP_NOT_EQUALS_IMMEDIATE)
+    stack[A] = janet_wrap_boolean(janet_unwrap_integer(stack[B]) != CS);
+    vm_pcnext();
+
     VM_OP(JOP_COMPARE)
     stack[A] = janet_wrap_integer(janet_compare(stack[B], stack[C]));
     vm_pcnext();
 
     VM_OP(JOP_NEXT)
+    vm_commit();
     stack[A] = janet_next(stack[B], stack[C]);
     vm_pcnext();
 
@@ -929,7 +952,7 @@ static JanetSignal run_vm(JanetFiber *fiber, Janet in) {
         if (janet_checktype(callee, JANET_FUNCTION)) {
             func = janet_unwrap_function(callee);
             if (func->gc.flags & JANET_FUNCFLAG_TRACE) {
-                vm_do_trace(func, fiber->stacktop - fiber->stackstart, stack);
+                vm_do_trace(func, fiber->stacktop - fiber->stackstart, fiber->data + fiber->stackstart);
             }
             janet_stack_frame(stack)->pc = pc;
             if (janet_fiber_funcframe(fiber, func)) {
@@ -968,7 +991,7 @@ static JanetSignal run_vm(JanetFiber *fiber, Janet in) {
         if (janet_checktype(callee, JANET_FUNCTION)) {
             func = janet_unwrap_function(callee);
             if (func->gc.flags & JANET_FUNCFLAG_TRACE) {
-                vm_do_trace(func, fiber->stacktop - fiber->stackstart, stack);
+                vm_do_trace(func, fiber->stacktop - fiber->stackstart, fiber->data + fiber->stackstart);
             }
             if (janet_fiber_funcframe_tail(fiber, func)) {
                 janet_stack_frame(fiber->data + fiber->frame)->pc = pc;
@@ -992,8 +1015,9 @@ static JanetSignal run_vm(JanetFiber *fiber, Janet in) {
                 retreg = call_nonfn(fiber, callee);
             }
             janet_fiber_popframe(fiber);
-            if (entrance_frame)
+            if (entrance_frame) {
-                vm_return(JANET_SIGNAL_OK, retreg);
+                vm_return_no_restore(JANET_SIGNAL_OK, retreg);
+            }
             vm_restore();
             stack[A] = retreg;
             vm_checkgc_pcnext();
@@ -1040,6 +1064,25 @@ static JanetSignal run_vm(JanetFiber *fiber, Janet in) {
         vm_return((int) sub_status, stack[B]);
     }
 
+    VM_OP(JOP_CANCEL) {
+        Janet retreg;
+        vm_assert_type(stack[B], JANET_FIBER);
+        JanetFiber *child = janet_unwrap_fiber(stack[B]);
+        if (janet_check_can_resume(child, &retreg)) {
+            vm_commit();
+            janet_panicv(retreg);
+        }
+        fiber->child = child;
+        JanetSignal sig = janet_continue_signal(child, stack[C], &retreg, JANET_SIGNAL_ERROR);
+        if (sig != JANET_SIGNAL_OK && !(child->flags & (1 << sig))) {
+            vm_return(sig, retreg);
+        }
+        fiber->child = NULL;
+        stack = fiber->data + fiber->frame;
+        stack[A] = retreg;
+        vm_checkgc_pcnext();
+    }
+
     VM_OP(JOP_PUT)
     vm_commit();
     fiber->flags |= JANET_FIBER_RESUME_NO_USEVAL;
@@ -1357,6 +1400,20 @@ JanetSignal janet_continue(JanetFiber *fiber, Janet in, Janet *out) {
     return janet_continue_no_check(fiber, in, out);
 }
 
+/* Enter the main vm loop but immediately raise a signal */
+JanetSignal janet_continue_signal(JanetFiber *fiber, Janet in, Janet *out, JanetSignal sig) {
+    JanetSignal tmp_signal = janet_check_can_resume(fiber, out);
+    if (tmp_signal) return tmp_signal;
+    if (sig != JANET_SIGNAL_OK) {
+        JanetFiber *child = fiber;
+        while (child->child) child = child->child;
+        child->gc.flags &= ~JANET_FIBER_STATUS_MASK;
+        child->gc.flags |= sig << JANET_FIBER_STATUS_OFFSET;
+        child->flags |= JANET_FIBER_RESUME_SIGNAL;
+    }
+    return janet_continue_no_check(fiber, in, out);
+}
+
 JanetSignal janet_pcall(
     JanetFunction *fun,
     int32_t argc,
@@ -1394,11 +1451,8 @@ int janet_init(void) {
     /* Garbage collection */
     janet_vm_blocks = NULL;
     janet_vm_next_collection = 0;
-    /* Setting memoryInterval to zero forces
+    janet_vm_gc_interval = 0x400000;
-     * a collection pretty much every cycle, which is
+    janet_vm_block_count = 0;
-     * incredibly horrible for performance, but can help ensure
-     * there are no memory bugs during development */
-    janet_vm_gc_interval = 0x10000;
     janet_symcache_init();
     /* Initialize gc roots */
     janet_vm_roots = NULL;
@@ -1419,6 +1473,8 @@ int janet_init(void) {
     janet_vm_traversal_top = NULL;
     /* Core env */
     janet_vm_core_env = NULL;
+    /* Dynamic bindings */
+    janet_vm_top_dyns = NULL;
     /* Seed RNG */
     janet_rng_seed(janet_default_rng(), 0);
     /* Fibers */
@@ -1443,6 +1499,7 @@ void janet_deinit(void) {
     janet_vm_registry = NULL;
     janet_vm_abstract_registry = NULL;
     janet_vm_core_env = NULL;
+    janet_vm_top_dyns = NULL;
     free(janet_vm_traversal_base);
     janet_vm_fiber = NULL;
     janet_vm_root_fiber = NULL;

src/include/janet.h

@@ -127,6 +127,12 @@ extern "C" {
 #define JANET_LITTLE_ENDIAN 1
 #endif
 
+/* Limits for converting doubles to 64 bit integers */
+#define JANET_INTMAX_DOUBLE 9007199254740992.0
+#define JANET_INTMIN_DOUBLE (-9007199254740992.0)
+#define JANET_INTMAX_INT64 9007199254740992
+#define JANET_INTMIN_INT64 (-9007199254740992)
+
 /* Check emscripten */
 #ifdef __EMSCRIPTEN__
 #define JANET_NO_DYNAMIC_MODULES
@@ -138,11 +144,6 @@ extern "C" {
 #define JANET_NO_UTC_MKTIME
 #endif
 
-/* Add some windows flags */
-#ifdef JANET_WINDOWS
-#define JANET_NO_REALPATH
-#endif
-
 /* Define how global janet state is declared */
 #ifdef JANET_SINGLE_THREADED
 #define JANET_THREAD_LOCAL
@@ -224,6 +225,11 @@ extern "C" {
  * To turn of nanboxing, for debugging purposes or for certain
  * architectures (Nanboxing only tested on x86 and x64), comment out
  * the JANET_NANBOX define.*/
+
+#if defined(_M_ARM64) || defined(_M_ARM) || defined(__aarch64__)
+#define JANET_NO_NANBOX
+#endif
+
 #ifndef JANET_NO_NANBOX
 #ifdef JANET_32
 #define JANET_NANBOX_32
@@ -711,7 +717,7 @@ JANET_API int janet_checkint64(Janet x);
 JANET_API int janet_checksize(Janet x);
 JANET_API JanetAbstract janet_checkabstract(Janet x, const JanetAbstractType *at);
 #define janet_checkintrange(x) ((x) >= INT32_MIN && (x) <= INT32_MAX && (x) == (int32_t)(x))
-#define janet_checkint64range(x) ((x) >= INT64_MIN && (x) <= INT64_MAX && (x) == (int64_t)(x))
+#define janet_checkint64range(x) ((x) >= JANET_INTMIN_DOUBLE && (x) <= JANET_INTMAX_DOUBLE && (x) == (int64_t)(x))
 #define janet_unwrap_integer(x) ((int32_t) janet_unwrap_number(x))
 #define janet_wrap_integer(x) janet_wrap_number((int32_t)(x))
 
@@ -1001,7 +1007,7 @@ struct JanetRNG {
 typedef struct JanetFile JanetFile;
 struct JanetFile {
     FILE *file;
-    int flags;
+    int32_t flags;
 };
 
 /* Thread types */
@@ -1123,6 +1129,9 @@ enum JanetOpCode {
     JOP_GREATER_THAN_EQUAL,
     JOP_LESS_THAN_EQUAL,
     JOP_NEXT,
+    JOP_NOT_EQUALS,
+    JOP_NOT_EQUALS_IMMEDIATE,
+    JOP_CANCEL,
     JOP_INSTRUCTION_COUNT
 };
 
@@ -1307,6 +1316,7 @@ JANET_API void janet_table_merge_table(JanetTable *table, JanetTable *other);
 JANET_API void janet_table_merge_struct(JanetTable *table, JanetStruct other);
 JANET_API JanetKV *janet_table_find(JanetTable *t, Janet key);
 JANET_API JanetTable *janet_table_clone(JanetTable *table);
+JANET_API void janet_table_clear(JanetTable *table);
 
 /* Fiber */
 JANET_API JanetFiber *janet_fiber(JanetFunction *callee, int32_t capacity, int32_t argc, const Janet *argv);
@@ -1377,7 +1387,7 @@ JANET_API int janet_verify(JanetFuncDef *def);
 JANET_API JanetBuffer *janet_pretty(JanetBuffer *buffer, int depth, int flags, Janet x);
 
 /* Misc */
-#ifndef JANET_NO_PRF
+#ifdef JANET_PRF
 #define JANET_HASH_KEY_SIZE 16
 JANET_API void janet_init_hash_key(uint8_t key[JANET_HASH_KEY_SIZE]);
 #endif
@@ -1403,6 +1413,7 @@ JANET_API int janet_symeq(Janet x, const char *cstring);
 JANET_API int janet_init(void);
 JANET_API void janet_deinit(void);
 JANET_API JanetSignal janet_continue(JanetFiber *fiber, Janet in, Janet *out);
+JANET_API JanetSignal janet_continue_signal(JanetFiber *fiber, Janet in, Janet *out, JanetSignal sig);
 JANET_API JanetSignal janet_pcall(JanetFunction *fun, int32_t argn, const Janet *argv, Janet *out, JanetFiber **f);
 JANET_API JanetSignal janet_step(JanetFiber *fiber, Janet in, Janet *out);
 JANET_API Janet janet_call(JanetFunction *fun, int32_t argc, const Janet *argv);
@@ -1529,12 +1540,16 @@ extern JANET_API const JanetAbstractType janet_file_type;
 #define JANET_FILE_BINARY 64
 #define JANET_FILE_SERIALIZABLE 128
 #define JANET_FILE_PIPED 256
+#define JANET_FILE_NONIL 512
 
-JANET_API Janet janet_makefile(FILE *f, int flags);
+JANET_API Janet janet_makefile(FILE *f, int32_t flags);
-JANET_API FILE *janet_getfile(const Janet *argv, int32_t n, int *flags);
+JANET_API FILE *janet_getfile(const Janet *argv, int32_t n, int32_t *flags);
 JANET_API FILE *janet_dynfile(const char *name, FILE *def);
+JANET_API JanetFile *janet_getjfile(const Janet *argv, int32_t n);
 JANET_API JanetAbstract janet_checkfile(Janet j);
-JANET_API FILE *janet_unwrapfile(Janet j, int *flags);
+JANET_API FILE *janet_unwrapfile(Janet j, int32_t *flags);
+
+JANET_API int janet_cryptorand(uint8_t *out, size_t n);
 
 /* Marshal API */
 JANET_API void janet_marshal_size(JanetMarshalContext *ctx, size_t value);
@@ -1587,6 +1602,8 @@ typedef enum {
     RULE_ERROR,        /* [rule] */
     RULE_DROP,         /* [rule] */
     RULE_BACKMATCH,    /* [tag] */
+    RULE_TO,           /* [rule] */
+    RULE_THRU,         /* [rule] */
     RULE_LENPREFIX,    /* [rule_a, rule_b (repeat rule_b rule_a times)] */
 } JanetPegOpcode;
 

src/mainclient/shell.c

@@ -40,6 +40,8 @@ void janet_line_deinit();
 void janet_line_get(const char *p, JanetBuffer *buffer);
 Janet janet_line_getter(int32_t argc, Janet *argv);
 
+static JANET_THREAD_LOCAL int gbl_cancel_current_repl_form = 0;
+
 /*
  * Line Editing
  */
@@ -54,7 +56,17 @@ Janet janet_line_getter(int32_t argc, Janet *argv) {
     gbl_complete_env = (argc >= 3) ? janet_gettable(argv, 2) : NULL;
     janet_line_get(str, buf);
     gbl_complete_env = NULL;
-    return janet_wrap_buffer(buf);
+
+    Janet result;
+    if (gbl_cancel_current_repl_form) {
+        gbl_cancel_current_repl_form = 0;
+
+        /* Signal that the user bailed out of the current form */
+        result = janet_ckeywordv("cancel");
+    } else {
+        result = janet_wrap_buffer(buf);
+    }
+    return result;
 }
 
 static void simpleline(JanetBuffer *buffer) {
@@ -72,7 +84,7 @@ static void simpleline(JanetBuffer *buffer) {
 }
 
 /* Windows */
-#ifdef JANET_WINDOWS
+#if defined(JANET_WINDOWS) || defined(JANET_SIMPLE_GETLINE)
 
 void janet_line_init() {
     ;
@@ -114,21 +126,28 @@ https://github.com/antirez/linenoise/blob/master/linenoise.c
 #define JANET_LINE_MAX 1024
 #define JANET_MATCH_MAX 256
 #define JANET_HISTORY_MAX 100
-static JANET_THREAD_LOCAL int gbl_israwmode = 0;
+static int gbl_israwmode = 0;
-static JANET_THREAD_LOCAL const char *gbl_prompt = "> ";
+static const char *gbl_prompt = "> ";
-static JANET_THREAD_LOCAL int gbl_plen = 2;
+static int gbl_plen = 2;
-static JANET_THREAD_LOCAL char gbl_buf[JANET_LINE_MAX];
+static char gbl_buf[JANET_LINE_MAX];
-static JANET_THREAD_LOCAL int gbl_len = 0;
+static int gbl_len = 0;
-static JANET_THREAD_LOCAL int gbl_pos = 0;
+static int gbl_pos = 0;
-static JANET_THREAD_LOCAL int gbl_cols = 80;
+static int gbl_cols = 80;
-static JANET_THREAD_LOCAL char *gbl_history[JANET_HISTORY_MAX];
+static char *gbl_history[JANET_HISTORY_MAX];
-static JANET_THREAD_LOCAL int gbl_history_count = 0;
+static int gbl_history_count = 0;
-static JANET_THREAD_LOCAL int gbl_historyi = 0;
+static int gbl_historyi = 0;
-static JANET_THREAD_LOCAL int gbl_sigint_flag = 0;
+static int gbl_sigint_flag = 0;
-static JANET_THREAD_LOCAL struct termios gbl_termios_start;
+static struct termios gbl_termios_start;
-static JANET_THREAD_LOCAL JanetByteView gbl_matches[JANET_MATCH_MAX];
+static JanetByteView gbl_matches[JANET_MATCH_MAX];
-static JANET_THREAD_LOCAL int gbl_match_count = 0;
+static int gbl_match_count = 0;
-static JANET_THREAD_LOCAL int gbl_lines_below = 0;
+static int gbl_lines_below = 0;
+
+/* Put a lock around this global state so we don't screw up
+ * the terminal in a multithreaded situation */
+#ifndef JANET_SINGLE_THREADED
+#include <pthread.h>
+static pthread_mutex_t gbl_lock = PTHREAD_MUTEX_INITIALIZER;
+#endif
 
 /* Unsupported terminal list from linenoise */
 static const char *badterms[] = {
@@ -150,6 +169,9 @@ static char *sdup(const char *s) {
 /* Ansi terminal raw mode */
 static int rawmode(void) {
     struct termios t;
+#ifndef JANET_SINGLE_THREADED
+    pthread_mutex_lock(&gbl_lock);
+#endif
     if (!isatty(STDIN_FILENO)) goto fatal;
     if (tcgetattr(STDIN_FILENO, &gbl_termios_start) == -1) goto fatal;
     t = gbl_termios_start;
@@ -158,18 +180,24 @@ static int rawmode(void) {
     t.c_lflag &= ~(ECHO | ICANON | IEXTEN | ISIG);
     t.c_cc[VMIN] = 1;
     t.c_cc[VTIME] = 0;
-    if (tcsetattr(STDIN_FILENO, TCSAFLUSH, &t) < 0) goto fatal;
+    if (tcsetattr(STDIN_FILENO, TCSADRAIN, &t) < 0) goto fatal;
     gbl_israwmode = 1;
     return 0;
 fatal:
     errno = ENOTTY;
+#ifndef JANET_SINGLE_THREADED
+    pthread_mutex_unlock(&gbl_lock);
+#endif
     return -1;
 }
 
 /* Disable raw mode */
 static void norawmode(void) {
-    if (gbl_israwmode && tcsetattr(STDIN_FILENO, TCSAFLUSH, &gbl_termios_start) != -1)
+    if (gbl_israwmode && tcsetattr(STDIN_FILENO, TCSADRAIN, &gbl_termios_start) != -1)
         gbl_israwmode = 0;
+#ifndef JANET_SINGLE_THREADED
+    pthread_mutex_unlock(&gbl_lock);
+#endif
 }
 
 static int curpos(void) {
@@ -746,8 +774,7 @@ static int line() {
                 kleft();
                 break;
             case 3:     /* ctrl-c */
-                errno = EAGAIN;
+                gbl_cancel_current_repl_form = 1;
-                gbl_sigint_flag = 1;
                 clearlines();
                 return -1;
             case 4:     /* ctrl-d, eof */
@@ -985,6 +1012,28 @@ int main(int argc, char **argv) {
     SetConsoleOutputCP(65001);
 #endif
 
+#if !defined(JANET_WINDOWS) && !defined(JANET_SIMPLE_GETLINE)
+    /* Try and not leave the terminal in a bad state */
+    atexit(norawmode);
+#endif
+
+#if defined(JANET_PRF)
+    uint8_t hash_key[JANET_HASH_KEY_SIZE + 1];
+#ifdef JANET_REDUCED_OS
+    char *envvar = NULL;
+#else
+    char *envvar = getenv("JANET_HASHSEED");
+#endif
+    if (NULL != envvar) {
+        strncpy((char *) hash_key, envvar, sizeof(hash_key) - 1);
+    } else if (janet_cryptorand(hash_key, JANET_HASH_KEY_SIZE) != 0) {
+        fputs("unable to initialize janet PRF hash function.\n", stderr);
+        return 1;
+    }
+    janet_init_hash_key(hash_key);
+#endif
+
+
     /* Set up VM */
     janet_init();
 

test/suite0000.janet

@@ -334,5 +334,88 @@
 (assert (deep= @{:a 3 :b 2} @{:a 1 :b 2 :a 3}) "table literal duplicate keys")
 (assert (deep= @{:a 3 :b 2} (table :a 1 :b 2 :a 3)) "table constructor duplicate keys")
 
+## Polymorphic comparison -- Issue #272
+
+# confirm polymorphic comparison delegation to primitive comparators:
+(assert (= 0 (cmp 3 3)) "compare-primitive integers (1)")
+(assert (= -1 (cmp 3 5)) "compare-primitive integers (2)")
+(assert (= 1 (cmp "foo" "bar")) "compare-primitive strings")
+(assert (= 0 (compare 1 1)) "compare integers (1)")
+(assert (= -1 (compare 1 2)) "compare integers (2)")
+(assert (= 1 (compare "foo" "bar")) "compare strings (1)")
+
+(assert (compare< 1 2 3 4 5 6) "compare less than integers")
+(assert (not (compare> 1 2 3 4 5 6)) "compare not greater than integers")
+(assert (compare< 1.0 2.0 3.0 4.0 5.0 6.0) "compare less than reals")
+(assert (compare> 6 5 4 3 2 1) "compare greater than integers")
+(assert (compare> 6.0 5.0 4.0 3.0 2.0 1.0) "compare greater than reals")
+(assert (not (compare< 6.0 5.0 4.0 3.0 2.0 1.0)) "compare less than reals")
+(assert (compare<= 1 2 3 3 4 5 6) "compare less than or equal to integers")
+(assert (compare<= 1.0 2.0 3.0 3.0 4.0 5.0 6.0) "compare less than or equal to reals")
+(assert (compare>= 6 5 4 4 3 2 1) "compare greater than or equal to integers")
+(assert (compare>= 6.0 5.0 4.0 4.0 3.0 2.0 1.0) "compare greater than or equal to reals")
+(assert (compare< 1.0 nil false true
+           (fiber/new (fn [] 1))
+           "hi"
+           (quote hello)
+           :hello
+           (array 1 2 3)
+           (tuple 1 2 3)
+           (table "a" "b" "c" "d")
+           (struct 1 2 3 4)
+           (buffer "hi")
+           (fn [x] (+ x x))
+           print) "compare type ordering")
+
+# test polymorphic compare with 'objects' (table/setproto)
+(def mynum
+  @{:type :mynum :v 0 :compare
+    (fn [self other]
+      (case (type other)
+      :number (cmp (self :v) other)
+      :table (when (= (get other :type) :mynum)
+               (cmp (self :v) (other :v)))))})
+
+(let [n3 (table/setproto @{:v 3} mynum)]
+  (assert (= 0 (compare 3 n3)) "compare num to object (1)")
+  (assert (= -1 (compare n3 4)) "compare object to num (2)")
+  (assert (= 1 (compare (table/setproto @{:v 4} mynum) n3)) "compare object to object")
+  (assert (compare< 2 n3 4) "compare< poly")
+  (assert (compare> 4 n3 2) "compare> poly")
+  (assert (compare<= 2 3 n3 4) "compare<= poly")
+  (assert (compare= 3 n3 (table/setproto @{:v 3} mynum)) "compare= poly")
+  (assert (deep= (sorted @[4 5 n3 2] compare<) @[2 n3 4 5]) "polymorphic sort"))
+
+(let [MAX_INT_64_STRING "9223372036854775807"
+      MAX_UINT_64_STRING "18446744073709551615"
+      MAX_INT_IN_DBL_STRING "9007199254740991"
+      NAN (math/log -1)
+      INF (/ 1 0)
+      MINUS_INF (/ -1 0)
+      compare-poly-tests
+      [[(int/s64 3) (int/u64 3) 0]
+       [(int/s64 -3) (int/u64 3) -1]
+       [(int/s64 3) (int/u64 2) 1] 
+       [(int/s64 3) 3 0] [(int/s64 3) 4 -1] [(int/s64 3) -9 1]
+       [(int/u64 3) 3 0] [(int/u64 3) 4 -1] [(int/u64 3) -9 1]
+       [3 (int/s64 3) 0] [3 (int/s64 4) -1] [3 (int/s64 -5) 1]
+       [3 (int/u64 3) 0] [3 (int/u64 4) -1] [3 (int/u64 2) 1]
+       [(int/s64 MAX_INT_64_STRING) (int/u64 MAX_UINT_64_STRING) -1]
+       [(int/s64 MAX_INT_IN_DBL_STRING) (scan-number MAX_INT_IN_DBL_STRING) 0]
+       [(int/u64 MAX_INT_IN_DBL_STRING) (scan-number MAX_INT_IN_DBL_STRING) 0]
+       [(+ 1 (int/u64 MAX_INT_IN_DBL_STRING)) (scan-number MAX_INT_IN_DBL_STRING) 1]
+       [(int/s64 0) INF -1] [(int/u64 0) INF -1]
+       [MINUS_INF (int/u64 0) -1] [MINUS_INF (int/s64 0) -1] 
+       [(int/s64 1) NAN 0] [NAN (int/u64 1) 0]]]
+  (each [x y c] compare-poly-tests
+    (assert (= c (compare x y)) (string/format "compare polymorphic %q %q %d" x y c))))
+
+(assert (= nil (any? [])) "any? 1")
+(assert (= nil (any? [false nil])) "any? 2")
+(assert (= nil (any? [nil false])) "any? 3")
+(assert (= 1 (any? [1])) "any? 4")
+(assert (nan? (any? [nil math/nan nil])) "any? 5")
+(assert (= true (any? [nil nil false nil nil true nil nil nil nil false :a nil])) "any? 6")
+
 (end-suite)
 

test/suite0003.janet

@@ -91,8 +91,8 @@
 # Assembly test
 # Fibonacci sequence, implemented with naive recursion.
 (def fibasm (asm '{
-  arity 1
+  :arity 1
-  bytecode [
+  :bytecode [
     (ltim 1 0 0x2)      # $1 = $0 < 2
     (jmpif 1 :done)     # if ($1) goto :done
     (lds 1)             # $1 = self

test/suite0004.janet

@@ -48,7 +48,9 @@
 (defn check-image
   "Run a marshaling test using the make-image and load-image functions."
   [x msg]
-  (assert-no-error msg (load-image (make-image x))))
+  (def im (make-image x))
+  # (printf "\nimage-hash: %d" (-> im string hash))
+  (assert-no-error msg (load-image im)))
 
 (check-image (fn [] (fn [] 1)) "marshal nested functions")
 (check-image (fiber/new (fn [] (fn [] 1))) "marshal nested functions in fiber")

test/suite0005.janet

@@ -58,6 +58,17 @@
 
 (assert (= ((unmarshal (marshal b)) 3) (b 3)) "marshal")
 
+# Issue 408
+(assert-error :invalid-type (tarray/new :int32 10 1 0 (int/u64 7)) "tarray/new should only allow tarray or buffer for last argument")
+(def ta (tarray/new :int32 10))
+(assert (= (next a nil) 0) "tarray next 1")
+(assert (= (next a 0) 1) "tarray next 2")
+(assert (= (next a 8) 9) "tarray next 3")
+(assert (nil? (next a 9)) "tarray next 4")
+(put ta 3 7)
+(put ta 9 7)
+(assert (= 2 (count |(= $ 7) ta)) "tarray count")
+
 # Array remove
 
 (assert (deep= (array/remove @[1 2 3 4 5] 2) @[1 2 4 5]) "array/remove 1")

test/suite0007.janet

@@ -23,7 +23,6 @@
 
 # Using a large test grammar
 
-(def- core-env (table/getproto (fiber/getenv (fiber/current))))
 (def- specials {'fn true
                'var true
                'do true
@@ -41,7 +40,7 @@
 (defn capture-sym
   [text]
   (def sym (symbol text))
-  [(if (or (core-env sym) (specials sym)) :coresym :symbol) text])
+  [(if (or (root-env sym) (specials sym)) :coresym :symbol) text])
 
 (def grammar
   ~{:ws (set " \v\t\r\f\n\0")
@@ -316,4 +315,9 @@
 
 (assert (= 40 counter) "if-with 1")
 
+(def a @[])
+(eachk x [:a :b :c :d]
+  (array/push a x))
+(assert (deep= (range 4) a) "eachk 1")
+
 (end-suite)

test/suite0008.janet

@@ -36,7 +36,7 @@
       :loop (/ (* "[" :main "]") ,(fn [& captures]
                                     ~(while (not= (get DATA POS) 0)
                                        ,;captures)))
-      :main (any (+ :s :loop :+ :- :> :< :.)) }))
+      :main (any (+ :s :loop :+ :- :> :< :.))}))
 
 (defn bf
   "Run brainfuck."
@@ -233,8 +233,8 @@ neldb\0\0\0\xD8\x05printG\x01\0\xDE\xDE\xDE'\x03\0marshal_tes/\x02
 (gccollect)
 
 (def v (unmarshal
-  @"\xD7\xCD0\xD4000000\0\x03\x01\xCE\00\0\x01\0\0000\x03\0\0\0000000000\xCC0\0000"
+         @"\xD7\xCD0\xD4000000\0\x03\x01\xCE\00\0\x01\0\0000\x03\0\0\0000000000\xCC0\0000"
-  load-image-dict))
+         load-image-dict))
 (gccollect)
 
 # in vs get regression
@@ -271,7 +271,7 @@ neldb\0\0\0\xD8\x05printG\x01\0\xDE\xDE\xDE'\x03\0marshal_tes/\x02
       :packet-body '(lenprefix (-> :header-len) 1)
 
       # header, followed by body, and drop the :header-len capture
-      :packet (/ (* :packet-header :packet-body) ,|$1) 
+      :packet (/ (* :packet-header :packet-body) ,|$1)
 
       # any exact seqence of packets (no extra characters)
       :main (* (any :packet) -1)}))
@@ -281,4 +281,72 @@ neldb\0\0\0\xD8\x05printG\x01\0\xDE\xDE\xDE'\x03\0marshal_tes/\x02
 (assert (= nil (peg/match peg2 "1:a2:bb:5:cccccc")) "lenprefix 6")
 (assert (= nil (peg/match peg2 "1:a2:bb:7:cccccc")) "lenprefix 7")
 
+# Regression #400
+(assert (= nil (while (and false false) (fn []) (error "should not happen"))) "strangeloop 1")
+(assert (= nil (while (not= nil nil) (fn []) (error "should not happen"))) "strangeloop 2")
+
+# Issue #412
+(assert (peg/match '(* "a" (> -1 "a") "b") "abc") "lookhead does not move cursor")
+
+(def peg3
+  ~{:main (* "(" (thru ")"))})
+
+(def peg4 (peg/compile ~(* (thru "(") '(to ")"))))
+
+(assert (peg/match peg3 "(12345)") "peg thru 1")
+(assert (not (peg/match peg3 " (12345)")) "peg thru 2")
+(assert (not (peg/match peg3 "(12345")) "peg thru 3")
+
+(assert (= "abc" (0 (peg/match peg4 "123(abc)"))) "peg thru/to 1")
+(assert (= "abc" (0 (peg/match peg4 "(abc)"))) "peg thru/to 2")
+(assert (not (peg/match peg4 "123(abc")) "peg thru/to 3")
+
+(def peg5 (peg/compile [3 "abc"]))
+
+(assert (:match peg5 "abcabcabc") "repeat alias 1")
+(assert (:match peg5 "abcabcabcac") "repeat alias 2")
+(assert (not (:match peg5 "abcabc")) "repeat alias 3")
+
+(defn check-jdn [x]
+  (assert (deep= (parse (string/format "%j" x)) x) "round trip jdn"))
+
+(check-jdn 0)
+(check-jdn nil)
+(check-jdn [])
+(check-jdn @[[] [] 1231 9.123123 -123123 0.1231231230001])
+(check-jdn -0.123123123123)
+(check-jdn 12837192371923)
+(check-jdn "a string")
+(check-jdn @"a buffer")
+
+# Issue 428
+(var result nil)
+(defn f [] (yield {:a :ok}))
+(assert-no-error "issue 428 1" (loop [{:a x} :generate (fiber/new f)] (set result x)))
+(assert (= result :ok) "issue 428 2")
+
+# Inline 3 argument get
+(assert (= 10 (do (var a 10) (set a (get '{} :a a)))) "inline get 1")
+
+# Keyword and Symbol slice
+(assert (= :keyword (keyword/slice "some_keyword_slice" 5 12)) "keyword slice")
+(assert (= 'symbol (symbol/slice "some_symbol_slice" 5 11)) "symbol slice")
+
+# Peg find and find-all
+(def p "/usr/local/bin/janet")
+(assert (= (peg/find '"n/" p) 13) "peg find 1")
+(assert (not (peg/find '"t/" p)) "peg find 2")
+(assert (deep= (peg/find-all '"/" p) @[0 4 10 14]) "peg find-all")
+
+# Peg replace and replace-all
+(var ti 0)
+(defn check-replacer
+  [x y z]
+  (assert (= (string/replace x y z) (string (peg/replace x y z))) "replacer test replace")
+  (assert (= (string/replace-all x y z) (string (peg/replace-all x y z))) "replacer test replace-all"))
+(check-replacer "abc" "Z" "abcabcabcabasciabsabc")
+(check-replacer "abc" "Z" "")
+(check-replacer "aba" "ZZZZZZ" "ababababababa")
+(check-replacer "aba" "" "ababababababa")
+
 (end-suite)

test/suite0010.janet

@@ -0,0 +1,64 @@
+# Copyright (c) 2020 Calvin Rose & contributors
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to
+# deal in the Software without restriction, including without limitation the
+# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+# sell copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+
+(import ./helper :prefix "" :exit true)
+(start-suite 10)
+
+# index-of
+(assert (= nil (index-of 10 [])) "index-of 1")
+(assert (= nil (index-of 10 [1 2 3])) "index-of 2")
+(assert (= 1 (index-of 2 [1 2 3])) "index-of 3")
+(assert (= 0 (index-of :a [:a :b :c])) "index-of 4")
+(assert (= nil (index-of :a {})) "index-of 5")
+(assert (= :a (index-of :A {:a :A :b :B})) "index-of 6")
+(assert (= :a (index-of :A @{:a :A :b :B})) "index-of 7")
+(assert (= 0 (index-of (chr "a") "abc")) "index-of 8")
+(assert (= nil (index-of (chr "a") "")) "index-of 9")
+(assert (= nil (index-of 10 @[])) "index-of 10")
+(assert (= nil (index-of 10 @[1 2 3])) "index-of 11")
+
+# Regression
+(assert (= {:x 10} (|(let [x $] ~{:x ,x}) 10)) "issue 463")
+
+# macex testing
+(assert (deep= (macex1 '~{1 2 3 4}) '~{1 2 3 4}) "macex1 qq struct")
+(assert (deep= (macex1 '~@{1 2 3 4}) '~@{1 2 3 4}) "macex1 qq table")
+(assert (deep= (macex1 '~(1 2 3 4)) '~[1 2 3 4]) "macex1 qq tuple")
+(assert (= :brackets (tuple/type (1 (macex1 '~[1 2 3 4])))) "macex1 qq bracket tuple")
+(assert (deep= (macex1 '~@[1 2 3 4 ,blah]) '~@[1 2 3 4 ,blah]) "macex1 qq array")
+
+# Cancel test
+(def f (fiber/new (fn [&] (yield 1) (yield 2) (yield 3) 4) :yti))
+(assert (= 1 (resume f)) "cancel resume 1")
+(assert (= 2 (resume f)) "cancel resume 2")
+(assert (= :hi (cancel f :hi)) "cancel resume 3")
+(assert (= :error (fiber/status f)) "cancel resume 4")
+
+# Curenv
+(assert (= (curenv) (curenv 0)) "curenv 1")
+(assert (= (table/getproto (curenv)) (curenv 1)) "curenv 2")
+(assert (= nil (curenv 1000000)) "curenv 3")
+(assert (= root-env (curenv 1)) "curenv 4")
+
+# Import macro test
+(assert-no-error "import macro 1" (macex '(import a :as b :fresh maybe)))
+(assert (deep= ~(,import* "a" :as "b" :fresh maybe) (macex '(import a :as b :fresh maybe))) "import macro 2")
+
+(end-suite)

tools/msi/janet.wxs

@@ -0,0 +1,200 @@
+<?xml version="1.0" encoding="UTF-8"?>
+
+<?define Name = "Janet" ?>
+<?define Description = "The Janet Programming Language" ?>
+<?define Manufacturer = "Janet-Lang.org" ?>
+<?define WebPage = "https://janet-lang.org" ?>
+<?ifdef env.JANET_VERSION ?>
+    <?define Version = "$(env.JANET_VERSION)" ?>
+<?else?>
+    <?define Version = "0.0.0" ?>
+<?endif?>
+<?if $(sys.BUILDARCH)="x64" ?>
+    <?define UpgradeCode="712CACD6-09AA-430A-831C-80FDFFE3F9ED" ?>
+    <?define ProgramFilesFolder="ProgramFiles64Folder" ?>
+    <?define Win64="yes" ?>
+    <?define Arch="(x64)" ?>
+<?elseif $(sys.BUILDARCH)="x86" ?>
+    <?define UpgradeCode="0411837a-82c4-4dc7-872b-134d0c1b0228" ?>
+    <?define ProgramFilesFolder="ProgramFilesFolder" ?>
+    <?define Win64="no" ?>
+    <?define Arch="(x86)" ?>
+<?else ?>
+    <?error Unsupported value of sys.BUILDARCH=$(sys.BUILDARCH)?>
+<?endif?>
+<?define BaseRegKey="Software\Microsoft\$(var.Manufacturer)\$(var.Name)" ?>
+
+
+<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi">
+    <Product Id="*"
+			 Name="$(var.Name)"
+			 Language="1033"
+			 Version="$(var.Version)"
+			 Manufacturer="$(var.Manufacturer)"
+			 UpgradeCode="$(var.UpgradeCode)">
+        <Package Compressed="yes"
+				 InstallScope="perUser"
+				 Manufacturer="$(var.Manufacturer)"
+				 Description="$(var.Description)" />
+        <MajorUpgrade DowngradeErrorMessage="A later version of [ProductName] is already installed. Setup will now exit."/>
+        <MediaTemplate EmbedCab="yes" />
+        <Property Id="DISABLEADVTSHORTCUTS" Value="1" />
+
+        <!-- Set UI images (use the -b option to light.exe to set where these files are) -->
+        <WixVariable Id="WixUIBannerBmp" Value="JanetTopBanner.png" />
+        <WixVariable Id="WixUIDialogBmp" Value="JanetDialog.png" />
+        <WixVariable Id="WixUILicenseRtf" Value="LICENSE.rtf" />
+
+        <Icon Id="Janet.ico" SourceFile="assets\icon.ico" />
+
+        <!-- Add some details to Add/Remove Programs entry -->
+        <Property Id="ARPPRODUCTICON" Value="Janet.ico" />
+        <Property Id='ARPCOMMENTS'>$(var.Description)</Property>
+        <Property Id='ARPURLINFOABOUT'>$(var.WebPage)</Property>
+        <Property Id='COMPANY'>$(var.Manufacturer)</Property>
+
+        <!-- Default to per-user installs -->
+        <Property Id="WixAppFolder" Value="WixPerUserFolder" />
+
+        <Property Id="ApplicationFolderName" Value="$(var.Name)" />
+
+        <!-- Fix WixUI_Advanced to work with x64 -->
+        <CustomAction Id="WixSetDefaultPerMachineFolderPerArch"
+            Property="WixPerMachineFolder"
+            Value="[$(var.ProgramFilesFolder)][ApplicationFolderName]"
+            Execute="immediate"/>
+        <InstallExecuteSequence>
+            <Custom Action="WixSetDefaultPerMachineFolderPerArch" Before="WixSetPerMachineFolder" />
+        </InstallExecuteSequence>
+        <InstallUISequence>
+            <Custom Action="WixSetDefaultPerMachineFolderPerArch" Before="WixSetPerMachineFolder" />
+        </InstallUISequence>
+
+        <Directory Id="TARGETDIR" Name="SourceDir">
+            <Directory Id="$(var.ProgramFilesFolder)">
+                <Directory Id="APPLICATIONFOLDER" Name="$(var.Name)">
+                    <Directory Id="BinDir" Name="bin"/>
+                    <Directory Id="CDir" Name="C"/>
+                    <Directory Id="DocsDir" Name="docs"/>
+                    <Directory Id="LibraryDir" Name="Library"/>
+                </Directory>
+            </Directory>
+            <Directory Id="ProgramMenuFolder">
+                <Directory Id="ApplicationProgramsFolder" Name="$(var.Name)" />
+            </Directory>
+        </Directory>
+
+        <!-- 
+          Define the files to be installed.
+          File/@Source is relative to where this file is compiled, the root of the repository in this case.
+          File/@Name is the destination file name, if not set it defaults to the file name part of Source.
+          Component/@Directory is the Id of the destination directory - where the directory name and
+            hierarchy is set in the section above
+        -->
+        <ComponentGroup Id="Files">
+            <Component Directory="APPLICATIONFOLDER">
+                <File Source="README.md"/>
+                <RemoveFolder Id="RemoveRootDir" On="uninstall" />
+            </Component>
+            <Component Directory="APPLICATIONFOLDER">
+                <File Source="LICENSE"/>
+            </Component>
+            <Component Directory="APPLICATIONFOLDER">
+                <File Source="assets\icon.ico"/>
+            </Component>
+
+            <Component Directory="BinDir">
+                <File Source="dist\janet.exe" KeyPath="yes">
+                    <Shortcut Id="JanetExeShortcut"
+                        Directory="ApplicationProgramsFolder"
+                        Name="$(var.Name)"
+                        Description="$(var.Description)"
+                        Icon="Janet.ico"
+                        Advertise="yes"
+                        WorkingDirectory="INSTALLFOLDER" />
+                </File>
+                <RemoveFolder Id="RemoveBinDir" On="uninstall" />
+            </Component>
+            <Component Directory="BinDir">
+                <File Source="jpm" Name="jpm.janet"/>
+            </Component>
+            <Component Directory="BinDir">
+                <File Source="tools\jpm.bat"/>
+            </Component>
+
+            <Component Directory="CDir">
+                <File Source="dist\janet.h"/>
+                <RemoveFolder Id="RemoveCDir" On="uninstall" />
+            </Component>
+            <Component Directory="CDir">
+                <File Source="dist\janetconf.h"/>
+            </Component>
+            <Component Directory="CDir">
+                <File Source="dist\janet.lib"/>
+            </Component>
+            <Component Directory="CDir">
+                <File Source="dist\janet.exp"/>
+            </Component>
+            <Component Directory="CDir">
+                <File Source="dist\janet.c"/>
+            </Component>
+            <Component Directory="CDir">
+                <File Source="dist\libjanet.lib"/>
+            </Component>
+
+            <Component Id="LibraryComponent" Directory="LibraryDir" Guid="3860e981-5f94-4002-b5d5-2d9ec0d2792d" KeyPath="yes">
+                <RemoveFolder Id="RemoveLibraryDir" On="uninstall" />
+            </Component>
+
+            <Component Id="DocsComponent" Directory="DocsDir">
+                <File Source="dist\doc.html" Name="docs.html" KeyPath="yes">
+                    <Shortcut Id="JanetDocsShortcut"
+                        Directory="ApplicationProgramsFolder"
+                        Name="$(var.Name) Documentation"
+                        Description="$(var.Description)"
+                        Advertise="yes"/>
+                </File>
+                <RemoveFolder Id="RemoveDocsDir" On="uninstall" />
+            </Component>
+        </ComponentGroup>
+
+        <Component Id="StartMenu" Directory="ApplicationProgramsFolder">
+            <RegistryValue Root="HKMU" Key="$(var.BaseRegKey)" Name="installed" Type="integer" Value="1" KeyPath="yes" />
+            <RemoveFolder Id="RemoveApplicationProgramsFolder" On="uninstall" />
+        </Component>
+
+        <!-- This component is duplicated with different conditions so that we can set system or user environment variables -->
+        <Component Id="SetEnvVarsPerMachine" Directory="ApplicationProgramsFolder" Guid="57b1e1ef-89c8-4ce4-9f0f-37618677c5a4" KeyPath="yes">
+            <Condition>ALLUSERS=1</Condition>
+            <Environment Id="PATH_PERMACHINE" Name="PATH" Value="[BinDir]" Action="set" Permanent="no" System="yes" Part="last"/>
+            <Environment Id="JANET_BINPATH_PERMACHINE" Name="JANET_BINPATH" Value="[BinDir]" Action="set" Permanent="no" System="yes"/>
+            <Environment Id="JANET_PATH_PERMACHINE" Name="JANET_PATH" Value="[LibraryDir]" Action="set" Permanent="no" System="yes" />
+            <Environment Id="JANET_HEADERPATH_PERMACHINE" Name="JANET_HEADERPATH" Value="[CDir]" Action="set" Permanent="no" System="yes"/>
+            <Environment Id="JANET_LIBPATH_PERMACHINE" Name="JANET_LIBPATH" Value="[CDir]" Action="set" Permanent="no" System="yes"/>
+        </Component>
+        <Component Id="SetEnvVarsPerUser" Directory="ApplicationProgramsFolder" Guid="128be307-488b-49aa-971a-d2ae00a1a584" KeyPath="yes">
+            <Condition>NOT ALLUSERS=1</Condition>
+            <Environment Id="PATH_PERUSER" Name="PATH" Value="[BinDir]" Action="set" Permanent="no" System="no" Part="last"/>
+            <Environment Id="JANET_BINPATH_PERUSER" Name="JANET_BINPATH" Value="[BinDir]" Action="set" Permanent="no" System="no"/>
+            <Environment Id="JANET_PATH_PERUSER" Name="JANET_PATH" Value="[LibraryDir]" Action="set" Permanent="no" System="no" />
+            <Environment Id="JANET_HEADERPATH_PERUSER" Name="JANET_HEADERPATH" Value="[CDir]" Action="set" Permanent="no" System="no"/>
+            <Environment Id="JANET_LIBPATH_PERUSER" Name="JANET_LIBPATH" Value="[CDir]" Action="set" Permanent="no" System="no"/>
+        </Component>
+
+        <Feature Id="MainFeature" Title="$(var.Name) $(var.Version)" 
+            Level="1" Absent="disallow" AllowAdvertise="no" InstallDefault="local"
+            Description="$(var.Description)">
+            <ComponentGroupRef Id="Files" />
+            <ComponentRef Id="StartMenu" />
+            <ComponentRef Id="SetEnvVarsPerMachine" />
+            <ComponentRef Id="SetEnvVarsPerUser" />
+        </Feature>
+
+        <UI>
+            <UIRef Id="WixUI_Advanced"/>
+            <!-- FindRelatedProducts runs before the user select the install scope, so we ask it to run again if the have changed the scope
+             -->
+            <Publish Dialog="InstallScopeDlg" Control="Next" Order="8" Event="DoAction" Value="FindRelatedProducts">WixAppFolder = "WixPerMachineFolder"</Publish>
+        </UI>
+    </Product>
+</Wix>

tools/patch-jpm.janet

@@ -0,0 +1,33 @@
+# Patch jpm to have the correct paths for the current install.
+# usage: janet patch-jpm.janet output --libdir=/usr/local/lib/x64-linux/ --binpath
+
+(def- argpeg
+  (peg/compile
+    '(* "--" '(to "=") "=" '(any 1))))
+
+(def- args (tuple/slice (dyn :args) 3))
+(def- len (length args))
+(var i :private 0)
+
+(def install-paths @{})
+
+# Get flags
+(each a args
+  (if-let [m (peg/match argpeg a)]
+    (let [[key value] m]
+      (put install-paths (keyword key) value))))
+
+(def- replace-peg
+  (peg/compile
+    ~(% (* '(to "###START###")
+           (constant ,(string/format "# Inserted by tools/patch-jpm.janet\n(defn- install-paths [] %j)" install-paths))
+           (thru "###END###")
+           '(any 1)))))
+
+(def source (slurp ((dyn :args) 1)))
+(def newsource (0 (peg/match replace-peg source)))
+
+(spit ((dyn :args) 2) newsource)
+
+(unless (= :windows (os/which))
+  (os/shell (string `chmod +x "` ((dyn :args) 2) `"`)))