Add root-env

This makes images smaller without needing to make sure
that no references to the root environment occur in the final
image.

.builds/.freebsd.yaml

@@ -1,11 +1,11 @@
-image: freebsd
+image: freebsd/latest
 packages:
   - gmake
-  - gcc
-sources:
-  - https://github.com/janet-lang/janet.git
 tasks:
   - build: |
       cd janet
-      gmake CC=gcc
-      gmake test CC=gcc
+      gmake
+      gmake test
+      sudo gmake install
+      gmake test-install
+      gmake test-amalg

.builds/.openbsd.yaml

@@ -0,0 +1,11 @@
+image: openbsd/6.5
+packages:
+  - gmake
+tasks:
+  - build: |
+      cd janet
+      gmake
+      gmake test
+      doas gmake install
+      gmake test-install
+      gmake test-amalg

.gitattributes

@@ -1,2 +0,0 @@
-# Use an approximate language for syntax highlighting (clojure is pretty close)
-*.janet linguist-language=clojure

.gitignore

@@ -4,6 +4,7 @@ dst
 janet
 !*/**/janet
 /build
+/builddir
 /Build
 /Release
 /Debug
@@ -12,6 +13,19 @@ janet
 janet-*.tar.gz
 dist
 
+# Kakoune (fzf via fd)
+.fdignore
+
+# VSCode
+.vscode
+
+# Eclipse
+.project
+.cproject
+
+# Gnome Builder
+.buildconfig
+
 # Local directory for testing
 local
 

.travis.yml

@@ -2,6 +2,9 @@ language: c
 script:
 - make
 - make test
+- sudo make install
+- make test-install
+- make test-amalg
 - make build/janet-${TRAVIS_TAG}-${TRAVIS_OS_NAME}.tar.gz
 compiler:
 - clang

CHANGELOG.md

@@ -1,7 +1,258 @@
 # Changelog
 All notable changes to this project will be documented in this file.
 
-## 0.3.0 - 2019-26-01
+### Unreleased
+- Allow seeding RNGs with any sequence of bytes. This provides
+  a wider key space for the RNG. Exposed in C as `janet_rng_longseed`.
+- Fix issue in `resume` and similar functions that could cause breakpoints to be skipped.
+- Add a number of new math functions.
+- Improve debugger experience and capabilities. See examples/debugger.janet
+  for what an interactive debugger could look like.
+- Add `debug/step` (janet\_step in the C API) for single stepping Janet bytecode.
+- The built in repl now can enter the debugger on any signal (errors, yields,
+  user signals, and debug signals). To enable this, type (setdyn :debug true)
+  in the repl environment.
+- When exiting the debugger, the fiber being debugged is resumed with the exit value
+  of the debug session (the value returned by `(quit return-value)`, or nil if user typed Ctrl-D).
+- `(quit)` can take an optional argument that is the return value. If a module
+  contains `(quit some-value)`, the value of that module returned to `(require "somemod")`
+  is the return value. This lets module writers completely customize a module without writing
+  a loader.
+- Add nested quasiquotation.
+- Add `os/cryptorand`
+- Add `prinf` and `eprinf` to be have like `printf` and `eprintf`. The latter two functions
+  now including a trailing newline, like the other print functions.
+- Add nan?
+- Add `janet_in` to C API.
+- Add `truthy?`
+- Add `os/environ`
+- Use `(doc)` with no arguments to see available bindings and dynamic bindings.
+- `jpm` will use `CC` and `AR` environment variables when compiling programs.
+- Add `comptime` macro for compile time evaluation.
+- Run `main` functions in scripts if they exist, just like jpm standalone binaries.
+- Add `protect` macro.
+- Add `root-env` to get the root environment table.
+- Change marshalling protocol with regard to abstract types.
+- Numerous small bug fixes and usability improvements.
+
+### 1.5.1 - 2019-11-16
+- Fix bug when printing buffer to self in some edge cases.
+- Fix bug with `jpm` on windows.
+- Fix `update` return value.
+
+## 1.5.0 - 2019-11-10
+- `os/date` now defaults to UTC.
+- Add `--test` flag to jpm to test libraries on installation.
+- Add `math/rng`, `math/rng-int`, and `math/rng-uniform`.
+- Add `in` function to index in a stricter manner. Conversely, `get` will
+  now not throw errors on bad keys.
+- Indexed types and byte sequences will now error when indexed out of range or
+  with bad keys.
+- Add rng functions to Janet. This also replaces the RNG behind `math/random`
+  and `math/seedrandom` with a consistent, platform independent RNG.
+- Add `with-vars` macro.
+- Add the `quickbin` command to jpm.
+- Create shell.c when making the amalgamated source. This can be compiled with
+  janet.c to make the janet interpreter.
+- Add `cli-main` function to the core, which invokes Janet's CLI interface.
+  This basically moves what was init.janet into boot.janet.
+- Improve flychecking, and fix flychecking bugs introduced in 1.4.0.
+- Add `prin`, `eprint`, `eprintf` and `eprin` functions. The
+  functions prefix with e print to `(dyn :err stderr)`
+- Print family of functions can now also print to buffers
+  (before, they could only print to files.) Output can also
+  be completely disabled with `(setdyn :out false)`.
+- `printf` is now a c function for optimizations in the case
+  of printing to buffers.
+
+## 1.4.0 - 2019-10-14
+- Add `quit` function to exit from a repl, but not always exit the entire
+  application.
+- Add `update-pkgs` to jpm.
+- Integrate jpm with https://github.com/janet-lang/pkgs.git. jpm can now
+  install packages based on their short names in the package listing, which
+  can be customized via an env variable.
+- Add `varfn` macro
+- Add compile time arity checking when function in function call is known.
+- Added `slice` to the core library.
+- The `*/slice` family of functions now can take nil as start or end to get
+  the same behavior as the defaults (0 and -1) for those parameters.
+- `string/` functions that take a pattern to search for will throw an error
+  when receiving the empty string.
+- Replace (start:end) style stacktrace source position information with
+  line, column. This should be more readable for humans. Also, range information
+  can be recovered by re-parsing source.
+
+## 1.3.1 - 2019-09-21
+- Fix some linking issues when creating executables with native dependencies.
+- jpm now runs each test script in a new interpreter.
+- Fix an issue that prevent some valid programs from compiling.
+- Add `mean` to core.
+- Abstract types that implement the `:+`, `:-`, `:*`, `:/`, `:>`, `:==`, `:<`,
+  `:<=`, and `:>=` methods will work with the corresponding built-in
+  arithmetic functions. This means built-in integer types can now be used as
+  normal number values in many contexts.
+- Allow (length x) on typed arrays an other abstract types that implement
+  the :length method.
+
+## 1.3.0 - 2019-09-05
+- Add `get-in`, `put-in`, `update-in`, and `freeze` to core.
+- Add `jpm run rule` and `jpm rules` to jpm to improve utility and discoverability of jpm.
+- Remove `cook` module and move `path` module to https://github.com/janet-lang/path.git.
+  The functionality in `cook` is now bundled directly in the `jpm` script.
+- Add `buffer/format` and `string/format` format flags `Q` and `q` to print colored and
+  non-colored single-line values, similar to `P` and `p`.
+- Change default repl to print long sequences on one line and color stacktraces if color is enabled.
+- Add `backmatch` pattern for PEGs.
+- jpm detects if not in a Developer Command prompt on windows for a better error message.
+- jpm install git submodules in dependencies
+- Change default fiber stack limit to the maximum value of a 32 bit signed integer.
+- Some bug fixes with `jpm`
+- Fix bugs with pegs.
+- Add `os/arch` to get ISA that janet was compiled for
+- Add color to stacktraces via `(dyn :err-color)`
+
+## 1.2.0 - 2019-08-08
+- Add `take` and `drop` functions that are easier to use compared to the
+  existing slice functions.
+- Add optional default value to `get`.
+- Add function literal short-hand via `|` reader macro, which maps to the
+  `short-fn` macro.
+- Add `int?` and `nat?` functions to the core.
+- Add `(dyn :executable)` at top level to get what used to be
+  `(process/args 0)`.
+- Add `:linux` to platforms returned by `(os/which)`.
+- Update jpm to build standalone executables. Use `declare-executable` for this.
+- Add `use` macro.
+- Remove `process/args` in favor of `(dyn :args)`.
+- Fix bug with Nanbox implementation allowing users to created
+  custom values of any type with typed array and marshal modules, which
+  was unsafe.
+- Add `janet_wrap_number_safe` to API, for converting numbers to Janets
+  where the number could be any 64 bit, user provided bit pattern. Certain
+  NaN values (which a machine will never generate as a result of a floating
+  point operation) are guarded against and converted to a default NaN value.
+
+## 1.1.0 - 2019-07-08
+- Change semantics of `-l` flag to be import rather than dofile.
+- Fix compiler regression in top level defs with destructuring.
+- Add `table/clone`.
+- Improve `jpm` tool with git and dependency capabilities, as well as better
+  module uninstalls.
+
+## 1.0.0 - 2019-07-01
+- Add `with` macro for resource handling.
+- Add `propagate` function so we can "rethrow" signals after they are
+  intercepted. This makes signals even more flexible.
+- Add `JANET_NO_DOCSTRINGS` and `JANET_NO_SOURCEMAPS` defines in janetconf.h
+  for shrinking binary size.
+  This seems to save about 50kB in most builds, so it's not usually worth it.
+- Update module system to allow relative imports. The `:cur:` pattern
+  in `module/expand-path` will expand to the directory part of the current file, or
+  whatever the value of `(dyn :current-file)` is. The `:dir:` pattern gets
+  the directory part of the input path name.
+- Remove `:native:` pattern in `module/paths`.
+- Add `module/expand-path`
+- Remove `module/*syspath*` and `module/*headerpath*` in favor of dynamic
+  bindings `:syspath` and `:headerpath`.
+- Compiled PEGs can now be marshaled and unmarshaled.
+- Change signature to `parser/state`
+- Add `:until` verb to loop.
+- Add `:p` flag to `fiber/new`.
+- Add `file/{fdopen,fileno}` functions.
+- Add `parser/clone` function.
+- Add optional argument to `parser/where` to set parser byte index.
+- Add optional `env` argument to `all-bindings` and `all-dynamics`.
+- Add scratch memory C API functions for auto-released memory on next gc.
+  Scratch memory differs from normal GCed memory as it can also be freed normally
+  for better performance.
+- Add API compatibility checking for modules. This will let native modules not load
+  when the host program is not of a compatible version or configuration.
+- Change signature of `os/execute` to be much more flexible.
+
+## 0.6.0 - 2019-05-29
+- `file/close` returns exit code when closing file opened with `file/popen`.
+- Add `os/rename`
+- Update windows installer to include tools like `jpm`.
+- Add `jpm` tool for building and managing projects.
+- Change interface to `cook` tool.
+- Add optional filters to `module/paths` to further refine import methods.
+- Add keyword arguments via `&keys` in parameter list.
+- Add `-k` flag for flychecking source.
+- Change signature to `compile` function.
+- Add `module/loaders` for custom loading functions.
+- Add external unification to `match` macro.
+- Add static library to main build.
+- Add `janet/*headerpath*` and change location of installed headers.
+- Let `partition` take strings.
+- Haiku OS support
+- Add `string/trim`, `string/trimr`, and `string/triml`.
+- Add `dofile` function.
+- Numbers require at least 1 significant digit.
+- `file/read` will return nil on end of file.
+- Fix various bugs.
+
+## 0.5.0 - 2019-05-09
+- Fix some bugs with buffers.
+- Add `trace` and `untrace` to the core library.
+- Add `string/has-prefix?` and `string/has-suffix?` to string module.
+- Add simple debugger to repl that activates on errors or debug signal
+- Remove `*env*` and `*doc-width*`.
+- Add `fiber/getenv`, `fiber/setenv`, and `dyn`, and `setdyn`.
+- Add support for dynamic bindings (via the `dyn` and `setdyn` functions).
+- Change signatures of some functions like `eval` which no longer takes
+  an optional environment.
+- Add printf function
+- Make `pp` configurable with dynamic binding `:pretty-format`.
+- Remove the `meta` function.
+- Add `with-dyns` for blocks with dynamic bindings assigned.
+- Allow leading and trailing newlines in backtick-delimited string (long strings).
+  These newlines will not be included in the actual string value.
+
+## 0.4.1 - 2019-04-14
+- Squash some bugs
+- Peg patterns can now make captures in any position in a grammar.
+- Add color to repl output
+- Add array/remove function
+- Add meson build support
+- Add int module for int types
+- Add meson build option
+- Add (break) special form and improve loop macro
+- Allow abstract types to specify custom tostring method
+- Extend C API for marshalling abstract types and other values
+- Add functions to `os` module.
+
+## 0.4.0 - 2019-03-08
+- Fix a number of smaller bugs
+- Added :export option to import and require
+- Added typed arrays
+- Remove `callable?`.
+- Remove `tuple/append` and `tuple/prepend`, which may have seemed like `O(1)`
+  operations. Instead, use the `splice` special to extend tuples.
+- Add `-m` flag to main client to allow specifying where to load
+  system modules from.
+- Add `-c` flag to main client to allow compiling Janet modules to images.
+- Add `string/format` and `buffer/format`.
+- Remove `string/pretty` and `string/number`.
+- `make-image` function creates pre compiled images for janet. These images
+  link to the core library. They can be loaded via require or manually via
+  `load-image`.
+- Add bracketed tuples as tuple constructor.
+- Add partition function to core library.
+- Pre-compile core library into an image for faster startup.
+- Add methods to parser values that mirror the api.
+- Add janet\_getmethod to CAPI for easier use of method like syntax.
+- Add get/set to abstract types to allow them to behave more
+  like objects with methods.
+- Add parser/insert to modify parser state programmatically
+- Add debug/stacktrace for easy, pretty stacktraces
+- Remove the status-pp function
+- Update API to run-context to be much more sane
+- Add :lflags option to cook/make-native
+- Disallow NaNs as table or struct keys
+- Update module resolution paths and format
+
+## 0.3.0 - 2019-01-26
 - Add amalgamated build to janet for easier embedding.
 - Add os/date function
 - Add slurp and spit to core library.

CONTRIBUTING.md

@@ -29,7 +29,12 @@ may require changes before being merged.
   run tests with `make test`. If you want to add a new test suite, simply add a file to
   the test folder and make sure it is run when`make test` is invoked.
 * Be consistent with the style. For C this means follow the indentation and style in
-  other files (files have MIT license at top, 4 spaces indentation, no trailing whitespace, cuddled brackets, etc.)
+  other files (files have MIT license at top, 4 spaces indentation, no trailing
+          whitespace, cuddled brackets, etc.) Use `make format` to
+  automatically format your C code with
+  [astyle](http://astyle.sourceforge.net/astyle.html). You will probably need
+  to install this, but it can be installed with most package managers.
+
   For janet code, the use lisp indentation with 2 spaces. One can use janet.vim to
   do this indentation, or approximate as close as possible.
 
@@ -39,7 +44,6 @@ For changes to the VM and Core code, you will probably need to know C. Janet is
 a subset of C99 that works with Microsoft Visual C++. This means most of C99 but with the following
 omissions.
 
-* No Variable Length Arrays (yes these may work in newer MSVC compilers)
 * No `restrict` 
 * Certain functions in the standard library are not always available
 
@@ -51,6 +55,11 @@ Code should compile warning free and run valgrind clean. I find that these two c
 of the easiest ways to protect against a large number of bugs in an unsafe language like C. To check for
 valgrind errors, run `make valtest` and check the output for undefined or flagged behavior.
 
+### Formatting
+
+Use [astyle](http://astyle.sourceforge.net/astyle.html) via `make format` to
+ensure a consistent code style for C.
+
 ## Janet style
 
 All janet code in the project should be formatted similar to the code in core.janet.

LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2019 Calvin Rose
+Copyright (c) 2019 Calvin Rose and contributors
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of
 this software and associated documentation files (the "Software"), to deal in

Makefile

@@ -24,67 +24,154 @@
 
 PREFIX?=/usr/local
 
-INCLUDEDIR=$(PREFIX)/include/janet
-LIBDIR=$(PREFIX)/lib
-BINDIR=$(PREFIX)/bin
+INCLUDEDIR?=$(PREFIX)/include
+BINDIR?=$(PREFIX)/bin
+LIBDIR?=$(PREFIX)/lib
 JANET_BUILD?="\"$(shell git log --pretty=format:'%h' -n 1)\""
-
-CFLAGS=-std=c99 -Wall -Wextra -Isrc/include -fpic -O2 -fvisibility=hidden \
-	   -DJANET_BUILD=$(JANET_BUILD)
-CLIBS=-lm -ldl
+CLIBS=-lm -lpthread
 JANET_TARGET=build/janet
 JANET_LIBRARY=build/libjanet.so
-JANET_PATH?=/usr/local/lib/janet
+JANET_STATIC_LIBRARY=build/libjanet.a
+JANET_PATH?=$(LIBDIR)/janet
+MANPATH?=$(PREFIX)/share/man/man1/
+PKG_CONFIG_PATH?=$(LIBDIR)/pkgconfig
 DEBUGGER=gdb
 
+CFLAGS=-std=c99 -Wall -Wextra -Isrc/include -Isrc/conf -fPIC -O2 -fvisibility=hidden \
+	   -DJANET_BUILD=$(JANET_BUILD)
+LDFLAGS=-rdynamic
+
+# For installation
+LDCONFIG:=ldconfig "$(LIBDIR)"
+
+# Check OS
 UNAME:=$(shell uname -s)
-LDCONFIG:=ldconfig
 ifeq ($(UNAME), Darwin)
-	# Add other macos/clang flags
+	CLIBS:=$(CLIBS) -ldl
 	LDCONFIG:=
-else
-	CFLAGS:=$(CFLAGS) -rdynamic
-	CLIBS:=$(CLIBS) -lrt
+else ifeq ($(UNAME), Linux)
+	CLIBS:=$(CLIBS) -lrt -ldl
+endif
+# For other unix likes, add flags here!
+ifeq ($(UNAME), Haiku)
+	LDCONFIG:=
+	LDFLAGS=-Wl,--export-dynamic
 endif
 
-$(shell mkdir -p build/core build/mainclient build/webclient)
+$(shell mkdir -p build/core build/mainclient build/webclient build/boot)
+all: $(JANET_TARGET) $(JANET_LIBRARY) $(JANET_STATIC_LIBRARY)
 
-# Source headers
-JANET_HEADERS=$(sort $(wildcard src/include/janet/*.h))
-JANET_LOCAL_HEADERS=$(sort $(wildcard src/*/*.h))
+######################
+##### Name Files #####
+######################
 
-# Source files
-JANET_CORE_SOURCES=$(sort $(wildcard src/core/*.c))
-JANET_MAINCLIENT_SOURCES=$(sort $(wildcard src/mainclient/*.c))
-JANET_WEBCLIENT_SOURCES=$(sort $(wildcard src/webclient/*.c))
+JANET_HEADERS=src/include/janet.h src/conf/janetconf.h
 
-all: $(JANET_TARGET) $(JANET_LIBRARY)
+JANET_LOCAL_HEADERS=src/core/util.h \
+					src/core/state.h \
+					src/core/gc.h \
+					src/core/vector.h \
+					src/core/fiber.h \
+					src/core/regalloc.h \
+					src/core/compile.h \
+					src/core/emit.h \
+					src/core/symcache.h
+
+JANET_CORE_SOURCES=src/core/abstract.c \
+				   src/core/array.c \
+				   src/core/asm.c \
+				   src/core/buffer.c \
+				   src/core/bytecode.c \
+				   src/core/capi.c \
+				   src/core/cfuns.c \
+				   src/core/compile.c \
+				   src/core/corelib.c \
+				   src/core/debug.c \
+				   src/core/emit.c \
+				   src/core/fiber.c \
+				   src/core/gc.c \
+				   src/core/inttypes.c \
+				   src/core/io.c \
+				   src/core/marsh.c \
+				   src/core/math.c \
+				   src/core/os.c \
+				   src/core/parse.c \
+				   src/core/peg.c \
+				   src/core/pp.c \
+				   src/core/regalloc.c \
+				   src/core/run.c \
+				   src/core/specials.c \
+				   src/core/string.c \
+				   src/core/strtod.c \
+				   src/core/struct.c \
+				   src/core/symcache.c \
+				   src/core/table.c \
+				   src/core/thread.c \
+				   src/core/tuple.c \
+				   src/core/typedarray.c \
+				   src/core/util.c \
+				   src/core/value.c \
+				   src/core/vector.c \
+				   src/core/vm.c \
+				   src/core/wrap.c
+
+JANET_BOOT_SOURCES=src/boot/array_test.c \
+				   src/boot/boot.c \
+				   src/boot/buffer_test.c \
+				   src/boot/number_test.c \
+				   src/boot/system_test.c \
+				   src/boot/table_test.c
+
+JANET_MAINCLIENT_SOURCES=src/mainclient/line.c src/mainclient/main.c
+
+JANET_WEBCLIENT_SOURCES=src/webclient/main.c
+
+##################################################################
+##### The bootstrap interpreter that compiles the core image #####
+##################################################################
+
+JANET_BOOT_OBJECTS=$(patsubst src/%.c,build/%.boot.o,$(JANET_CORE_SOURCES) $(JANET_BOOT_SOURCES)) \
+	build/boot.gen.o
+
+build/%.boot.o: src/%.c $(JANET_HEADERS) $(JANET_LOCAL_HEADERS)
+	$(CC) $(CFLAGS) -DJANET_BOOTSTRAP -o $@ -c $<
+
+build/janet_boot: $(JANET_BOOT_OBJECTS)
+	$(CC) $(CFLAGS) -DJANET_BOOTSTRAP -o $@ $^ $(CLIBS)
+
+# Now the reason we bootstrap in the first place
+build/core_image.c: build/janet_boot
+	build/janet_boot $@ JANET_PATH '$(JANET_PATH)' JANET_HEADERPATH '$(INCLUDEDIR)/janet'
 
 ##########################################################
 ##### The main interpreter program and shared object #####
 ##########################################################
 
-JANET_CORE_OBJECTS=$(patsubst src/%.c,build/%.o,$(JANET_CORE_SOURCES)) build/core.gen.o
-JANET_MAINCLIENT_OBJECTS=$(patsubst src/%.c,build/%.o,$(JANET_MAINCLIENT_SOURCES)) build/init.gen.o
+JANET_CORE_OBJECTS=$(patsubst src/%.c,build/%.o,$(JANET_CORE_SOURCES)) build/core_image.o
+JANET_MAINCLIENT_OBJECTS=$(patsubst src/%.c,build/%.o,$(JANET_MAINCLIENT_SOURCES))
 
-%.gen.o: %.gen.c
+# Compile the core image generated by the bootstrap build
+build/core_image.o: build/core_image.c $(JANET_HEADERS) $(JANET_LOCAL_HEADERS)
 	$(CC) $(CFLAGS) -o $@ -c $<
 
 build/%.o: src/%.c $(JANET_HEADERS) $(JANET_LOCAL_HEADERS)
 	$(CC) $(CFLAGS) -o $@ -c $<
 
 $(JANET_TARGET): $(JANET_CORE_OBJECTS) $(JANET_MAINCLIENT_OBJECTS)
-	$(CC) $(CFLAGS) -o $@ $^ $(CLIBS)
+	$(CC) $(LDFLAGS) $(CFLAGS) -o $@ $^ $(CLIBS)
 
 $(JANET_LIBRARY): $(JANET_CORE_OBJECTS)
-	$(CC) $(CFLAGS) -shared -o $@ $^ $(CLIBS)
+	$(CC) $(LDFLAGS) $(CFLAGS) -shared -o $@ $^ $(CLIBS)
+
+$(JANET_STATIC_LIBRARY): $(JANET_CORE_OBJECTS)
+	$(AR) rcs $@ $^
 
 ######################
 ##### Emscripten #####
 ######################
 
 EMCC=emcc
-EMCFLAGS=-std=c99 -Wall -Wextra -Isrc/include -O2 \
+EMCFLAGS=-std=c99 -Wall -Wextra -Isrc/include -Isrc/conf -O2 \
 		  -s EXTRA_EXPORTED_RUNTIME_METHODS='["cwrap"]' \
 		  -s ALLOW_MEMORY_GROWTH=1 \
 		  -s AGGRESSIVE_VARIABLE_ELIMINATION=1 \
@@ -92,11 +179,14 @@ EMCFLAGS=-std=c99 -Wall -Wextra -Isrc/include -O2 \
 JANET_EMTARGET=build/janet.js
 JANET_WEB_SOURCES=$(JANET_CORE_SOURCES) $(JANET_WEBCLIENT_SOURCES)
 JANET_EMOBJECTS=$(patsubst src/%.c,build/%.bc,$(JANET_WEB_SOURCES)) \
-				build/webinit.gen.bc build/core.gen.bc
+				build/webinit.gen.bc build/core_image.bc
 
 %.gen.bc: %.gen.c
 	$(EMCC) $(EMCFLAGS) -o $@ -c $<
 
+build/core_image.bc: build/core_image.c $(JANET_HEADERS) $(JANET_LOCAL_HEADERS)
+	$(EMCC) $(EMCFLAGS) -o $@ -c $<
+
 build/%.bc: src/%.c $(JANET_HEADERS) $(JANET_LOCAL_HEADERS)
 	$(EMCC) $(EMCFLAGS) -o $@ -c $<
 
@@ -109,26 +199,35 @@ emscripten: $(JANET_EMTARGET)
 ##### Generated C files #####
 #############################
 
+%.gen.o: %.gen.c
+	$(CC) $(CFLAGS) -o $@ -c $<
+
 build/xxd: tools/xxd.c
 	$(CC) $< -o $@
 
-build/core.gen.c: src/core/core.janet build/xxd
-	build/xxd $< $@ janet_gen_core
-build/init.gen.c: src/mainclient/init.janet build/xxd
-	build/xxd $< $@ janet_gen_init
 build/webinit.gen.c: src/webclient/webinit.janet build/xxd
 	build/xxd $< $@ janet_gen_webinit
+build/boot.gen.c: src/boot/boot.janet build/xxd
+	build/xxd $< $@ janet_gen_boot
 
 ########################
 ##### Amalgamation #####
 ########################
 
-amalg: build/janet.c build/janet.h
+amalg: build/shell.c build/janet.c build/janet.h build/core_image.c build/janetconf.h
 
-build/janet.c: $(JANET_LOCAL_HEADERS) $(JANET_CORE_SOURCES) tools/amalg.janet $(JANET_TARGET)
-	$(JANET_TARGET) tools/amalg.janet > $@
+AMALG_SOURCE=$(JANET_LOCAL_HEADERS) $(JANET_CORE_SOURCES) build/core_image.c
+build/janet.c: $(AMALG_SOURCE) tools/amalg.janet $(JANET_TARGET)
+	$(JANET_TARGET) tools/amalg.janet $(AMALG_SOURCE) > $@
 
-build/janet.h: src/include/janet/janet.h
+AMALG_SHELL_SOURCE=src/mainclient/line.h src/mainclient/line.c src/mainclient/main.c
+build/shell.c: $(JANET_TARGET) tools/amalg.janet $(AMALG_SHELL_SOURCE)
+	$(JANET_TARGET) tools/amalg.janet $(AMALG_SHELL_SOURCE) > $@
+
+build/janet.h: src/include/janet.h
+	cp $< $@
+
+build/janetconf.h: src/conf/janetconf.h
 	cp $< $@
 
 ###################
@@ -149,13 +248,17 @@ valgrind: $(JANET_TARGET)
 	$(VALGRIND_COMMAND) ./$(JANET_TARGET)
 
 test: $(JANET_TARGET) $(TEST_PROGRAMS)
-	for f in test/*.janet; do ./$(JANET_TARGET) "$$f" || exit; done
+	for f in test/suite*.janet; do ./$(JANET_TARGET) "$$f" || exit; done
+	for f in examples/*.janet; do ./$(JANET_TARGET) -k "$$f"; done
+	./$(JANET_TARGET) -k auxbin/jpm
 
 valtest: $(JANET_TARGET) $(TEST_PROGRAMS)
-	for f in test/*.janet; do $(VALGRIND_COMMAND) ./$(JANET_TARGET) "$$f" || exit; done
+	for f in test/suite*.janet; do $(VALGRIND_COMMAND) ./$(JANET_TARGET) "$$f" || exit; done
+	for f in examples/*.janet; do ./$(JANET_TARGET) -k "$$f"; done
+	$(VALGRIND_COMMAND) ./$(JANET_TARGET) -k auxbin/jpm
 
 callgrind: $(JANET_TARGET)
-	for f in test/*.janet; do valgrind --tool=callgrind ./$(JANET_TARGET) "$$f" || exit; done
+	for f in test/suite*.janet; do valgrind --tool=callgrind ./$(JANET_TARGET) "$$f" || exit; done
 
 ########################
 ##### Distribution #####
@@ -163,10 +266,14 @@ callgrind: $(JANET_TARGET)
 
 dist: build/janet-dist.tar.gz
 
-build/janet-%.tar.gz: $(JANET_TARGET) src/include/janet/janet.h \
-	janet.1 LICENSE CONTRIBUTING.md $(JANET_LIBRARY) \
-	build/doc.html README.md
-	tar -czvf $@ $^
+build/janet-%.tar.gz: $(JANET_TARGET) \
+	src/include/janet.h src/conf/janetconf.h \
+	jpm.1 janet.1 LICENSE CONTRIBUTING.md $(JANET_LIBRARY) $(JANET_STATIC_LIBRARY) \
+	build/doc.html README.md build/janet.c build/shell.c auxbin/jpm
+	$(eval JANET_DIST_DIR = "janet-$(shell basename $*)")
+	mkdir -p build/$(JANET_DIST_DIR)
+	cp -r $^ build/$(JANET_DIST_DIR)/
+	cd build && tar -czvf ../$@ $(JANET_DIST_DIR)
 
 #########################
 ##### Documentation #####
@@ -177,10 +284,63 @@ docs: build/doc.html
 build/doc.html: $(JANET_TARGET) tools/gendoc.janet
 	$(JANET_TARGET) tools/gendoc.janet > build/doc.html
 
+########################
+##### Installation #####
+########################
+
+SONAME=libjanet.so.1
+
+.PHONY: build/janet.pc
+build/janet.pc: $(JANET_TARGET)
+	echo 'prefix=$(PREFIX)' > $@
+	echo 'exec_prefix=$${prefix}' >> $@
+	echo 'includedir=$(INCLUDEDIR)/janet' >> $@
+	echo 'libdir=$(LIBDIR)' >> $@
+	echo "" >> $@
+	echo "Name: janet" >> $@
+	echo "Url: https://janet-lang.org" >> $@
+	echo "Description: Library for the Janet programming language." >> $@
+	$(JANET_TARGET) -e '(print "Version: " janet/version)' >> $@
+	echo 'Cflags: -I$${includedir}' >> $@
+	echo 'Libs: -L$${libdir} -ljanet $(LDFLAGS)' >> $@
+	echo 'Libs.private: $(CLIBS)' >> $@
+
+install: $(JANET_TARGET) build/janet.pc
+	mkdir -p '$(BINDIR)'
+	cp $(JANET_TARGET) '$(BINDIR)/janet'
+	mkdir -p '$(INCLUDEDIR)/janet'
+	cp -rf $(JANET_HEADERS) '$(INCLUDEDIR)/janet'
+	mkdir -p '$(JANET_PATH)'
+	mkdir -p '$(LIBDIR)'
+	cp $(JANET_LIBRARY) '$(LIBDIR)/libjanet.so.$(shell $(JANET_TARGET) -e '(print janet/version)')'
+	cp $(JANET_STATIC_LIBRARY) '$(LIBDIR)/libjanet.a'
+	ln -sf $(SONAME) '$(LIBDIR)/libjanet.so'
+	ln -sf libjanet.so.$(shell $(JANET_TARGET) -e '(print janet/version)') $(LIBDIR)/$(SONAME)
+	cp -rf auxbin/* '$(BINDIR)'
+	mkdir -p '$(MANPATH)'
+	cp janet.1 '$(MANPATH)'
+	cp jpm.1 '$(MANPATH)'
+	mkdir -p '$(PKG_CONFIG_PATH)'
+	cp build/janet.pc '$(PKG_CONFIG_PATH)/janet.pc'
+	-$(LDCONFIG)
+
+uninstall:
+	-rm '$(BINDIR)/janet'
+	-rm '$(BINDIR)/jpm'
+	-rm -rf '$(INCLUDEDIR)/janet'
+	-rm -rf '$(LIBDIR)'/libjanet.*
+	-rm '$(PKG_CONFIG_PATH)/janet.pc'
+	-rm '$(MANPATH)/janet.1'
+	-rm '$(MANPATH)/jpm.1'
+	# -rm -rf '$(JANET_PATH)'/* - err on the side of correctness here
+
 #################
 ##### Other #####
 #################
 
+format:
+	tools/format.sh
+
 grammar: build/janet.tmLanguage
 build/janet.tmLanguage: tools/tm_lang_gen.janet $(JANET_TARGET)
 	$(JANET_TARGET) $< > $@
@@ -188,26 +348,28 @@ build/janet.tmLanguage: tools/tm_lang_gen.janet $(JANET_TARGET)
 clean:
 	-rm -rf build vgcore.* callgrind.*
 
-install: $(JANET_TARGET)
-	mkdir -p $(BINDIR)
-	cp $(JANET_TARGET) $(BINDIR)/janet
-	mkdir -p $(INCLUDEDIR)
-	cp $(JANET_HEADERS) $(INCLUDEDIR)
-	mkdir -p $(LIBDIR)
-	cp $(JANET_LIBRARY) $(LIBDIR)/libjanet.so
-	mkdir -p $(JANET_PATH)
-	cp tools/cook.janet $(JANET_PATH)
-	cp tools/highlight.janet $(JANET_PATH)
-	cp janet.1 /usr/local/share/man/man1/
-	mandb
-	$(LDCONFIG)
+test-install:
+	cd test/install \
+		&& rm -rf build .cache .manifests \
+		&& jpm --verbose build \
+		&& jpm --verbose test \
+		&& build/testexec \
+		&& jpm --verbose quickbin testexec.janet build/testexec2 \
+		&& build/testexec2 \
+		&& jpm --verbose --testdeps --modpath=. install https://github.com/janet-lang/json.git
+	cd test/install && jpm --verbose --test --modpath=. install https://github.com/janet-lang/jhydro.git
+	cd test/install && jpm --verbose --test --modpath=. install https://github.com/janet-lang/path.git
+	cd test/install && jpm --verbose --test --modpath=. install https://github.com/janet-lang/argparse.git
 
-uninstall:
-	-rm $(BINDIR)/../$(JANET_TARGET)
-	-rm $(LIBDIR)/../$(JANET_LIBRARY)
-	-rm -rf $(INCLUDEDIR)
-	$(LDCONFIG)
+build/embed_janet.o: build/janet.c $(JANET_HEADERS)
+	$(CC) $(CFLAGS) -c $< -o $@
+build/embed_main.o: test/amalg/main.c $(JANET_HEADERS)
+	$(CC) $(CFLAGS) -c $< -o $@
+build/embed_test: build/embed_janet.o build/embed_main.o
+	$(CC) $(LDFLAGS) $(CFLAGS) -o $@ $^ $(CLIBS)
+
+test-amalg: build/embed_test
+	./build/embed_test
 
 .PHONY: clean install repl debug valgrind test amalg \
-	valtest emscripten dist uninstall docs grammar \
-	$(TEST_PROGRAM_PHONIES) $(TEST_PROGRAM_VALPHONIES)
+	valtest emscripten dist uninstall docs grammar format

README.md

@@ -1,27 +1,23 @@
+[![Join the chat](https://badges.gitter.im/janet-language/community.svg)](https://gitter.im/janet-language/community)
+ 
+[![Appveyor Status](https://ci.appveyor.com/api/projects/status/bjraxrxexmt3sxyv/branch/master?svg=true)](https://ci.appveyor.com/project/bakpakin/janet/branch/master)
 [![Build Status](https://travis-ci.org/janet-lang/janet.svg?branch=master)](https://travis-ci.org/janet-lang/janet)
-[![Appveyor Status](https://ci.appveyor.com/api/projects/status/32r7s2skrgm9ubva?svg=true)](https://ci.appveyor.com/project/janet-lang/janet)
+[![builds.sr.ht status](https://builds.sr.ht/~bakpakin/janet/.freebsd.yaml.svg)](https://builds.sr.ht/~bakpakin/janet/.freebsd.yaml?)
+[![builds.sr.ht status](https://builds.sr.ht/~bakpakin/janet/.openbsd.yaml.svg)](https://builds.sr.ht/~bakpakin/janet/.openbsd.yaml?)
 
 <img src="https://raw.githubusercontent.com/janet-lang/janet/master/assets/janet-w200.png" alt="Janet logo" width=200 align="left">
 
 **Janet** is a functional and imperative programming language and bytecode interpreter. It is a
 modern lisp, but lists are replaced
 by other data structures with better utility and performance (arrays, tables, structs, tuples).
-The language also bridging bridging to native code written in C, meta-programming with macros, and bytecode assembly.
+The language also supports bridging to native code written in C, meta-programming with macros, and bytecode assembly.
 
 There is a repl for trying out the language, as well as the ability
 to run script files. This client program is separate from the core runtime, so
 janet could be embedded into other programs. Try janet in your browser at
 [https://janet-lang.org](https://janet-lang.org).
 
-#
-
-Implemented in mostly standard C99, janet runs on Windows, Linux and macOS.
-The few features that are not standard C (dynamic library loading, compiler specific optimizations),
-are fairly straight forward. Janet can be easily ported to new platforms.
-
-For syntax highlighting, there is some preliminary vim syntax highlighting in [janet.vim](https://github.com/janet-lang/janet.vim).
-Generic lisp syntax highlighting should, however, provide good results. One can also generate a janet.tmLanguage
-file for other programs with `make grammar`.
+<br>
 
 ## Use Cases
 
@@ -53,111 +49,67 @@ Janet makes a good system scripting language, or a language to embed in other pr
 
 ## Documentation
 
-Documentation can be found in the doc directory of
-the repository. There is an introduction
-section contains a good overview of the language.
+* For a quick tutorial, see [the introduction](https://janet-lang.org/docs/index.html) for more details.
+* For the full API for all functions in the core library, see [the core API doc](https://janet-lang.org/api/index.html)
 
-API documentation for all bindings can also be generated
-with `make docs`, which will create `build/doc.html`, which
-can be viewed with any web browser. This
-includes all forms in the core library except special forms.
-
-For individual bindings from within the REPL, use the `(doc symbol-name)` macro to get API
-documentation for the core library. For example,
+Documentation is also available locally in the repl.
+Use the `(doc symbol-name)` macro to get API
+documentation for symbols in the core library. For example,
 ```
 (doc doc)
 ```
 Shows documentation for the doc macro.
 
 To get a list of all bindings in the default
-environment, use the `(all-symbols)` function.
+environment, use the `(all-bindings)` function.
 
-## Installation
+## Source
 
-Install a stable version of janet from the [releases page](https://github.com/janet-lang/janet/releases).
-Janet is prebuilt for a few systems, but if you want to develop janet, run janet on a non-x86 system, or
-get the latest, you must build janet from source. Janet is in alpha and may change
-in backwards incompatible ways.
+You can get the source on [GitHub](https://github.com/janet-lang/janet) or
+[SourceHut](https://git.sr.ht/~bakpakin/janet). While the GitHub repo is the official repo,
+the SourceHut mirror is actively maintained.
 
-## Usage
-
-A repl is launched when the binary is invoked with no arguments. Pass the -h flag
-to display the usage information. Individual scripts can be run with `./janet myscript.janet`
-
-If you are looking to explore, you can print a list of all available macros, functions, and constants
-by entering the command `(all-symbols)` into the repl.
-
-```
-$ ./janet
-Janet 0.0.0 alpha  Copyright (C) 2017-2018 Calvin Rose
-janet:1:> (+ 1 2 3)
-6
-janet:2:> (print "Hello, World!")
-Hello, World!
-nil
-janet:3:> (os.exit)
-$ ./janet -h
-usage: ./janet [options] scripts...
-Options are:
-  -h Show this help
-  -v Print the version string
-  -s Use raw stdin instead of getline like functionality
-  -e Execute a string of janet
-  -r Enter the repl after running all scripts
-  -p Keep on executing if there is a top level error (persistent)
-  -- Stop handling option
-$
-```
-
-## Embedding
-
-The C API for Janet is not yet documented but coming soon.
-
-Janet can be embedded in a host program very easily. There is a make target `make amalg`
-which creates the file `build/janet.c`, which is a single C file that contains all the source
-to Janet. This file, along with `src/include/janet/janet.h` can dragged into any C project
-and compiled into the project. Janet should be compiled with `-std=c99` on most compilers, and
-will need to be linked to the math library, `-lm`, and the dynamic linker, `-ldl`, if one wants
-to be able to load dynamic modules. If there is no need for dynamic modules, add the define
-`-DJANET_NO_DYNAMIC_MODULES` to the compiler options.
-
-## Compiling and Running
-
-Janet only uses Make and batch files to compile on Posix and windows
-respectively. To configure janet, edit the header file src/include/janet/janet.h
-before compilation.
+## Building
 
 ### macos and Unix-like
 
-On most platforms, use Make to build janet. The resulting binary will be in `build/janet`.
+The Makefile is non-portable and requires GNU-flavored make.
 
-```sh
+```
 cd somewhere/my/projects/janet
 make
 make test
+make repl
 ```
 
-After building, run `make install` to install the janet binary and libs.
-Will install in `/usr/local` by default, see the Makefile to customize.
+### 32-bit Haiku
 
-It's also recommended to set the `JANET_PATH` variable in your profile.
-This is where janet will look for imported libraries after the current directory.
+32-bit Haiku build instructions are the same as the unix-like build instructions,
+but you need to specify an alternative compiler, such as `gcc-x86`.
+
+```
+cd somewhere/my/projects/janet
+make CC=gcc-x86
+make test
+make repl
+```
 
 ### FreeBSD
 
 FreeBSD build instructions are the same as the unix-like build instuctions,
-but you need `gmake` and `gcc` to compile.
+but you need `gmake` to compile. Alternatively, install directly from
+packages, using `pkg install lang/janet`.
 
 ```
 cd somewhere/my/projects/janet
-gmake CC=gcc
-gmake test CC=gcc
+gmake
+gmake test
+gmake repl
 ```
 
 ### Windows
 
-1. Install [Visual Studio](https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=Community&rel=15#)
-or [Visual Studio Build Tools](https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=BuildTools&rel=15#)
+1. Install [Visual Studio](https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=Community&rel=15#) or [Visual Studio Build Tools](https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=BuildTools&rel=15#)
 2. Run a Visual Studio Command Prompt (cl.exe and link.exe need to be on the PATH) and cd to the directory with janet.
 3. Run `build_win` to compile janet.
 4. Run `build_win test` to make sure everything is working.
@@ -171,17 +123,112 @@ The JavaScript build is what runs the repl on the main website,
 but really serves mainly as a proof of concept. Janet will run slower in a browser.
 Building with emscripten on windows is currently unsupported.
 
+### Meson
+
+Janet also has a build file for [Meson](https://mesonbuild.com/), a cross platform build
+system. Although Meson has a python dependency, Meson is a very complete build system that
+is maybe more convenient and flexible for integrating into existing pipelines.
+Meson also provides much better IDE integration than Make or batch files, as well as support
+for cross compilation.
+
+For the impatient, building with Meson is as simple as follows. The options provided to
+`meson setup` below emulate Janet's Makefile.
+
+```sh
+git clone https://github.com/janet-lang/janet.git
+cd janet
+meson setup build \
+          --buildtype release \
+          --optimization 2 \
+          -Dgit_hash=$(git log --pretty=format:'%h' -n 1)
+ninja -C build
+
+# Run the binary
+build/janet
+
+# Installation
+ninja -C build install
+```
+
+## Development
+
+Janet can be hacked on with pretty much any environment you like, but for IDE
+lovers, [Gnome Builder](https://wiki.gnome.org/Apps/Builder) is probably the
+best option, as it has excellent meson integration. It also offers code completion
+for Janet's C API right out of the box, which is very useful for exploring.
+
+## Installation
+
+See [the Introduction](https://janet-lang.org/introduction.html) for more details. If you just want
+to try out the language, you don't need to install anything. You can also simply move the `janet` executable wherever you want on your system and run it.
+
+## Usage
+
+A repl is launched when the binary is invoked with no arguments. Pass the -h flag
+to display the usage information. Individual scripts can be run with `./janet myscript.janet`
+
+If you are looking to explore, you can print a list of all available macros, functions, and constants
+by entering the command `(all-bindings)` into the repl.
+
+```
+$ ./janet
+Janet 0.0.0 alpha  Copyright (C) 2017-2018 Calvin Rose
+janet:1:> (+ 1 2 3)
+6
+janet:2:> (print "Hello, World!")
+Hello, World!
+nil
+janet:3:> (os/exit)
+$ ./janet -h
+usage: ./janet [options] scripts...
+Options are:
+  -h Show this help
+  -v Print the version string
+  -s Use raw stdin instead of getline like functionality
+  -e Execute a string of janet
+  -r Enter the repl after running all scripts
+  -p Keep on executing if there is a top level error (persistent)
+  -- Stop handling option
+$
+```
+
+If installed, you can also run `man janet` to get usage information.
+
+## Embedding
+
+The C API for Janet is not yet documented but coming soon.
+
+Janet can be embedded in a host program very easily. There is a make target
+`make amalg` which creates the file `build/janet.c`, which is a single C file
+that contains all the source to Janet. This file, along with
+`src/include/janet.h` and `src/include/janetconf.h` can dragged into any C
+project and compiled into the project. Janet should be compiled with `-std=c99`
+on most compilers, and will need to be linked to the math library, `-lm`, and
+the dynamic linker, `-ldl`, if one wants to be able to load dynamic modules. If
+there is no need for dynamic modules, add the define
+`-DJANET_NO_DYNAMIC_MODULES` to the compiler options.
+
 ## Examples
 
 See the examples directory for some example janet code.
 
-## IRC
+## Discussion
 
-Feel free to ask questions and join discussion on [the #janet channel on Freenode](https://webchat.freenode.net/)
+Feel free to ask questions and join discussion on the [Janet Gitter Channel](https://gitter.im/janet-language/community).
+Alternatively, check out [the #janet channel on Freenode](https://webchat.freenode.net/)
+
+## FAQ
+
+### Why is my terminal is spitting out junk when I run the repl?
+
+Make sure your terminal supports ANSI escape codes. Most modern terminals will
+support these, but some older terminals, windows consoles, or embedded terminals
+will not. If your terminal does not support ANSI escape codes, run the repl with
+the `-n` flag, which disables color output. You can also try the `-s` if further issues
+ensue.
 
 ## Why Janet
 
 Janet is named after the almost omniscient and friendly artificial being in [The Good Place](https://en.wikipedia.org/wiki/The_Good_Place).
 
 <img src="https://raw.githubusercontent.com/janet-lang/janet/master/assets/janet-the-good-place.gif" alt="Janet logo" width="115px" align="left">
-

appveyor.yml

@@ -1,12 +1,13 @@
 version: build-{build}
 clone_folder: c:\projects\janet
 image:
-- Visual Studio 2017
+- Visual Studio 2019
 configuration:
 - Release
 - Debug
 platform:
 - x64
+- x86
 environment:
   matrix:
   - arch: Win64
@@ -15,34 +16,49 @@ matrix:
 
 # skip unsupported combinations
 init:
-- call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvars64.bat"
+    - call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" %platform%
 
 install:
-    - build_win
-    - build_win test
+    - set JANET_BUILD=%appveyor_repo_commit:~0,7%
     - choco install nsis -y -pre
-    - call "C:\Program Files (x86)\NSIS\makensis.exe" janet-installer.nsi 
-    - build_win dist
-    - copy janet-install.exe dist\install.exe 
-    
+    # Replace makensis.exe and files with special long string build. This should
+    # prevent issues when setting PATH during installation.
+    - 7z e "tools\nsis-3.04-strlen_8192.zip" -o"C:\Program Files (x86)\NSIS\" -y
+    - build_win all
+    - refreshenv
+    # We need to reload vcvars after refreshing
+    - call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" %platform%
+    - build_win test-install
+    - set janet_outname=%appveyor_repo_tag_name%
+    - if "%janet_outname%"=="" set janet_outname=v1.6.0
 build: off
 
-only_commits:
-  files:
-    - appveyor.yml
-    - src/
-
 artifacts:
-    - path: dist
-      name: janet-windows
+    - name: janet.c
+      path: dist\janet.c
+      type: File
+    - name: janet.h
+      path: dist\janet.h
+      type: File
+    - name: janetconf.h
+      path: dist\janetconf.h
+      type: File
+    - name: shell.c
+      path: dist\shell.c
+      type: File
+    - name: "janet-$(janet_outname)-windows-%platform%"
+      path: dist
       type: Zip
+    - path: "janet-$(janet_outname)-windows-installer.exe"
+      name: "janet-$(janet_outname)-windows-%platform%-installer.exe"
+      type: File
 
 deploy:
   description: 'The Janet Programming Language.'
   provider: GitHub
   auth_token:
     secure: lwEXy09qhj2jSH9s1C/KvCkAUqJSma8phFR+0kbsfUc3rVxpNK5uD3z9Md0SjYRx
-  artifact: janet-windows
+  artifact: /(janet|shell).*/
   draft: true
   on:
       APPVEYOR_REPO_TAG: true

assets/icon_svg.svg

@@ -0,0 +1,11 @@
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 20010904//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<svg version="1.0" xmlns="http://www.w3.org/2000/svg" width="64px" height="64px" viewBox="0 0 640 640" preserveAspectRatio="xMidYMid meet">
+<g id="layer101" fill="#d45500" stroke="none">
+ <path d="M145 531 c-46 -31 -58 -75 -30 -118 21 -32 30 -22 44 47 7 30 19 62 27 71 26 29 1 29 -41 0z"/>
+ <path d="M341 534 c-23 -29 -26 -50 -11 -88 10 -28 64 -60 86 -52 12 5 12 2 0 -22 -24 -47 -51 -64 -116 -71 -51 -6 -65 -12 -85 -37 -14 -16 -24 -32 -25 -36 0 -12 -35 -9 -48 4 -7 7 -12 24 -12 38 0 41 -11 43 -47 8 -47 -46 -46 -90 5 -138 20 -19 49 -51 63 -70 l27 -35 88 0 c49 0 106 4 127 8 46 10 106 62 143 125 25 42 28 58 30 142 0 52 4 103 9 113 11 27 -14 75 -49 93 -41 21 -115 44 -143 44 -12 0 -31 -12 -42 -26z m89 -119 c0 -3 -2 -5 -5 -5 -3 0 -5 2 -5 5 0 3 2 5 5 5 3 0 5 -2 5 -5z"/>
+ </g>
+<g id="layer102" fill="#deaa87" stroke="none">
+ <path d="M186 549 c-33 -31 -38 -43 -56 -137 -26 -135 -26 -163 3 -190 33 -31 49 -28 85 17 28 35 36 39 87 43 46 4 61 10 90 38 18 18 39 46 46 62 10 25 9 32 -5 46 -17 16 -19 16 -29 1 -8 -14 -15 -15 -34 -6 -27 12 -40 65 -24 96 10 17 8 23 -12 36 -13 8 -44 18 -69 21 -42 6 -49 4 -82 -27z"/>
+ </g>
+
+</svg>

auxbin/jpm

@@ -0,0 +1,979 @@
+#!/usr/bin/env janet
+
+# CLI tool for building janet projects.
+
+#
+# Basic Path Settings
+#
+
+# Windows is the OS outlier
+(def- is-win (= (os/which) :windows))
+(def- is-mac (= (os/which) :macos))
+(def- sep (if is-win "\\" "/"))
+(def- objext (if is-win ".obj" ".o"))
+(def- modext (if is-win ".dll" ".so"))
+(def- statext (if is-win ".static.lib" ".a"))
+(def- absprefix (if is-win "C:\\" "/"))
+
+#
+# Rule Engine
+#
+
+(defn- getrules []
+  (if-let [rules (dyn :rules)] rules (setdyn :rules @{})))
+
+(defn- gettarget [target]
+  (def item ((getrules) target))
+  (unless item (error (string "No rule for target " target)))
+  item)
+
+(defn- rule-impl
+  [target deps thunk &opt phony]
+  (put (getrules) target @[(array/slice deps) thunk phony]))
+
+(defmacro rule
+  "Add a rule to the rule graph."
+  [target deps & body]
+  ~(,rule-impl ,target ,deps (fn [] nil ,;body)))
+
+(defmacro phony
+  "Add a phony rule to the rule graph. A phony rule will run every time
+  (it is always considered out of date). Phony rules are good for defining
+  user facing tasks."
+  [target deps & body]
+  ~(,rule-impl ,target ,deps (fn [] nil ,;body) true))
+
+(defn add-dep
+  "Add a dependency to an existing rule. Useful for extending phony
+  rules or extending the dependency graph of existing rules."
+  [target dep]
+  (def [deps] (gettarget target))
+  (array/push deps dep))
+
+(defn- add-thunk
+  [target more]
+  (def item (gettarget target))
+  (def [_ thunk] item)
+  (put item 1 (fn [] (more) (thunk))))
+
+(defmacro add-body
+  "Add recipe code to an existing rule. This makes existing rules do more but
+  does not modify the dependency graph."
+  [target & body]
+  ~(,add-thunk ,target (fn [] ,;body)))
+
+(defn- needs-build
+  [dest src]
+  (let [mod-dest (os/stat dest :modified)
+        mod-src (os/stat src :modified)]
+    (< mod-dest mod-src)))
+
+(defn- needs-build-some
+  [dest sources]
+  (def f (file/open dest))
+  (if (not f) (break true))
+  (file/close f)
+  (some (partial needs-build dest) sources))
+
+(defn do-rule
+  "Evaluate a given rule."
+  [target]
+  (def item ((getrules) target))
+  (unless item
+    (if (os/stat target :mode)
+      (break target)
+      (error (string "No rule for file " target " found."))))
+  (def [deps thunk phony] item)
+  (def realdeps (seq [dep :in deps :let [x (do-rule dep)] :when x] x))
+  (when (or phony (needs-build-some target realdeps))
+    (thunk))
+  (unless phony target))
+
+#
+# Configuration
+#
+
+(def JANET_MODPATH (or (os/getenv "JANET_MODPATH") (dyn :syspath)))
+(def JANET_HEADERPATH (or (os/getenv "JANET_HEADERPATH")
+                          (if-let [j (dyn :syspath)]
+                            (string j "/../../include/janet"))))
+(def JANET_BINPATH (or (os/getenv "JANET_BINPATH")
+                       (if-let [j (dyn :syspath)]
+                         (string j "/../../bin"))))
+(def JANET_LIBPATH (or (os/getenv "JANET_LIBPATH")
+                       (if-let [j (dyn :syspath)]
+                         (string j "/.."))))
+
+#
+# Compilation Defaults
+#
+
+(def default-compiler (or (os/getenv "CC") (if is-win "cl.exe" "cc")))
+(def default-linker (or (os/getenv "CC") (if is-win "link.exe" "cc")))
+(def default-archiver (or (os/getenv "AR") (if is-win "lib.exe" "ar")))
+
+# Detect threads
+(def env (fiber/getenv (fiber/current)))
+(def threads? (not (not (env 'thread/new))))
+
+(print "threads " threads?)
+
+# Default flags for natives, but not required
+(def default-lflags (if is-win ["/nologo"] []))
+(def default-cflags
+  (if is-win
+    ["/nologo" "/MD"]
+    ["-std=c99" "-Wall" "-Wextra"]))
+
+
+# Link to pthreads
+(def- thread-flags (if is-win [] (if threads? ["-lpthread"] [])))
+
+# Required flags for dynamic libraries. These
+# are used no matter what for dynamic libraries.
+(def- dynamic-cflags
+  (if is-win
+    ["/LD"]
+    ["-fPIC"]))
+(def- dynamic-lflags
+    (if is-win
+      ["/DLL" ;thread-flags]
+      (if is-mac
+        ["-shared" "-undefined" "dynamic_lookup" ;thread-flags]
+        ["-shared" ;thread-flags])))
+
+(defn- opt
+  "Get an option, allowing overrides via dynamic bindings AND some
+  default value dflt if no dynamic binding is set."
+  [opts key dflt]
+  (def ret (or (opts key) (dyn key dflt)))
+  (if (= nil ret)
+    (error (string "option :" key " not set")))
+  ret)
+
+(defn check-cc
+  "Ensure we have a c compiler"
+  []
+  (if is-win
+    (do
+      (if (os/getenv "INCLUDE") (break))
+      (error "Run jpm inside a Developer Command Prompt.
+ jpm needs a c compiler to compile natives. You can install the MSVC compiler from
+ microsoft.com"))
+    (do)))
+
+#
+# Importing a file
+#
+
+(def- _env (fiber/getenv (fiber/current)))
+
+(defn- proto-flatten
+  [into x]
+  (when x
+    (proto-flatten into (table/getproto x))
+    (loop [k :keys x]
+      (put into k (x k))))
+  into)
+
+(defn import-rules
+  "Import another file that defines more rules. This ruleset
+  is merged into the current ruleset."
+  [path]
+  (def env (make-env))
+  (unless (os/stat path :mode)
+    (error (string "cannot open " path)))
+  (loop [k :keys _env :when (symbol? k)]
+     (unless ((_env k) :private) (put env k (_env k))))
+  (def currenv (proto-flatten @{} (fiber/getenv (fiber/current))))
+  (loop [k :keys currenv :when (keyword? k)]
+    (put env k (currenv k)))
+  (dofile path :env env :exit true)
+  (when-let [rules (env :rules)] (merge-into (getrules) rules)))
+
+#
+# OS and shell helpers
+#
+
+(def- path-splitter
+  "split paths on / and \\."
+  (peg/compile ~(any (* '(any (if-not (set `\/`) 1)) (+ (set `\/`) -1)))))
+
+(def- filepath-replacer
+  "Convert url with potential bad characters into a file path element."
+  (peg/compile ~(% (any (+ (/ '(set "<>:\"/\\|?*") "_") '1)))))
+
+(defn filepath-replace
+  "Remove special characters from a string or path
+  to make it into a path segment."
+  [repo]
+  (get (peg/match filepath-replacer repo) 0))
+
+(defn shell
+  "Do a shell command"
+  [& args]
+  (if (dyn :verbose)
+    (print ;(interpose " " args)))
+  (def res (os/execute args :p))
+  (unless (zero? res)
+    (error (string "command exited with status " res))))
+
+(defn rm
+  "Remove a directory and all sub directories."
+  [path]
+  (try
+    (if (= (os/stat path :mode) :directory)
+      (do
+        (each subpath (os/dir path)
+          (rm (string path sep subpath)))
+        (os/rmdir path))
+      (os/rm path))
+    ([err f] (unless (string/has-prefix? "No such file or directory" err)
+               (propagate err f)))))
+
+(defn copy
+  "Copy a file or directory recursively from one location to another."
+  [src dest]
+  (print "copying " src " to " dest "...")
+  (if is-win
+    (let [end (last (peg/match path-splitter src))
+          isdir (= (os/stat src :mode) :directory)]
+        (shell "xcopy" src (if isdir (string dest "\\" end) dest) "/y" "/s" "/e" "/i"))
+    (shell "cp" "-rf" src dest)))
+
+(defn mkdir
+  "Create a directory if it doesn't exist. If it does exist, do nothing.
+  If we can't create it, give a friendly error. Return true if created, false if
+  existing. Throw an error if we can't create it."
+  [dir]
+  (if (os/mkdir dir)
+    true
+    (if (os/stat dir :mode)
+      false
+      (error (string "Could not create " dir " - this could be a permission issue.")))))
+
+#
+# C Compilation
+#
+
+(defn- embed-name
+  "Rename a janet symbol for embedding."
+  [path]
+  (->> path
+       (string/replace-all "\\" "___")
+       (string/replace-all "/" "___")
+       (string/replace-all ".janet" "")))
+
+(defn- out-path
+  "Take a source file path and convert it to an output path."
+  [path from-ext to-ext]
+  (->> path
+       (string/replace-all "\\" "___")
+       (string/replace-all "/" "___")
+       (string/replace-all from-ext to-ext)
+       (string "build" sep)))
+
+(defn- make-define
+  "Generate strings for adding custom defines to the compiler."
+  [define value]
+  (if value
+    (string (if is-win "/D" "-D") define "=" value)
+    (string (if is-win "/D" "-D") define)))
+
+(defn- make-defines
+  "Generate many defines. Takes a dictionary of defines. If a value is
+  true, generates -DNAME (/DNAME on windows), otherwise -DNAME=value."
+  [defines]
+  (seq [[d v] :pairs defines] (make-define d (if (not= v true) v))))
+
+(defn- getcflags
+  "Generate the c flags from the input options."
+  [opts]
+  @[;(opt opts :cflags default-cflags)
+    (string (if is-win "/I" "-I") (dyn :headerpath JANET_HEADERPATH))
+    (string (if is-win "/O" "-O") (opt opts :optimize 2))])
+
+(defn- entry-name
+  "Name of symbol that enters static compilation of a module."
+  [name]
+  (string "janet_module_entry_" (filepath-replace name)))
+
+(defn- compile-c
+  "Compile a C file into an object file."
+  [opts src dest &opt static?]
+  (def cc (opt opts :compiler default-compiler))
+  (def cflags [;(getcflags opts) ;(if static? [] dynamic-cflags)])
+  (def entry-defines (if-let [n (opts :entry-name)]
+                       [(make-define "JANET_ENTRY_NAME" n)]
+                       []))
+  (def defines [;(make-defines (opt opts :defines {})) ;entry-defines])
+  (def headers (or (opts :headers) []))
+  (rule dest [src ;headers]
+        (check-cc)
+        (print "compiling " dest "...")
+        (if is-win
+          (shell cc ;defines "/c" ;cflags (string "/Fo" dest) src)
+          (shell cc "-c" src ;defines ;cflags "-o" dest))))
+
+(defn- libjanet
+  "Find libjanet.a (or libjanet.lib on windows) at compile time"
+  []
+  (def libpath (dyn :libpath JANET_LIBPATH))
+  (unless libpath
+    (error "cannot find libpath: provide --libpath or JANET_LIBPATH"))
+  (string (dyn :libpath JANET_LIBPATH)
+          sep
+          (if is-win "libjanet.lib" "libjanet.a")))
+
+(defn- win-import-library
+  "On windows, an import library is needed to link to a dll statically."
+  []
+  (def hpath (dyn :headerpath JANET_HEADERPATH))
+  (unless hpath
+    (error "cannot find headerpath: provide --headerpath or JANET_HEADERPATH"))
+  (string hpath `\\janet.lib`))
+
+(defn- link-c
+  "Link object files together to make a native module."
+  [opts target & objects]
+  (def linker (opt opts (if is-win :linker :compiler) default-linker))
+  (def cflags (getcflags opts))
+  (def lflags [;(opt opts :lflags default-lflags)
+               ;(if (opts :static) [] dynamic-lflags)])
+  (rule target objects
+        (check-cc)
+        (print "linking " target "...")
+        (if is-win
+          (shell linker ;lflags (string "/OUT:" target) ;objects (win-import-library))
+          (shell linker ;cflags `-o` target ;objects ;lflags))))
+
+(defn- archive-c
+  "Link object files together to make a static library."
+  [opts target & objects]
+  (def ar (opt opts :archiver default-archiver))
+  (rule target objects
+        (check-cc)
+        (print "creating static library " target "...")
+        (if is-win
+          (shell ar "/nologo" (string "/out:" target) ;objects)
+          (shell ar "rcs" target ;objects))))
+
+(defn- create-buffer-c-impl
+  [bytes dest name]
+  (def out (file/open dest :w))
+  (def chunks (seq [b :in bytes] (string b)))
+  (file/write out
+              "#include <janet.h>\n"
+              "static const unsigned char bytes[] = {"
+              (string/join (interpose ", " chunks))
+              "};\n\n"
+              "const unsigned char *" name "_embed = bytes;\n"
+              "size_t " name "_embed_size = sizeof(bytes);\n")
+  (file/close out))
+
+(defn- create-buffer-c
+  "Inline raw byte file as a c file."
+  [source dest name]
+  (rule dest [source]
+        (print "generating " dest "...")
+        (with [f (file/open source :r)]
+          (create-buffer-c-impl (:read f :all) dest name))))
+
+(def- root-env (table/getproto (fiber/getenv (fiber/current))))
+
+(defn- modpath-to-meta
+  "Get the meta file path (.meta.janet) corresponding to a native module path (.so)."
+  [path]
+  (string (string/slice path 0 (- (length modext))) "meta.janet"))
+
+(defn- modpath-to-static
+  "Get the static library (.a) path corresponding to a native module path (.so)."
+  [path]
+  (string (string/slice path 0 (- -1 (length modext))) statext))
+
+(defn- create-executable
+  "Links an image with libjanet.a (or .lib) to produce an
+  executable. Also will try to link native modules into the
+  final executable as well."
+  [opts source dest]
+
+  # Create executable's janet image
+  (def cimage_dest (string dest ".c"))
+  (rule dest [source]
+        (check-cc)
+        (print "generating executable c source...")
+        # Load entry environment and get main function.
+        (def entry-env (dofile source))
+        (def main ((entry-env 'main) :value))
+        (def dep-lflags @[])
+
+        # Create marshalling dictionary
+        (def mdict (invert (env-lookup root-env)))
+        # Load all native modules
+        (def prefixes @{})
+        (def static-libs @[])
+        (loop [[name m] :pairs module/cache
+               :let [n (m :native)]
+               :when n
+               :let [prefix (gensym)]]
+          (print "found native " n "...")
+          (put prefixes prefix n)
+          (array/push static-libs (modpath-to-static n))
+          (def oldproto (table/getproto m))
+          (table/setproto m nil)
+          (loop [[sym value] :pairs (env-lookup m)]
+            (put mdict value (symbol prefix sym)))
+          (table/setproto m oldproto))
+
+        # Find static modules
+        (def declarations @"")
+        (def lookup-into-invocations @"")
+        (loop [[prefix name] :pairs prefixes]
+          (def meta (eval-string (slurp (modpath-to-meta name))))
+          (buffer/push-string lookup-into-invocations
+                              "    temptab = janet_table(0);\n"
+                              "    temptab->proto = env;\n"
+                              "    " (meta :static-entry) "(temptab);\n"
+                              "    janet_env_lookup_into(lookup, temptab, \""
+                              prefix
+                              "\", 0);\n\n")
+          (when-let [lfs (meta :lflags)]
+            (array/concat dep-lflags lfs))
+          (buffer/push-string declarations
+                              "extern void "
+                              (meta :static-entry)
+                              "(JanetTable *);\n"))
+
+
+        # Build image
+        (def image (marshal main mdict))
+        # Make image byte buffer
+        (create-buffer-c-impl image cimage_dest "janet_payload_image")
+        # Append main function
+        (spit cimage_dest (string
+                            "\n"
+                            declarations
+```
+
+int main(int argc, const char **argv) {
+    janet_init();
+
+    /* Get core env */
+    JanetTable *env = janet_core_env(NULL);
+    JanetTable *lookup = janet_env_lookup(env);
+    JanetTable *temptab;
+    int handle = janet_gclock();
+
+    /* Load natives into unmarshalling dictionary */
+
+```
+                            lookup-into-invocations
+```
+    /* Unmarshal bytecode */
+    Janet marsh_out = janet_unmarshal(
+      janet_payload_image_embed,
+      janet_payload_image_embed_size,
+      0,
+      lookup,
+      NULL);
+
+    /* Verify the marshalled object is a function */
+    if (!janet_checktype(marsh_out, JANET_FUNCTION)) {
+      fprintf(stderr, "invalid bytecode image - expected function.");
+      return 1;
+    }
+    JanetFunction *jfunc = janet_unwrap_function(marsh_out);
+
+    /* Check arity */
+    janet_arity(argc, jfunc->def->min_arity, jfunc->def->max_arity);
+
+    /* Collect command line arguments */
+    JanetArray *args = janet_array(argc);
+    for (int i = 0; i < argc; i++) {
+      janet_array_push(args, janet_cstringv(argv[i]));
+    }
+
+    /* Create enviornment */
+    temptab = janet_table(0);
+    temptab = env;
+    janet_table_put(temptab, janet_ckeywordv("args"), janet_wrap_array(args));
+    janet_gcroot(janet_wrap_table(temptab));
+
+    /* Unlock GC */
+    janet_gcunlock(handle);
+
+    /* Run everything */
+    JanetFiber *fiber = janet_fiber(jfunc, 64, argc, argc ? args->data : NULL);
+    fiber->env = temptab;
+    Janet out;
+    JanetSignal result = janet_continue(fiber, janet_wrap_nil(), &out);
+    if (result) {
+      janet_stacktrace(fiber, out);
+      janet_deinit();
+      return result;
+    }
+    janet_deinit();
+    return 0;
+}
+
+```) :ab)
+
+# Compile and link final exectable
+(do
+  (def extra-lflags (case (os/which)
+                      :macos ["-ldl" "-lm" ;thread-flags]
+                      :windows [;thread-flags]
+                      :linux ["-lm" "-ldl" "-lrt" ;thread-flags]
+                      #default
+                      ["-lm" ;thread-flags]))
+  (def cc (opt opts :compiler default-compiler))
+  (def lflags [;dep-lflags ;(opt opts :lflags default-lflags) ;extra-lflags])
+  (def cflags (getcflags opts))
+  (def defines (make-defines (opt opts :defines {})))
+  (print "compiling and linking " dest "...")
+  (if is-win
+    (shell cc ;cflags cimage_dest ;static-libs (libjanet) ;lflags `/link` (string "/OUT:" dest))
+    (shell cc ;cflags `-o` dest cimage_dest ;static-libs (libjanet) ;lflags)))))
+
+(defn- abspath
+  "Create an absolute path. Does not resolve . and .. (useful for
+  generating entries in install manifest file)."
+  [path]
+  (if (if is-win
+        (peg/match '(+ "\\" (* (range "AZ" "az") ":\\")) path)
+        (string/has-prefix? "/" path))
+    path
+    (string (os/cwd) sep path)))
+
+#
+# Public utilities
+#
+
+(defn find-manifest-dir
+  "Get the path to the directory containing manifests for installed
+  packages."
+  []
+  (string (dyn :modpath JANET_MODPATH) sep ".manifests"))
+
+(defn find-manifest
+  "Get the full path of a manifest file given a package name."
+  [name]
+  (string (find-manifest-dir) sep name ".txt"))
+
+(defn find-cache
+  "Return the path to the global cache."
+  []
+  (def path (dyn :modpath JANET_MODPATH))
+  (string path sep ".cache"))
+
+(defn uninstall
+  "Uninstall bundle named name"
+  [name]
+  (def manifest (find-manifest name))
+  (def f (file/open manifest :r))
+  (unless f (print manifest " does not exist") (break))
+  (loop [line :iterate (:read f :line)]
+    (def path ((string/split "\n" line) 0))
+    (def path ((string/split "\r" path) 0))
+    (print "removing " path)
+    (rm path))
+  (:close f)
+  (print "removing " manifest)
+  (rm manifest)
+  (print "Uninstalled."))
+
+(defn clear-cache
+  "Clear the global git cache."
+  []
+  (def cache (find-cache))
+  (print "clearing " cache "...")
+  (if is-win
+    # Git for windows decided that .git should be hidden and everything in it read-only.
+    # This means we can't delete things easily.
+    (os/shell (string `rmdir /S /Q "` cache `"`))
+    (rm cache)))
+
+(def- default-pkglist (or (os/getenv "JANET_PKGLIST") "https://github.com/janet-lang/pkgs.git"))
+
+(defn install-git
+  "Install a bundle from git. If the bundle is already installed, the bundle
+   is reinistalled (but not rebuilt if artifacts are cached)."
+  [repotab &opt recurse]
+  (def repo (if (string? repotab) repotab (repotab :repo)))
+  (def tag (unless (string? repotab) (repotab :tag)))
+  # prevent infinite recursion (very unlikely, but consider
+  # 'my-package "my-package" in the package listing)
+  (when (> (or recurse 0) 100)
+    (error "too many references resolving package url"))
+  # Handle short names
+  (unless (string/find ":" repo)
+    (def pkgs
+      (try (require "pkgs")
+                   ([err f]
+                    (install-git (dyn :pkglist default-pkglist))
+                    (require "pkgs"))))
+    (def next-repo (get-in pkgs ['packages :value (symbol repo)]))
+    (unless next-repo
+      (error (string "package " repo " not found.")))
+    (unless (or (string? next-repo) (dictionary? next-repo))
+      (error (string "expected string or table for repository, got " next-repo)))
+    (break (install-git next-repo (if recurse (inc recurse) 0))))
+  (def cache (find-cache))
+  (mkdir cache)
+  (def id (filepath-replace repo))
+  (def module-dir (string cache sep id))
+  (var fresh false)
+  (when (mkdir module-dir)
+      (set fresh true)
+      (print "cloning repository " repo " to " module-dir)
+      (os/execute ["git" "clone" repo module-dir] :p))
+  (def olddir (os/cwd))
+  (try
+    (with-dyns [:rules @{}
+                :modpath (abspath (dyn :modpath JANET_MODPATH))
+                :headerpath (abspath (dyn :headerpath JANET_HEADERPATH))
+                :libpath (abspath (dyn :libpath JANET_LIBPATH))
+                :binpath (abspath (dyn :binpath JANET_BINPATH))]
+      (os/cd module-dir)
+      (unless fresh
+        (os/execute ["git" "pull" "origin" "master"] :p))
+      (when tag
+        (os/execute ["git" "reset" "--hard" tag] :p))
+      (os/execute ["git" "submodule" "update" "--init" "--recursive"] :p)
+      (import-rules "./project.janet")
+      (do-rule "install-deps")
+      (do-rule "build")
+      (do-rule "install"))
+    ([err] (print "Error building git repository dependency: " err)))
+  (os/cd olddir))
+
+(defn install-rule
+  "Add install and uninstall rule for moving file from src into destdir."
+  [src destdir]
+  (def parts (peg/match path-splitter src))
+  (def name (last parts))
+  (def path (string destdir sep name))
+  (array/push (dyn :installed-files) path)
+  (add-body "install"
+            (mkdir destdir)
+            (copy src destdir)))
+
+#
+# Declaring Artifacts - used in project.janet, targets specifically
+# tailored for janet.
+#
+
+(defn declare-native
+  "Declare a native module. This is a shared library that can be loaded
+  dynamically by a janet runtime. This also builds a static libary that
+  can be used to bundle janet code and native into a single executable."
+  [&keys opts]
+  (def sources (opts :source))
+  (def name (opts :name))
+  (def path (dyn :modpath JANET_MODPATH))
+
+  # Make dynamic module
+  (def lname (string "build" sep name modext))
+  (loop [src :in sources]
+    (compile-c opts src (out-path src ".c" objext)))
+  (def objects (map (fn [path] (out-path path ".c" objext)) sources))
+  (when-let [embedded (opts :embedded)]
+            (loop [src :in embedded]
+              (def c-src (out-path src ".janet" ".janet.c"))
+              (def o-src (out-path src ".janet" (if is-win ".janet.obj" ".janet.o")))
+              (array/push objects o-src)
+              (create-buffer-c src c-src (embed-name src))
+              (compile-c opts c-src o-src)))
+  (link-c opts lname ;objects)
+  (add-dep "build" lname)
+  (install-rule lname path)
+
+  # Add meta file
+  (def metaname (modpath-to-meta lname))
+  (def ename (entry-name name))
+  (rule metaname []
+        (print "generating meta file " metaname "...")
+        (spit metaname (string/format
+                         "# Metadata for static library %s\n\n%.20p"
+                         (string name statext)
+                         {:static-entry ename
+                          :lflags ~',(opts :lflags)})))
+  (add-dep "build" metaname)
+  (install-rule metaname path)
+
+  # Make static module
+  (unless (dyn :nostatic)
+    (def sname (string "build" sep name statext))
+    (def opts (merge @{:entry-name ename} opts))
+    (def sobjext (string ".static" objext))
+    (def sjobjext (string ".janet" sobjext))
+    (loop [src :in sources]
+      (compile-c opts src (out-path src ".c" sobjext) true))
+    (def sobjects (map (fn [path] (out-path path ".c" sobjext)) sources))
+    (when-let [embedded (opts :embedded)]
+              (loop [src :in embedded]
+                (def c-src (out-path src ".janet" ".janet.c"))
+                (def o-src (out-path src ".janet" sjobjext))
+                (array/push sobjects o-src)
+                # Buffer c-src is already declared by dynamic module
+                (compile-c opts c-src o-src true)))
+    (archive-c opts sname ;sobjects)
+    (add-dep "build" sname)
+    (install-rule sname path)))
+
+(defn declare-source
+  "Create a Janet modules. This does not actually build the module(s),
+  but registers it for packaging and installation."
+  [&keys {:source sources}]
+  (def path (dyn :modpath JANET_MODPATH))
+  (if (bytes? sources)
+    (install-rule sources path)
+    (each s sources
+      (install-rule s path))))
+
+(defn declare-bin
+  "Declare a generic file to be installed as an executable."
+  [&keys {:main main}]
+  (install-rule main (dyn :binpath JANET_BINPATH)))
+
+(defn declare-executable
+  "Declare a janet file to be the entry of a standalone executable program. The entry
+  file is evaluated and a main function is looked for in the entry file. This function
+  is marshalled into bytecode which is then embedded in a final executable for distribution.\n\n
+  This executable can be installed as well to the --binpath given."
+  [&keys {:install install :name name :entry entry :headers headers}]
+  (def name (if is-win (string name ".exe") name))
+  (def dest (string "build" sep name))
+  (create-executable @{} entry dest)
+  (add-dep "build" dest)
+  (when headers
+    (each h headers (add-dep dest h)))
+  (when install
+    (install-rule dest (dyn :binpath JANET_BINPATH))))
+
+(defn declare-binscript
+  "Declare a janet file to be installed as an executable script. Creates
+  a shim on windows."
+  [&keys opts]
+  (def main (opts :main))
+  (def binpath (dyn :binpath JANET_BINPATH))
+  (install-rule main binpath)
+  # Create a dud batch file when on windows.
+  (when is-win
+    (def name (last (peg/match path-splitter main)))
+    (def fullname (string binpath sep name))
+    (def bat (string "@echo off\r\njanet \"" fullname "\" %*"))
+    (def newname (string binpath sep name ".bat"))
+    (array/push (dyn :installed-files) newname)
+    (add-body "install"
+              (spit newname bat))))
+
+(defn declare-archive
+  "Build a janet archive. This is a file that bundles together many janet
+  scripts into a janet image. This file can the be moved to any machine with
+  a janet vm and the required dependencies and run there."
+  [&keys opts]
+  (def entry (opts :entry))
+  (def name (opts :name))
+  (def iname (string "build" sep name ".jimage"))
+  (rule iname (or (opts :deps) [])
+        (spit iname (make-image (require entry))))
+  (def path (dyn :modpath JANET_MODPATH))
+  (add-dep "build" iname)
+  (install-rule iname path))
+
+(defn declare-project
+  "Define your project metadata. This should
+  be the first declaration in a project.janet file.
+  Also sets up basic phony targets like clean, build, test, etc."
+  [&keys meta]
+  (setdyn :project meta)
+
+  (def installed-files @[])
+  (def manifests (find-manifest-dir))
+  (def manifest (find-manifest (meta :name)))
+  (setdyn :manifest manifest)
+  (setdyn :manifest-dir manifests)
+  (setdyn :installed-files installed-files)
+
+  (rule "./build" [] (mkdir "build"))
+  (phony "build" ["./build"])
+
+  (phony "manifest" []
+         (print "generating " manifest "...")
+         (mkdir manifests)
+         (spit manifest (string (string/join installed-files "\n") "\n")))
+  (phony "install" ["uninstall" "build" "manifest"]
+         (when (dyn :test)
+           (do-rule "test"))
+         (print "Installed as '" (meta :name) "'."))
+
+  (phony "install-deps" []
+         (if-let [deps (meta :dependencies)]
+           (each dep deps
+             (install-git dep))
+           (print "no dependencies found")))
+
+  (phony "uninstall" []
+         (uninstall (meta :name)))
+
+  (phony "clean" []
+         (when (os/stat "./build" :mode)
+           (rm "build")
+           (print "Deleted build directory.")))
+
+  (phony "test" ["build"]
+         (defn dodir
+           [dir]
+           (each sub (sort (os/dir dir))
+             (def ndir (string dir sep sub))
+             (case (os/stat ndir :mode)
+               :file (when (string/has-suffix? ".janet" ndir)
+                       (print "running " ndir " ...")
+                       (def result (os/execute [(dyn :executable "janet") ndir] :p))
+                       (when (not= 0 result)
+                         (os/exit result)))
+               :directory (dodir ndir))))
+         (dodir "test")
+         (print "All tests passed.")))
+
+#
+# CLI
+#
+
+(def- argpeg
+  (peg/compile
+    '(* "--" '(some (if-not "=" 1)) (+ (* "=" '(any 1)) -1))))
+
+(defn- local-rule
+  [rule]
+  (import-rules "./project.janet")
+  (do-rule rule))
+
+(defn- help
+  []
+  (print `
+usage: jpm [--key=value, --flag] ... [subcommand] [args] ...
+
+Run from a directory containing a project.janet file to perform operations
+on a project, or from anywhere to do operations on the global module cache (modpath).
+
+Subcommands are:
+  build : build all artifacts
+  help : show this help text
+  install (repo or name) : install artifacts. If a repo is given, install the contents of that
+                   git repository, assuming that the repository is a jpm project. If not, build
+                   and install the current project.
+  uninstall (module) : uninstall a module. If no module is given, uninstall the module
+                       defined by the current directory.
+  clean : remove any generated files or artifacts
+  test : run tests. Tests should be .janet files in the test/ directory relative to project.janet.
+  deps : install dependencies for the current project.
+  clear-cache : clear the git cache. Useful for updating dependencies.
+  run rule : run a rule. Can also run custom rules added via (phony "task" [deps...] ...)
+             or (rule "ouput.file" [deps...] ...).
+  rules : list rules available with run.
+  update-pkgs : Update the current package listing from the remote git repository selected.
+  quickbin entry executable : Create an executable from a janet script with a main function.
+
+Keys are:
+  --modpath : The directory to install modules to. Defaults to $JANET_MODPATH, $JANET_PATH, or (dyn :syspath)
+  --headerpath : The directory containing janet headers. Defaults to $JANET_HEADERPATH.
+  --binpath : The directory to install binaries and scripts. Defaults to $JANET_BINPATH.
+  --libpath : The directory containing janet C libraries (libjanet.*). Defaults to $JANET_LIBPATH.
+  --compiler : C compiler to use for natives. Defaults to $CC or cc (cl.exe on windows).
+  --archiver : C compiler to use for static libraries. Defaults to $AR ar (lib.exe on windows).
+  --linker : C linker to use for linking natives. Defaults to link.exe on windows, not used on
+             other platforms.
+  --pkglist : URL of git repository for package listing. Defaults to $JANET_PKGLIST or https://github.com/janet-lang/pkgs.git
+
+Flags are:
+  --verbose : Print shell commands as they are executed.
+  --test : If passed to jpm install, runs tests before installing. Will run tests recursively on dependencies.
+    `))
+
+(defn- show-help
+  []
+  (print help))
+
+(defn- build
+  []
+  (local-rule "build"))
+
+(defn- clean
+  []
+  (local-rule "clean"))
+
+(defn- install
+  [&opt repo]
+  (if repo
+    (install-git repo)
+    (local-rule "install")))
+
+(defn- test
+  []
+  (local-rule "test"))
+
+(defn- uninstall-cmd
+  [&opt what]
+  (if what
+    (uninstall what)
+    (local-rule "uninstall")))
+
+(defn- deps
+  []
+  (local-rule "install-deps"))
+
+(defn- list-rules
+  []
+  (import-rules "./project.janet")
+  (def ks (sort (seq [k :keys (dyn :rules)] k)))
+  (each k ks (print k)))
+
+(defn- update-pkgs
+  []
+  (install-git (dyn :pkglist default-pkglist)))
+
+(defn- quickbin
+  [input output]
+  (create-executable @{} input output)
+  (do-rule output))
+
+(def- subcommands
+  {"build" build
+   "clean" clean
+   "help" show-help
+   "install" install
+   "test" test
+   "help" help
+   "deps" deps
+   "clear-cache" clear-cache
+   "run" local-rule
+   "rules" list-rules
+   "update-pkgs" update-pkgs
+   "uninstall" uninstall-cmd
+   "quickbin" quickbin})
+
+(def- args (tuple/slice (dyn :args) 1))
+(def- len (length args))
+(var i :private 0)
+
+# Get flags
+(while (< i len)
+  (if-let [m (peg/match argpeg (args i))]
+    (if (= 2 (length m))
+      (let [[key value] m]
+        (setdyn (keyword key) value))
+      (setdyn (keyword (m 0)) true))
+    (break))
+  (++ i))
+
+# Run subcommand
+(if (= i len)
+  (help)
+  (do
+    (if-let [com (subcommands (args i))]
+      (com ;(tuple/slice args (+ i 1)))
+      (do
+        (print "invalid command " (args i))
+        (help)))))

build_win.bat

@@ -13,50 +13,93 @@
 @if "%1"=="clean" goto CLEAN
 @if "%1"=="test" goto TEST
 @if "%1"=="dist" goto DIST
+@if "%1"=="install" goto INSTALL
+@if "%1"=="test-install" goto TESTINSTALL
+@if "%1"=="all" goto ALL
 
 @rem Set compile and link options here
 @setlocal
-@set JANET_COMPILE=cl /nologo /Isrc\include /c /O2 /W3 /LD /D_CRT_SECURE_NO_WARNINGS
+@set JANET_COMPILE=cl /nologo /Isrc\include /Isrc\conf /c /O2 /W3 /D_CRT_SECURE_NO_WARNINGS /MD
 @set JANET_LINK=link /nologo
+@set JANET_LINK_STATIC=lib /nologo
+
+@rem Add janet build tag
+if not "%JANET_BUILD%" == "" (
+    @set JANET_COMPILE=%JANET_COMPILE% /DJANET_BUILD="\"%JANET_BUILD%\""
+)
 
 mkdir build
 mkdir build\core
 mkdir build\mainclient
+mkdir build\boot
 
 @rem Build the xxd tool for generating sources
-@cl /nologo /c tools/xxd.c /Fobuild\xxd.obj
+cl /nologo /c tools/xxd.c /Fobuild\xxd.obj
 @if errorlevel 1 goto :BUILDFAIL
-@link /nologo /out:build\xxd.exe build\xxd.obj
+link /nologo /out:build\xxd.exe build\xxd.obj
 @if errorlevel 1 goto :BUILDFAIL
 
 @rem Generate the embedded sources
-@build\xxd.exe src\core\core.janet build\core.gen.c janet_gen_core
-@if errorlevel 1 goto :BUILDFAIL
-@build\xxd.exe src\mainclient\init.janet build\init.gen.c janet_gen_init
+build\xxd.exe src\boot\boot.janet build\boot.gen.c janet_gen_boot
 @if errorlevel 1 goto :BUILDFAIL
 
 @rem Build the generated sources
-@%JANET_COMPILE% /Fobuild\core\core.gen.obj build\core.gen.c
+%JANET_COMPILE% /Fobuild\boot\boot.gen.obj build\boot.gen.c
 @if errorlevel 1 goto :BUILDFAIL
-@%JANET_COMPILE% /Fobuild\mainclient\init.gen.obj build\init.gen.c
+
+@rem Build the bootstrap interpreter
+for %%f in (src\core\*.c) do (
+    %JANET_COMPILE% /DJANET_BOOTSTRAP /Fobuild\boot\%%~nf.obj %%f
+    @if errorlevel 1 goto :BUILDFAIL
+)
+for %%f in (src\boot\*.c) do (
+    %JANET_COMPILE% /DJANET_BOOTSTRAP /Fobuild\boot\%%~nf.obj %%f
+    @if errorlevel 1 goto :BUILDFAIL
+)
+%JANET_LINK% /out:build\janet_boot.exe build\boot\*.obj
+@if errorlevel 1 goto :BUILDFAIL
+build\janet_boot build\core_image.c
+
+@rem Build the core image
+%JANET_COMPILE% /Fobuild\core_image.obj build\core_image.c
 @if errorlevel 1 goto :BUILDFAIL
 
 @rem Build the sources
 for %%f in (src\core\*.c) do (
-    @%JANET_COMPILE% /Fobuild\core\%%~nf.obj %%f
+    %JANET_COMPILE% /Fobuild\core\%%~nf.obj %%f
     @if errorlevel 1 goto :BUILDFAIL
 )
 
+@rem Build the resources
+rc /nologo /fobuild\janet_win.res janet_win.rc
+
 @rem Build the main client
 for %%f in (src\mainclient\*.c) do (
-    @%JANET_COMPILE% /Fobuild\mainclient\%%~nf.obj %%f
+    %JANET_COMPILE% /Fobuild\mainclient\%%~nf.obj %%f
     @if errorlevel 1 goto :BUILDFAIL
 )
 
 @rem Link everything to main client
-%JANET_LINK% /out:janet.exe build\core\*.obj build\mainclient\*.obj
+%JANET_LINK% /out:janet.exe build\core\*.obj build\mainclient\*.obj build\core_image.obj build\janet_win.res
 @if errorlevel 1 goto :BUILDFAIL
 
+@rem Build static library (libjanet.a)
+%JANET_LINK_STATIC% /out:build\libjanet.lib build\core\*.obj build\core_image.obj
+@if errorlevel 1 goto :BUILDFAIL
+
+@rem Gen amlag
+setlocal enabledelayedexpansion
+set "amalg_files="
+for %%f in (src\core\*.c) do (
+    set "amalg_files=!amalg_files! %%f"
+)
+janet.exe tools\amalg.janet src\core\util.h src\core\state.h src\core\gc.h src\core\vector.h src\core\fiber.h src\core\regalloc.h src\core\compile.h src\core\emit.h src\core\symcache.h %amalg_files% build\core_image.c > build\janet.c
+janet.exe tools\removecr.janet build\janet.c
+
+@rem Gen shell.c
+janet.exe tools\amalg.janet src\mainclient\line.h src\mainclient\line.c src\mainclient\main.c > build\shell.c
+janet.exe tools\removecr.janet build\shell.c
+
 echo === Successfully built janet.exe for Windows ===
 echo === Run 'build_win test' to run tests. ==
 echo === Run 'build_win clean' to delete build artifacts. ===
@@ -78,15 +121,16 @@ exit /b 0
 
 @rem Clean build artifacts 
 :CLEAN
-del janet.exe janet.exp janet.lib
+del *.exe *.lib *.exp
 rd /s /q build
+rd /s /q dist
 exit /b 0
 
 @rem Run tests
 :TEST
 for %%f in (test/suite*.janet) do (
     janet.exe test\%%f
-    @if errorlevel 1 goto :TESTFAIL
+    @if errorlevel 1 goto TESTFAIL
 )
 exit /b 0
 
@@ -94,13 +138,75 @@ exit /b 0
 :DIST
 mkdir dist
 janet.exe tools\gendoc.janet > dist\doc.html
-janet.exe tools\amalg.janet > dist\janet.c
+janet.exe tools\removecr.janet dist\doc.html
+
+copy build\janet.c dist\janet.c
+copy build\shell.c dist\shell.c
 copy janet.exe dist\janet.exe
 copy LICENSE dist\LICENSE
 copy README.md dist\README.md
+
 copy janet.lib dist\janet.lib
 copy janet.exp dist\janet.exp
-copy src\include\janet\janet.h dist\janet.h
+
+copy src\include\janet.h dist\janet.h
+copy src\conf\janetconf.h dist\janetconf.h
+copy build\libjanet.lib dist\libjanet.lib
+
+copy auxbin\jpm dist\jpm
+copy tools\jpm.bat dist\jpm.bat
+
+@rem Create installer
+"C:\Program Files (x86)\NSIS\makensis.exe" janet-installer.nsi
+exit /b 0
+
+@rem Run the installer. (Installs to the local user with default settings)
+:INSTALL
+@echo Running Installer...
+FOR %%a in (janet-*-windows-installer.exe) DO (
+    %%a /S /CurrentUser
+)
+exit /b 0
+
+@rem Test the installation.
+:TESTINSTALL
+pushd test\install
+call jpm clean
+@if errorlevel 1 goto :TESTINSTALLFAIL
+call jpm test
+@if errorlevel 1 goto :TESTINSTALLFAIL
+call jpm --verbose --modpath=. install https://github.com/janet-lang/json.git
+@if errorlevel 1 goto :TESTINSTALLFAIL
+call build\testexec
+@if errorlevel 1 goto :TESTINSTALLFAIL
+call jpm --verbose quickbin testexec.janet build\testexec2.exe
+@if errorlevel 1 goto :TESTINSTALLFAIL
+call build\testexec2.exe
+@if errorlevel 1 goto :TESTINSTALLFAIL
+call jpm --verbose --test --modpath=. install https://github.com/janet-lang/jhydro.git
+@if errorlevel 1 goto :TESTINSTALLFAIL
+call jpm --verbose --test --modpath=. install https://github.com/janet-lang/path.git
+@if errorlevel 1 goto :TESTINSTALLFAIL
+call jpm --verbose --test --modpath=. install https://github.com/janet-lang/argparse.git
+@if errorlevel 1 goto :TESTINSTALLFAIL
+popd
+exit /b 0
+
+:TESTINSTALLFAIL
+popd
+goto :TESTFAIL
+
+@rem build, test, dist, install. Useful for local dev.
+:ALL
+call %0 build
+@if errorlevel 1 exit /b 1
+call %0 test
+@if errorlevel 1 exit /b 1
+call %0 dist
+@if errorlevel 1 exit /b 1
+call %0 install
+@if errorlevel 1 exit /b 1
+@echo Done!
 exit /b 0
 
 :TESTFAIL

examples/3sum.janet

@@ -14,5 +14,5 @@
   (map keys (keys solutions)))
 
 (def arr @[2 4 1 3 8 7 -3 -1 12 -5 -8])
-(print "3sum of " (string/pretty arr) ":")
-(print (string/pretty (sum3 arr)))
+(printf "3sum of %P: " arr)
+(printf "%P\n" (sum3 arr))

examples/debug.janet

@@ -0,0 +1,11 @@
+# Load this file and run (myfn) to see the debugger
+
+(defn myfn
+  []
+  (debug)
+  (for i 0 10 (print i)))
+
+(debug/fbreak myfn 3)
+
+# Enable debugging in repl with
+# (setdyn :debug true)

examples/debugger.janet

@@ -0,0 +1,153 @@
+###
+### A useful debugger library for Janet. Should be used
+### inside a debug repl.
+###
+
+(defn .fiber
+  "Get the current fiber being debugged."
+  []
+  (if-let [entry (dyn '_fiber)]
+    (entry :value)
+    (dyn :fiber)))
+
+(defn .stack
+  "Print the current fiber stack"
+  []
+  (print)
+  (debug/stacktrace (.fiber) "")
+  (print))
+
+(defn .frame
+  "Show a stack frame"
+  [&opt n]
+  (def stack (debug/stack (.fiber)))
+  (in stack (or n 0)))
+
+(defn .fn
+  "Get the current function"
+  [&opt n]
+  (in (.frame n) :function))
+
+(defn .slots
+  "Get an array of slots in a stack frame"
+  [&opt n]
+  (in (.frame n) :slots))
+
+(defn .slot
+  "Get the value of the nth slot."
+  [&opt nth frame-idx]
+  (in (.slots frame-idx) (or nth 0)))
+
+(defn .quit
+  "Resume (dyn :fiber) with the value passed to it after exiting the debugger."
+  [&opt val]
+  (setdyn :exit true)
+  (setdyn :resume-value val)
+  nil)
+
+(defn .disasm
+  "Gets the assembly for the current function."
+  [&opt n]
+  (def frame (.frame n))
+  (def func (frame :function))
+  (disasm func))
+
+(defn .bytecode
+  "Get the bytecode for the current function."
+  [&opt n]
+  ((.disasm n) 'bytecode))
+
+(defn .ppasm
+  "Pretty prints the assembly for the current function"
+  [&opt n]
+  (def frame (.frame n))
+  (def func (frame :function))
+  (def dasm (disasm func))
+  (def bytecode (dasm 'bytecode))
+  (def pc (frame :pc))
+  (def sourcemap (dasm 'sourcemap))
+  (var last-loc [-2 -2])
+  (print "\n  function:   " (dasm 'name) " [" (in dasm 'source "") "]")
+  (when-let [constants (dasm 'constants)]
+    (printf "  constants:  %.4Q\n" constants))
+  (printf "  slots:      %.4Q\n\n" (frame :slots))
+  (def padding (string/repeat " " 20))
+  (loop [i :range [0 (length bytecode)]
+         :let [instr (bytecode i)]]
+    (prin (if (= (tuple/type instr) :brackets) "*" " "))
+    (prin (if (= i pc) "> " "  "))
+    (printf "\e[33m%.20s\e[0m" (string (string/join (map string instr) " ") padding))
+    (when sourcemap
+      (let [[sl sc] (sourcemap i)
+            loc [sl sc]]
+        (when (not= loc last-loc)
+          (set last-loc loc)
+          (prin " # line " sl ", column " sc))))
+    (print))
+  (print))
+
+(defn .source
+  "Show the source code for the function being debugged."
+  [&opt n]
+  (def frame (.frame n))
+  (def s (frame :source))
+  (def all-source (slurp s))
+  (print "\n\e[33m" all-source "\e[0m\n"))
+
+(defn .breakall
+  "Set breakpoints on all instructions in the current function."
+  [&opt n]
+  (def fun (.fn n))
+  (def bytecode (.bytecode n))
+  (for i 0 (length bytecode)
+    (debug/fbreak fun i))
+  (print "Set " (length bytecode) " breakpoints in " fun))
+
+(defn .clearall
+  "Clear all breakpoints on the current function."
+  [&opt n]
+  (def fun (.fn n))
+  (def bytecode (.bytecode n))
+  (for i 0 (length bytecode)
+    (debug/unfbreak fun i))
+  (print "Cleared " (length bytecode) " breakpoints in " fun))
+
+(defn .break
+  "Set breakpoint at the current pc."
+  []
+  (def frame (.frame))
+  (def fun (frame :function))
+  (def pc (frame :pc))
+  (debug/fbreak fun pc)
+  (print "Set breakpoint in " fun " at pc=" pc))
+
+(defn .clear
+  "Clear the current breakpoint"
+  []
+  (def frame (.frame))
+  (def fun (frame :function))
+  (def pc (frame :pc))
+  (debug/unfbreak fun pc)
+  (print "Cleared breakpoint in " fun " at pc=" pc))
+
+(defn .next
+  "Go to the next breakpoint."
+  [&opt n]
+  (var res nil)
+  (for i 0 (or n 1)
+    (set res (resume (.fiber))))
+  res)
+
+(defn .nextc
+  "Go to the next breakpoint, clearing the current breakpoint."
+  [&opt n]
+  (.clear)
+  (.next n))
+
+(defn .step
+  "Execute the next n instructions."
+  [&opt n]
+  (var res nil)
+  (for i 0 (or n 1)
+    (set res (debug/step (.fiber))))
+  res)

examples/life.janet

@@ -4,11 +4,11 @@
   (seq [x :range [-1 2]
          y :range [-1 2]
          :when (not (and (zero? x) (zero? y)))]
-        (tuple x y)))
+       [x y]))
 
 (defn- neighbors
   [[x y]]
-  (map (fn [[x1 y1]] (tuple (+ x x1) (+ y y1))) window))
+  (map (fn [[x1 y1]] [(+ x x1) (+ y y1)]) window))
 
 (defn tick
   "Get the next state in the Game Of Life."
@@ -28,7 +28,7 @@
   (loop [x :range [x1 (+ 1 x2)]
          :after (print)
          y :range [y1 (+ 1 y2)]]
-    (file/write stdout (if (get cellset (tuple x y)) "X " ". ")))
+    (file/write stdout (if (get cellset [x y]) "X " ". ")))
   (print))
 
 #

examples/maxtriangle.janet

@@ -2,10 +2,8 @@
 # of the triangle to the leaves of the triangle.
 
 (defn myfold [xs ys]
-  (let [xs1 (tuple/prepend xs 0)
-        xs2 (tuple/append xs 0)
-        m1 (map + xs1 ys)
-        m2 (map + xs2 ys)]
+  (let [m1 (map + [;xs 0] ys)
+        m2 (map + [0 ;xs] ys)]
     (map max m1 m2)))
 
 (defn maxpath [t]

examples/numarray/.gitignore

@@ -0,0 +1 @@
+/build

examples/numarray/build.janet

@@ -0,0 +1,23 @@
+(import cook)
+
+(cook/make-native
+    :name "numarray"
+    :source @["numarray.c"])
+
+(import build/numarray :as numarray)
+
+(def a (numarray/new 30))
+(print (get a 20))
+(print (a 20))
+
+(put a 5 3.14)
+(print (a 5))
+(set (a 5) 100)
+(print (a 5))
+
+# (numarray/scale a 5))
+# ((a :scale) a 5)
+(:scale a 5)
+(for i 0 10 (print (a i)))
+
+(print "sum=" (:sum a))

examples/numarray/numarray.c

@@ -0,0 +1,117 @@
+#include <stdlib.h>
+#include <janet.h>
+
+typedef struct {
+    double *data;
+    size_t size;
+} num_array;
+
+static num_array *num_array_init(num_array *array, size_t size) {
+    array->data = (double *)calloc(size, sizeof(double));
+    array->size = size;
+    return array;
+}
+
+static void num_array_deinit(num_array *array) {
+    free(array->data);
+}
+
+static int num_array_gc(void *p, size_t s) {
+    (void) s;
+    num_array *array = (num_array *)p;
+    num_array_deinit(array);
+    return 0;
+}
+
+Janet num_array_get(void *p, Janet key);
+void num_array_put(void *p, Janet key, Janet value);
+
+static const JanetAbstractType num_array_type = {
+    "numarray",
+    num_array_gc,
+    NULL,
+    num_array_get,
+    num_array_put
+};
+
+static Janet num_array_new(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    int32_t size = janet_getinteger(argv, 0);
+    num_array *array = (num_array *)janet_abstract(&num_array_type, sizeof(num_array));
+    num_array_init(array, size);
+    return janet_wrap_abstract(array);
+}
+
+static Janet num_array_scale(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 2);
+    num_array *array = (num_array *)janet_getabstract(argv, 0, &num_array_type);
+    double factor = janet_getnumber(argv, 1);
+    size_t i;
+    for (i = 0; i < array->size; i++) {
+        array->data[i] *= factor;
+    }
+    return argv[0];
+}
+
+static Janet num_array_sum(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    num_array *array = (num_array *)janet_getabstract(argv, 0, &num_array_type);
+    double sum = 0;
+    for (size_t i = 0; i < array->size; i++) sum += array->data[i];
+    return janet_wrap_number(sum);
+}
+
+void num_array_put(void *p, Janet key, Janet value) {
+    size_t index;
+    num_array *array = (num_array *)p;
+    if (!janet_checkint(key))
+        janet_panic("expected integer key");
+    if (!janet_checktype(value, JANET_NUMBER))
+        janet_panic("expected number value");
+
+    index = (size_t)janet_unwrap_integer(key);
+    if (index < array->size) {
+        array->data[index] = janet_unwrap_number(value);
+    }
+}
+
+static const JanetMethod methods[] = {
+    {"scale", num_array_scale},
+    {"sum", num_array_sum},
+    {NULL, NULL}
+};
+
+Janet num_array_get(void *p, Janet key) {
+    size_t index;
+    Janet value;
+    num_array *array = (num_array *)p;
+    if (janet_checktype(key, JANET_KEYWORD))
+        return janet_getmethod(janet_unwrap_keyword(key), methods);
+    if (!janet_checkint(key))
+        janet_panic("expected integer key");
+    index = (size_t)janet_unwrap_integer(key);
+    if (index >= array->size) {
+        value = janet_wrap_nil();
+    } else {
+        value = janet_wrap_number(array->data[index]);
+    }
+    return value;
+}
+
+static const JanetReg cfuns[] = {
+    {
+        "new", num_array_new,
+        "(numarray/new size)\n\n"
+        "Create new numarray"
+    },
+    {
+        "scale", num_array_scale,
+        "(numarray/scale numarray factor)\n\n"
+        "scale numarray by factor"
+    },
+    {NULL, NULL, NULL}
+};
+
+JANET_MODULE_ENTRY(JanetTable *env) {
+    janet_cfuns(env, "numarray", cfuns);
+}

examples/primes.janet

@@ -13,4 +13,4 @@
     (if isprime? (array/push list i)))
   list)
 
-(print (string/pretty (primes 100)))
+(pp (primes 100))

examples/rtest.janet

@@ -0,0 +1,11 @@
+# How random is the RNG really?
+
+(def counts (seq [_ :range [0 100]] 0))
+
+(for i 0 1000000
+  (let [x (math/random)
+        intrange (math/floor (* 100 x))
+        oldcount (counts intrange)]
+    (put counts intrange (if oldcount (+ 1 oldcount) 1))))
+
+(pp counts)

examples/tarray.janet

@@ -0,0 +1,73 @@
+# naive matrix implementation for testing typed array
+
+(defn matrix [nrow ncol] {:nrow nrow :ncol ncol :array (tarray/new :float64 (* nrow ncol))})
+
+(defn matrix/row [mat i]
+  (def {:nrow nrow :ncol ncol :array array} mat)
+  (tarray/new :float64 ncol 1 (* i ncol)  array))
+
+(defn matrix/column [mat j]
+  (def {:nrow nrow :ncol ncol :array array} mat)
+  (tarray/new :float64 nrow ncol j array))
+
+(defn matrix/set [mat i j value]
+  (def {:nrow nrow :ncol ncol :array array} mat)
+  (set (array (+ (* i ncol) j)) value))
+
+(defn matrix/get [mat i j value]
+  (def {:nrow nrow :ncol ncol :array array} mat)
+  (array (+ (* i ncol) j)))
+
+
+# other variants to test rows and cols views
+
+(defn matrix/set* [mat i j value]
+  (set ((matrix/row mat i) j) value))
+
+(defn matrix/set** [mat i j value]
+  (set ((matrix/column mat j) i) value))
+
+
+(defn matrix/get* [mat i j value]
+  ((matrix/row mat i) j))
+
+(defn matrix/get** [mat i j value]
+  ((matrix/column mat j) i))
+
+
+(defn tarray/print [arr]
+  (def size (tarray/length arr))
+  (prinf "[%2i]" size)
+  (for i 0 size
+    (prinf " %+6.3f " (arr i)))
+  (print))
+
+(defn matrix/print [mat]
+  (def {:nrow nrow :ncol ncol :array tarray} mat)
+  (printf "matrix %iX%i %p" nrow ncol tarray)
+  (for i 0 nrow
+    (tarray/print (matrix/row mat i))))
+
+
+(def nr 5)
+(def nc 4)
+(def A (matrix nr nc))
+
+(loop (i :range (0 nr) j :range (0 nc)) 
+  (matrix/set A i j i))
+(matrix/print A)
+
+(loop (i :range (0 nr) j :range (0 nc)) 
+  (matrix/set* A i j i))
+(matrix/print A)
+
+(loop (i :range (0 nr) j :range (0 nc)) 
+  (matrix/set** A i j i))
+(matrix/print A)
+
+
+(printf "properties:\n%p" (tarray/properties (A :array)))
+(for i 0 nr  
+  (printf "row properties:[%i]\n%p" i (tarray/properties (matrix/row A i))))
+(for i 0 nc  
+  (printf "col properties:[%i]\n%p" i (tarray/properties (matrix/column A i))))

examples/threads.janet

@@ -0,0 +1,70 @@
+(defn worker-main
+  "Sends 11 messages back to parent"
+  [parent]
+  (def name (thread/receive))
+  (def interval (thread/receive))
+  (for i 0 10
+    (os/sleep interval)
+    (:send parent (string/format "thread %s wakeup no. %d" name i)))
+  (:send parent name))
+
+(defn make-worker
+  [name interval]
+  (-> (thread/new)
+      (:send worker-main)
+      (:send name)
+      (:send interval)))
+
+(def bob (make-worker "bob" 0.02))
+(def joe (make-worker "joe" 0.03))
+(def sam (make-worker "sam" 0.05))
+
+# Receive out of order
+(for i 0 33
+  (print (thread/receive)))
+
+#
+# Recursive Thread Tree - should pause for a bit, and then print a cool zigzag.
+#
+
+(def rng (math/rng (os/cryptorand 16)))
+
+(defn choose [& xs]
+  (in xs (:int rng (length xs))))
+
+(defn worker-tree
+  [parent]
+  (def name (thread/receive))
+  (def depth (thread/receive))
+  (if (< depth 5)
+    (do
+    (defn subtree []
+      (-> (thread/new)
+          (:send worker-tree)
+          (:send (string name "/" (choose "bob" "marley" "harry" "suki" "anna" "yu")))
+          (:send (inc depth))))
+    (let [l (subtree)
+          r (subtree)
+          lrep (thread/receive)
+          rrep (thread/receive)]
+      (:send parent [name ;lrep ;rrep])))
+    (do
+      (:send parent [name]))))
+
+(-> (thread/new) (:send worker-tree) (:send "adam") (:send 0))
+(def lines (thread/receive))
+(map print lines)
+
+#
+# Receive timeout
+#
+
+(def slow (make-worker "slow-loras" 0.5))
+(for i 0 50
+  (try
+    (let [msg (thread/receive 0.1)]
+      (print "\n" msg))
+    ([err] (prin ".") (:flush stdout))))
+
+(print "\ndone timing, timeouts ending.")
+(try (while true (print (thread/receive))) ([err] (print "done")))

examples/urlloader.janet

@@ -0,0 +1,29 @@
+# An example of using Janet's extensible module system
+# to import files from URL. To try this, run `janet -l examples/urlloader.janet`
+# from the repl, and then:
+#
+# (import https://raw.githubusercontent.com/janet-lang/janet/master/examples/colors.janet :as c)
+#
+# This will import a file using curl. You can then try
+#
+# (print (c/color :green "Hello!"))
+#
+# This is a bit of a toy example (it just shells out to curl), but it is very
+# powerful and will work well in many cases.
+
+(defn- load-url
+  [url args]
+  (def f (file/popen (string "curl " url)))
+  (def res (dofile f :source url ;args))
+  (try (file/close f) ([err] nil))
+  res)
+
+(defn- check-http-url
+  [path]
+  (if (or (string/has-prefix? "http://" path)
+          (string/has-prefix? "https://" path))
+    path))
+
+# Add the module loader and path tuple to right places
+(array/push module/paths [check-http-url :janet-http])
+(put module/loaders :janet-http load-url)

janet-installer.nsi

@@ -1,55 +1,202 @@
+# Version
+!define VERSION "1.6.0"
+!define PRODUCT_VERSION "${VERSION}.0"
+VIProductVersion "${PRODUCT_VERSION}"
+VIFileVersion "${PRODUCT_VERSION}"
+
+# Use the modern UI
 !define MULTIUSER_EXECUTIONLEVEL Highest
 !define MULTIUSER_MUI
 !define MULTIUSER_INSTALLMODE_COMMANDLINE
-!define MULTIUSER_INSTALLMODE_INSTDIR "janet"
+!define MULTIUSER_INSTALLMODE_DEFAULT_REGISTRY_KEY "Software\Janet\${VERSION}"
+!define MULTIUSER_INSTALLMODE_DEFAULT_REGISTRY_VALUENAME ""
+!define MULTIUSER_INSTALLMODE_INSTDIR_REGISTRY_KEY "Software\Janet\${VERSION}"
+!define MULTIUSER_INSTALLMODE_INSTDIR_REGISTRY_VALUENAME ""
+!define MULTIUSER_INSTALLMODE_INSTDIR "Janet-${VERSION}"
+
+# For now, use 32 bit folder as build is 32 bit
+# !define MULTIUSER_USE_PROGRAMFILES64
+
+# Includes
 !include "MultiUser.nsh"
 !include "MUI2.nsh"
- 
+!include ".\tools\EnvVarUpdate.nsh"
+!include "LogicLib.nsh"
+
+# Basics
 Name "Janet"
-OutFile "janet-install.exe"
- 
+
+# Do some NSIS-fu to figure out at compile time if we are in appveyor
+!define OUTNAME $%APPVEYOR_REPO_TAG_NAME%
+!define "CHECK_${OUTNAME}"
+!define DOLLAR "$"
+!ifdef CHECK_${DOLLAR}%APPVEYOR_REPO_TAG_NAME%
+    # We are not in the appveyor environment, use version name
+    !define OUTNAME_PART v${VERSION}
+!else
+    # We are in appveyor, use git tag name for installer
+    !define OUTNAME_PART ${OUTNAME}
+!endif
+OutFile "janet-${OUTNAME_PART}-windows-installer.exe"
+
+# Some Configuration
+!define APPNAME "Janet"
+!define DESCRIPTION "The Janet Programming Language"
+!define HELPURL "http://janet-lang.org"
+BrandingText "The Janet Programming Language"
+
+# Macros for setting registry values
+!define UNINST_KEY "Software\Microsoft\Windows\CurrentVersion\Uninstall\Janet-${VERSION}"
+!macro WriteEnv key value
+    ${If} $MultiUser.InstallMode == "AllUsers"
+        WriteRegExpandStr HKLM "SYSTEM\CurrentControlSet\Control\Session Manager\Environment" "${key}" "${value}"
+    ${Else}
+        WriteRegExpandStr HKCU "Environment" "${key}" "${value}"
+    ${EndIf}
+!macroend
+!macro DelEnv key
+    ${If} $MultiUser.InstallMode == "AllUsers"
+        DeleteRegValue HKLM "SYSTEM\CurrentControlSet\Control\Session Manager\Environment" "${key}"
+    ${Else}
+        DeleteRegValue HKCU "Environment" "${key}"
+    ${EndIf}
+!macroend
+
+# MUI Configuration
+!define MUI_ICON "assets\icon.ico"
+!define MUI_UNICON "assets\icon.ico"
+!define MUI_HEADERIMAGE
+!define MUI_HEADERIMAGE_BITMAP "assets\janet-w200.png"
+!define MUI_HEADERIMAGE_RIGHT
 !define MUI_ABORTWARNING
- 
+
+# Show a welcome page first
 !insertmacro MUI_PAGE_WELCOME
 !insertmacro MUI_PAGE_LICENSE "LICENSE"
-!insertmacro MUI_PAGE_COMPONENTS
+
+# Pick Install Directory
 !insertmacro MULTIUSER_PAGE_INSTALLMODE
 !insertmacro MUI_PAGE_DIRECTORY
-
 !insertmacro MUI_PAGE_INSTFILES
 
+# Done
 !insertmacro MUI_PAGE_FINISH
 
-!insertmacro MUI_UNPAGE_CONFIRM
-!insertmacro MUI_UNPAGE_INSTFILES
- 
+# Need to set a language.
 !insertmacro MUI_LANGUAGE "English"
- 
-Section "Janet" BfWSection
-  SetOutPath $INSTDIR
-  File "janet.exe"
-  WriteUninstaller "$INSTDIR\janet-uninstall.exe"
-  
-  # Start Menu
-  CreateShortCut "$SMPROGRAMS\Janet.lnk" "$INSTDIR\janet.exe" "" ""
-SectionEnd
 
-Function .onInit
-  !insertmacro MULTIUSER_INIT
-  !insertmacro MUI_LANGDLL_DISPLAY
-FunctionEnd
+function .onInit
+    !insertmacro MULTIUSER_INIT
+functionEnd
 
-!insertmacro MUI_FUNCTION_DESCRIPTION_BEGIN
-  !insertmacro MUI_DESCRIPTION_TEXT ${BfWSection} "The Janet programming language."
-!insertmacro MUI_FUNCTION_DESCRIPTION_END
- 
-Section "Uninstall"
-  Delete "$INSTDIR\janet.exe"
-  Delete "$INSTDIR\janet-uninstall.exe"
-  RMDir "$INSTDIR"
-SectionEnd
- 
-Function un.onInit
-  !insertmacro MULTIUSER_UNINIT
-  !insertmacro MUI_UNGETLANGUAGE
-FunctionEnd
+section "Janet" BfWSection
+
+    createDirectory "$INSTDIR\Library"
+    createDirectory "$INSTDIR\C"
+    createDirectory "$INSTDIR\bin"
+    createDirectory "$INSTDIR\docs"
+    setOutPath "$INSTDIR"
+
+    # Bin files
+    file /oname=bin\janet.exe dist\janet.exe
+    file /oname=logo.ico assets\icon.ico
+    file /oname=bin\jpm.janet auxbin\jpm
+    file /oname=bin\jpm.bat tools\jpm.bat
+
+    # C headers and library files
+    file /oname=C\janet.h dist\janet.h
+    file /oname=C\janetconf.h dist\janetconf.h
+    file /oname=C\janet.lib dist\janet.lib
+    file /oname=C\janet.exp dist\janet.exp
+    file /oname=C\janet.c dist\janet.c
+    file /oname=C\libjanet.lib dist\libjanet.lib
+
+    # Documentation
+    file /oname=docs\docs.html dist\doc.html
+
+    # Other
+    file README.md
+    file LICENSE
+
+    # Uninstaller - See function un.onInit and section "uninstall" for configuration
+    writeUninstaller "$INSTDIR\uninstall.exe"
+
+    # Start Menu
+    createShortCut "$SMPROGRAMS\Janet.lnk" "$INSTDIR\bin\janet.exe" "" "$INSTDIR\logo.ico"
+
+    # Set up Environment variables
+    !insertmacro WriteEnv JANET_PATH "$INSTDIR\Library"
+    !insertmacro WriteEnv JANET_HEADERPATH "$INSTDIR\C"
+    !insertmacro WriteEnv JANET_LIBPATH "$INSTDIR\C"
+    !insertmacro WriteEnv JANET_BINPATH "$INSTDIR\bin"
+
+    SendMessage ${HWND_BROADCAST} ${WM_WININICHANGE} 0 "STR:Environment" /TIMEOUT=5000
+
+    # Update path
+    ${If} $MultiUser.InstallMode == "AllUsers"
+        ${EnvVarUpdate} $0 "PATH" "A" "HKLM" "$INSTDIR\bin" ; Append
+    ${Else}
+        ${EnvVarUpdate} $0 "PATH" "A" "HKCU" "$INSTDIR\bin" ; Append
+    ${EndIf}
+
+    # Registry information for add/remove programs
+    WriteRegStr SHCTX "${UNINST_KEY}" "DisplayName" "Janet"
+    WriteRegStr SHCTX "${UNINST_KEY}" "InstallLocation" "$INSTDIR"
+    WriteRegStr SHCTX "${UNINST_KEY}" "DisplayIcon" "$INSTDIR\logo.ico"
+    WriteRegStr SHCTX "${UNINST_KEY}" "Publisher" "Janet-Lang.org"
+    WriteRegStr SHCTX "${UNINST_KEY}" "HelpLink" "${HELPURL}"
+    WriteRegStr SHCTX "${UNINST_KEY}" "URLUpdateInfo" "${HELPURL}"
+    WriteRegStr SHCTX "${UNINST_KEY}" "URLInfoAbout" "${HELPURL}"
+    WriteRegStr SHCTX "${UNINST_KEY}" "DisplayVersion" "${VERSION}"
+    WriteRegDWORD SHCTX "${UNINST_KEY}" "NoModify" 1
+    WriteRegDWORD SHCTX "${UNINST_KEY}" "NoRepair" 1
+    WriteRegDWORD SHCTX "${UNINST_KEY}" "EstimatedSize" 1000
+    # Add uninstall
+    WriteRegStr SHCTX "${UNINST_KEY}" "UninstallString" "$\"$INSTDIR\uninstall.exe$\" /$MultiUser.InstallMode"
+    WriteRegStr SHCTX "${UNINST_KEY}" "QuietUninstallString" "$\"$INSTDIR\uninstall.exe$\" /$MultiUser.InstallMode /S"
+
+sectionEnd
+
+# Uninstaller
+
+function un.onInit
+    !insertmacro MULTIUSER_UNINIT
+functionEnd
+
+section "uninstall"
+
+    # Remove Start Menu launcher
+    delete "$SMPROGRAMS\Janet.lnk"
+
+    # Remove files
+    delete "$INSTDIR\logo.ico"
+    delete "$INSTDIR\README.md"
+    delete "$INSTDIR\LICENSE"
+    rmdir /r "$INSTDIR\Library"
+    rmdir /r "$INSTDIR\bin"
+    rmdir /r "$INSTDIR\C"
+    rmdir /r "$INSTDIR\docs"
+
+    # Remove env vars
+    !insertmacro DelEnv JANET_PATH
+    !insertmacro DelEnv JANET_HEADERPATH
+    !insertmacro DelEnv JANET_LIBPATH
+    !insertmacro DelEnv JANET_BINPATH
+
+    # Unset PATH
+    ${If} $MultiUser.InstallMode == "AllUsers"
+        ${un.EnvVarUpdate} $0 "PATH" "R" "HKLM" "$INSTDIR\bin" ; Remove
+    ${Else}
+        ${un.EnvVarUpdate} $0 "PATH" "R" "HKCU" "$INSTDIR\bin" ; Remove
+    ${EndIf}
+
+    # make sure windows knows about the change
+    SendMessage ${HWND_BROADCAST} ${WM_WININICHANGE} 0 "STR:Environment" /TIMEOUT=5000
+
+    # Always delete uninstaller as the last action
+    delete "$INSTDIR\uninstall.exe"
+
+    # Remove uninstaller information from the registry
+    DeleteRegKey SHCTX "${UNINST_KEY}"
+
+sectionEnd

janet.1

@@ -3,20 +3,23 @@
 janet \- run the Janet language abstract machine
 .SH SYNOPSIS
 .B janet
-[\fB\-hvsrp\fR]
-[\fB\-e\fR \fIJANET SOURCE\fR]
+[\fB\-hvsrpnqk\fR]
+[\fB\-e\fR \fISOURCE\fR]
+[\fB\-l\fR \fIMODULE\fR]
+[\fB\-m\fR \fIPATH\fR]
+[\fB\-c\fR \fIMODULE JIMAGE\fR]
 [\fB\-\-\fR]
 .IR script
 .IR args ...
 .SH DESCRIPTION
 Janet is a functional and imperative programming language and bytecode interpreter.
 It is a modern lisp, but lists are replaced by other data structures with better utility
-and performance (arrays, tables, structs, tuples). The language also bridging bridging
+and performance (arrays, tables, structs, tuples). The language also features bridging
 to native code written in C, meta-programming with macros, and bytecode assembly.
 
 There is a repl for trying out the language, as well as the ability to run script files.
 This client program is separate from the core runtime, so Janet could be embedded
-into other programs. Try Janet in your browser at https://Janet-lang.org.
+into other programs. Try Janet in your browser at https://janet-lang.org.
 
 Implemented in mostly standard C99, Janet runs on Windows, Linux and macOS.
 The few features that are not standard C99 (dynamic library loading, compiler
@@ -41,8 +44,13 @@ Shows the version text and exits immediately.
 Read raw input from stdin and forgo prompt history and other readline-like features.
 
 .TP
-.BR \-q
-Quiet output. Don't print a repl prompt or expression results to stdout.
+.BR \-e\ code
+Execute a string of Janet source. Source code is executed in the order it is encountered, so earlier
+arguments are executed before later ones.
+
+.TP
+.BR \-n
+Disable ANSI colors in the repl. Has no effect if no repl is run.
 
 .TP
 .BR \-r
@@ -56,19 +64,35 @@ it will immediately exit after printing the error message. In persistent mode, J
 after an error. Persistent mode can be good for debugging and testing.
 
 .TP
-.BR \-e
-Execute a string of Janet source. Source code is executed in the order it is encountered, so earlier
-arguments are executed before later ones.
+.BR \-q
+Quiet output. Don't print a repl prompt or expression results to stdout.
 
 .TP
-.BR \-l
+.BR \-k
+Don't execute a script, only compile it to check for errors. Useful for linting scripts.
+
+.TP
+.BR \-m\ syspath
+Set the dynamic binding :syspath to the string syspath so that Janet will load system modules
+from a directory different than the default. The default is set when Janet is built, and defaults to
+/usr/local/lib/janet on Linux/Posix, and C:/Janet/Library on Windows. This option supersedes JANET_PATH.
+
+.TP
+.BR \-c\ source\ output
+Precompiles Janet source code into an image, a binary dump that can be efficiently loaded later.
+Source should be a path to the Janet module to compile, and output should be the file path of
+resulting image. Output should usually end with the .jimage extension.
+
+.TP
+.BR \-l\ path
 Load a Janet file before running a script or repl. Multiple files can be loaded
 in this manner, and exports from each file will be made available to the script
 or repl.
 
 .TP
 .BR \-\-
-Stop parsing command line arguments. All arguments after this one will be considered file names.
+Stop parsing command line arguments. All arguments after this one will be considered file names
+and then arguments to the script.
 
 .SH ENVIRONMENT
 
@@ -76,9 +100,7 @@ Stop parsing command line arguments. All arguments after this one will be consid
 .RS
 The location to look for Janet libraries. This is the only environment variable Janet needs to
 find native and source code modules. If no JANET_PATH is set, Janet will look in
-/usr/local/lib/Janet for modules.
-To make Janet search multiple locations, modify the module.paths
-array in Janet.
+the default location set at compile time.
 .RE
 
 .SH AUTHOR

janet_win.rc

@@ -0,0 +1 @@
+IDI_MYICON ICON "assets\icon.ico"

jpm.1

@@ -0,0 +1,202 @@
+.TH JPM 1
+.SH NAME
+jpm \- the Janet Project Manager, a build tool for Janet 
+.SH SYNOPSIS
+.B jpm
+[\fB\-\-flag ...\fR]
+[\fB\-\-option=value ...\fR]
+.IR command
+.IR args ...
+.SH DESCRIPTION
+jpm is the build tool that ships with a standard Janet install. It is
+used for building Janet projects, installing dependencies, installing
+projects, building native modules, and exporting your Janet project to a
+standalone executable. Although not required for working with Janet, it
+removes much of the boilerplate with installing dependencies and
+building native modules. jpm requires only Janet to run, and uses git
+to install dependencies (jpm will work without git installed).
+.SH DOCUMENTATION
+
+jpm has several subcommands, each used for managing either a single Janet project or
+all Janet modules installed on the system. Global commands, those that manage modules
+at the system level, do things like install and uninstall packages, as well as clear the cache.
+More interesting are the local commands. For more information on jpm usage, see https://janet-lang.org/docs/index.html
+
+.SH FLAGS
+
+.TP
+.BR \-\-verbose
+Print detailed messages of what jpm is doing, including compilation commands and other shell commands.
+
+.TP
+.BR \-\-test
+If passed to jpm install, runs tests before installing. Will run tests recursively on dependencies.
+
+.SH OPTIONS
+
+.TP
+.BR \-\-modpath=/some/path
+Set the path to install modules to. Defaults to $JANET_MODPATH, $JANET_PATH, or (dyn :syspath) in that order.
+
+.TP
+.BR \-\-headerpath=/some/path
+Set the path the jpm will include when building C source code. This lets
+you specify the location of janet.h and janetconf.h on your system. On a
+normal install, this option is not needed.
+
+.TP
+.BR \-\-binpath=/some/path
+Set the path that jpm will install scripts and standalone executables to. Executables
+defined via declare-execuatble or scripts declared via declare-binscript will be installed
+here when jpm install is run. Defaults to $JANET_BINPATH, or a reasonable default for the system.
+See JANET_BINPATH for more.
+
+.TP
+.BR \-\-libpath=/some/path
+Sets the path jpm will use to look for libjanet.a for building standalone executables. libjanet.so
+is \fBnot\fR used for building native modules or standalone executables, only
+for linking into applications that want to embed janet as a dynamic module.
+Linking statically might be a better idea, even in that case. Defaults to
+$JANET_LIBPATH, or a reasonable default. See JANET_LIBPATH for more.
+
+.TP
+.BR \-\-compiler=$CC
+Sets the compiler used for compiling native modules and standalone executables. Defaults
+to cc.
+
+.TP
+.BR \-\-linker
+Sets the linker used to create native modules and executables. Only used on windows, where
+it defaults to link.exe.
+
+.TP
+.BR \-\-pkglist=https://github.com/janet-lang/pkgs.git
+Sets the git repository for the package listing used to resolve shorthand package names.
+
+.TP
+.BR \-\-archiver=$AR
+Sets the command used for creating static libraries, use for linking into the standalone executable.
+Native modules are compiled twice, once a normal native module (shared object), and once as an
+archive. Defaults to ar.
+
+.SH COMMANDS
+.TP
+.BR help
+Shows the usage text and exits immediately.
+
+.TP
+.BR build
+Builds all artifacts specified in the project.janet file in the current directory. Artifacts will
+be created in the ./build/ directory.
+
+.TP
+.BR install\ [\fBrepo\fR]
+
+When run with no arguments, installs all installable artifacts in the current project to
+the current JANET_MODPATH for modules and JANET_BINPATH for executables and scripts. Can also
+take an optional git repository URL and will install all artifacts in that repository instead.
+When run with an argument, install does not need to be run from a jpm project directory.
+
+.TP
+.BR uninstall\ [\fBname\fR]
+Uninstall a project installed with install. uninstall expects the name of the project, not the
+repository url, path to installed file or executable name. The name of the project must be specified
+at the top of the project.janet file in the declare-project form. If no name is given, uninstalls
+the current project if installed.
+
+.TP
+.BR clean
+Remove all artifacts created by jpm. This just deletes the build folder.
+
+.TP
+.BR test
+Runs jpm tests. jpm will run all janet source files in the test directory as tests. A test
+is considered failing if it exits with a non-zero exit code.
+
+.TP
+.BR deps
+Install all dependencies that this project requires recursively. jpm does not
+resolve dependency issues, like conflicting versions of the same module are required, or
+different modules with the same name. Dependencies are installed with git, so deps requires
+git to be on the PATH.
+
+.TP
+.BR clear-cache
+jpm caches git repositories that are needed to install modules from a remote
+source in a global cache ($JANET_PATH/.cache). If these dependencies are out of
+date or too large, clear-cache will remove the cache and jpm will rebuild it
+when needed. clear-cache is a global command, so a project.janet is not
+required.
+
+.TP
+.BR run\ [\fBrule\fR]
+Run a given rule defined in project.janet. Project definitions files (project.janet) usually
+contain a few artifact declarations, which set up rules that jpm can then resolve, or execute.
+A project.janet can also create custom rules to create arbitrary files or run arbitrary code, much
+like make. run will run a single rule or build a single file.
+
+.TP
+.BR rules
+List all rules that can be run via run. This is useful for exploring rules in the project.
+
+.TP
+.BR update-pkgs
+Update the package listing by installing the 'pkgs' package. Same as jpm install pkgs
+
+.TP
+.BR quickbin [\fBentry\fR] [\fBexecutable\fR]
+Create a standalone, statically linked executable from a Janet source file that contains a main function.
+The main function is the entry point of the program and will receive command line arguments
+as function arguments. The entry file can import other modules, including native C modules, and
+jpm will attempt to include the dependencies into the generated executable.
+
+.SH ENVIRONMENT
+
+.B JANET_PATH
+.RS
+The location to look for Janet libraries. This is the only environment variable Janet needs to
+find native and source code modules. If no JANET_PATH is set, Janet will look in
+the default location set at compile time, which can be determined with (dyn :syspath)
+.RE
+
+.B JANET_MODPATH
+.RS
+The location that jpm will use to install libraries to. Defaults to JANET_PATH, but you could
+set this to a different directory if you want to. Doing so would let you import Janet modules
+on the normal system path (JANET_PATH or (dyn :syspath)), but install to a different directory. It is also a more reliable way to install
+This variable is overwritten by the --modpath=/some/path if it is provided.
+.RE
+
+.B JANET_HEADERPATH
+.RS
+The location that jpm will look for janet header files (janet.h and janetconf.h) that are used
+to build native modules and standalone executables. If janet.h and janetconf.h are available as
+default includes on your system, this value is not required. If not provided, will default to
+(dyn :syspath)/../../include/janet. The --headerpath=/some/path will override this variable.
+.RE
+
+.B JANET_LIBPATH
+.RS
+Similar to JANET_HEADERPATH, this path is where jpm will look for
+libjanet.a for creating standalong executables. This does not need to be
+set on a normal install. 
+If not provided, this will default to (dyn :syspath)/../../lib.
+The --libpath=/some/path will override this variable.
+.RE
+
+.B JANET_BINPATH
+.RS
+The directory where jpm will install binary scripts and executables to.
+Defaults to
+(dyn :syspath)/../../lib.
+The --binpath=/some/path will override this variable.
+.RE
+
+.B JANET_PKGLIST
+.RS
+The git repository URL that contains a listing of packages. This allows installing packages with shortnames, which
+is mostly a convenience. However, package dependencies can use short names, package listings
+can be used to choose a particular set of dependency versions for a whole project.
+
+.SH AUTHOR
+Written by Calvin Rose <calsrose@gmail.com>

meson.build

@@ -0,0 +1,261 @@
+# Copyright (c) 2019 Calvin Rose and contributors
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to
+# deal in the Software without restriction, including without limitation the
+# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+# sell copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+
+project('janet', 'c',
+  default_options : ['c_std=c99', 'b_lundef=false', 'default_library=both'],
+  version : '1.6.0-dev')
+
+# Global settings
+janet_path = join_paths(get_option('prefix'), get_option('libdir'), 'janet')
+header_path = join_paths(get_option('prefix'), get_option('includedir'), 'janet')
+
+# Link math library on all systems
+cc = meson.get_compiler('c')
+m_dep = cc.find_library('m', required : false)
+dl_dep = cc.find_library('dl', required : false)
+thread_dep = dependency('threads')
+
+# Link options
+if build_machine.system() != 'windows'
+    add_project_link_arguments('-rdynamic', language : 'c')
+endif
+
+# Generate custom janetconf.h
+conf = configuration_data()
+version_parts = meson.project_version().split('.')
+last_parts = version_parts[2].split('-')
+if last_parts.length() > 1
+  conf.set_quoted('JANET_VERSION_EXTRA', '-' + last_parts[1])
+else
+  conf.set_quoted('JANET_VERSION_EXTRA', '')
+endif
+conf.set('JANET_VERSION_MAJOR', version_parts[0].to_int())
+conf.set('JANET_VERSION_MINOR', version_parts[1].to_int())
+conf.set('JANET_VERSION_PATCH', last_parts[0].to_int())
+conf.set_quoted('JANET_VERSION', meson.project_version())
+# Use options
+conf.set_quoted('JANET_BUILD', get_option('git_hash'))
+conf.set('JANET_NO_NANBOX', not get_option('nanbox'))
+conf.set('JANET_SINGLE_THREADED', get_option('single_threaded'))
+conf.set('JANET_NO_DYNAMIC_MODULES', not get_option('dynamic_modules'))
+conf.set('JANET_NO_DOCSTRINGS', not get_option('docstrings'))
+conf.set('JANET_NO_SOURCEMAPS', not get_option('sourcemaps'))
+conf.set('JANET_NO_ASSEMBLER', not get_option('assembler'))
+conf.set('JANET_NO_PEG', not get_option('peg'))
+conf.set('JANET_REDUCED_OS', get_option('reduced_os'))
+conf.set('JANET_NO_TYPED_ARRAY', not get_option('typed_array'))
+conf.set('JANET_NO_INT_TYPES', not get_option('int_types'))
+conf.set('JANET_RECURSION_GUARD', get_option('recursion_guard'))
+conf.set('JANET_MAX_PROTO_DEPTH', get_option('max_proto_depth'))
+conf.set('JANET_MAX_MACRO_EXPAND', get_option('max_macro_expand'))
+conf.set('JANET_STACK_MAX', get_option('stack_max'))
+if get_option('os_name') != ''
+  conf.set('JANET_OS_NAME', get_option('os_name'))
+endif
+if get_option('arch_name') != ''
+  conf.set('JANET_ARCH_NAME', get_option('arch_name'))
+endif
+jconf = configure_file(output : 'janetconf.h',
+  configuration : conf)
+
+# Include directories
+incdir = include_directories(['src/include', '.'])
+
+# Building generated sources
+xxd = executable('xxd', 'tools/xxd.c', native : true)
+gen = generator(xxd,
+  output : '@BASENAME@.gen.c',
+  arguments : ['@INPUT@', '@OUTPUT@', '@EXTRA_ARGS@'])
+boot_gen = gen.process('src/boot/boot.janet', extra_args: 'janet_gen_boot')
+
+# Order is important here, as some headers
+# depend on other headers for the amalg target
+core_headers = [
+  'src/core/util.h',
+  'src/core/state.h',
+  'src/core/gc.h',
+  'src/core/vector.h',
+  'src/core/fiber.h',
+  'src/core/regalloc.h',
+  'src/core/compile.h',
+  'src/core/emit.h',
+  'src/core/symcache.h'
+]
+
+core_src = [
+  'src/core/abstract.c',
+  'src/core/array.c',
+  'src/core/asm.c',
+  'src/core/buffer.c',
+  'src/core/bytecode.c',
+  'src/core/capi.c',
+  'src/core/cfuns.c',
+  'src/core/compile.c',
+  'src/core/corelib.c',
+  'src/core/debug.c',
+  'src/core/emit.c',
+  'src/core/fiber.c',
+  'src/core/gc.c',
+  'src/core/inttypes.c',
+  'src/core/io.c',
+  'src/core/marsh.c',
+  'src/core/math.c',
+  'src/core/os.c',
+  'src/core/parse.c',
+  'src/core/peg.c',
+  'src/core/pp.c',
+  'src/core/regalloc.c',
+  'src/core/run.c',
+  'src/core/specials.c',
+  'src/core/string.c',
+  'src/core/strtod.c',
+  'src/core/struct.c',
+  'src/core/symcache.c',
+  'src/core/table.c',
+  'src/core/thread.c',
+  'src/core/tuple.c',
+  'src/core/typedarray.c',
+  'src/core/util.c',
+  'src/core/value.c',
+  'src/core/vector.c',
+  'src/core/vm.c',
+  'src/core/wrap.c'
+]
+
+boot_src = [
+  'src/boot/array_test.c',
+  'src/boot/boot.c',
+  'src/boot/buffer_test.c',
+  'src/boot/number_test.c',
+  'src/boot/system_test.c',
+  'src/boot/table_test.c',
+]
+
+mainclient_src = [
+  'src/mainclient/line.c',
+  'src/mainclient/main.c'
+]
+
+# Build boot binary
+janet_boot = executable('janet-boot', core_src, boot_src, boot_gen,
+  include_directories : incdir,
+  c_args : '-DJANET_BOOTSTRAP',
+  dependencies : [m_dep, dl_dep, thread_dep],
+  native : true)
+
+# Build core image
+core_image = custom_target('core_image',
+  input : [janet_boot],
+  output : 'core_image.gen.c',
+  command : [janet_boot, '@OUTPUT@', 'JANET_PATH', janet_path, 'JANET_HEADERPATH', header_path])
+
+libjanet = library('janet', core_src, core_image,
+  include_directories : incdir,
+  dependencies : [m_dep, dl_dep, thread_dep],
+  install : true)
+
+# Extra c flags - adding -fvisibility=hidden matches the Makefile and
+# shaves off about 10k on linux x64, likely similar on other platforms.
+native_cc = meson.get_compiler('c', native: true)
+cross_cc = meson.get_compiler('c', native: false)
+if native_cc.has_argument('-fvisibility=hidden')
+  extra_native_cflags = ['-fvisibility=hidden']
+else
+  extra_native_cflags = []
+endif
+if cross_cc.has_argument('-fvisibility=hidden')
+  extra_cross_cflags = ['-fvisibility=hidden']
+else
+  extra_cross_cflags = []
+endif
+
+janet_mainclient = executable('janet', core_src, core_image, mainclient_src,
+  include_directories : incdir,
+  dependencies : [m_dep, dl_dep, thread_dep],
+  c_args : extra_native_cflags,
+  install : true)
+
+if meson.is_cross_build()
+  janet_nativeclient = executable('janet-native', core_src, core_image, mainclient_src,
+    include_directories : incdir,
+    dependencies : [m_dep, dl_dep, thread_dep],
+    c_args : extra_cross_cflags,
+    native : true)
+else
+  janet_nativeclient = janet_mainclient
+endif
+
+# Documentation
+docs = custom_target('docs',
+  input : ['tools/gendoc.janet'],
+  output : ['doc.html'],
+  capture : true,
+  command : [janet_nativeclient, '@INPUT@'])
+
+# Amalgamated source
+amalg = custom_target('amalg',
+  input : ['tools/amalg.janet', core_headers, core_src, core_image],
+  output : ['janet.c'],
+  capture : true,
+  command : [janet_nativeclient, '@INPUT@'])
+amalg_shell = custom_target('amalg-shell',
+  input : ['tools/amalg.janet', 'src/mainclient/line.h',
+    'src/mainclient/line.c', 'src/mainclient/main.c'],
+  output : ['shell.c'],
+  capture : true,
+  command : [janet_nativeclient, '@INPUT@'])
+
+# Amalgamated client
+janet_amalgclient = executable('janet-amalg', amalg, amalg_shell,
+  include_directories : incdir,
+  dependencies : [m_dep, dl_dep, thread_dep],
+  build_by_default : false)
+
+# Tests
+test_files = [
+  'test/suite0.janet',
+  'test/suite1.janet',
+  'test/suite2.janet',
+  'test/suite3.janet',
+  'test/suite4.janet',
+  'test/suite5.janet',
+  'test/suite6.janet',
+  'test/suite7.janet'
+]
+foreach t : test_files
+  test(t, janet_nativeclient, args : files([t]), workdir : meson.current_source_dir())
+endforeach
+
+# Repl
+run_target('repl', command : [janet_nativeclient])
+
+# For use as meson subproject (wrap)
+janet_dep = declare_dependency(include_directories : incdir,
+  link_with : libjanet)
+
+# Installation
+install_man('janet.1')
+install_man('jpm.1')
+install_headers(['src/include/janet.h', jconf], subdir: 'janet')
+janet_binscripts = [
+  'auxbin/jpm'
+]
+install_data(sources : janet_binscripts, install_dir : 'bin')
+install_data(sources : ['tools/.keep'], install_dir : 'lib/janet')

meson_options.txt

@@ -0,0 +1,20 @@
+option('git_hash', type : 'string', value : 'meson')
+
+option('single_threaded', type : 'boolean', value : false)
+option('nanbox', type : 'boolean', value : true)
+option('dynamic_modules', type : 'boolean', value : true)
+option('docstrings', type : 'boolean', value : true)
+option('sourcemaps', type : 'boolean', value : true)
+option('reduced_os', type : 'boolean', value : false)
+option('assembler', type : 'boolean', value : true)
+option('peg', type : 'boolean', value : true)
+option('typed_array', type : 'boolean', value : true)
+option('int_types', type : 'boolean', value : true)
+
+option('recursion_guard', type : 'integer', min : 10, max : 8000, value : 1024)
+option('max_proto_depth', type : 'integer', min : 10, max : 8000, value : 200)
+option('max_macro_expand', type : 'integer', min : 1, max : 8000, value : 200)
+option('stack_max', type : 'integer', min : 8096, max : 0x7fffffff, value : 0x7fffffff)
+
+option('arch_name', type : 'string', value: '')
+option('os_name', type : 'string', value: '')

src/boot/array_test.c

@@ -20,16 +20,16 @@
 * IN THE SOFTWARE.
 */
 
-#include <janet/janet.h>
+#include <janet.h>
 #include <assert.h>
 
-int main() {
+#include "tests.h"
+
+int array_test() {
 
     int i;
     JanetArray *array1, *array2;
 
-    janet_init();
-
     array1 = janet_array(10);
     array2 = janet_array(0);
 
@@ -62,7 +62,5 @@ int main() {
 
     assert(array1->count == 5);
 
-    janet_deinit();
-
     return 0;
 }

src/boot/boot.c

@@ -0,0 +1,78 @@
+/*
+* Copyright (c) 2019 Calvin Rose
+*
+* Permission is hereby granted, free of charge, to any person obtaining a copy
+* of this software and associated documentation files (the "Software"), to
+* deal in the Software without restriction, including without limitation the
+* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+* sell copies of the Software, and to permit persons to whom the Software is
+* furnished to do so, subject to the following conditions:
+*
+* The above copyright notice and this permission notice shall be included in
+* all copies or substantial portions of the Software.
+*
+* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+* IN THE SOFTWARE.
+*/
+
+#include <janet.h>
+#include "tests.h"
+
+extern const unsigned char *janet_gen_boot;
+extern int32_t janet_gen_boot_size;
+
+int main(int argc, const char **argv) {
+
+    /* Init janet */
+    janet_init();
+
+    /* Run tests */
+    array_test();
+    buffer_test();
+    number_test();
+    system_test();
+    table_test();
+
+    /* C tests passed */
+
+    /* Set up VM */
+    int status;
+    JanetTable *env;
+
+    env = janet_core_env(NULL);
+
+    /* Create args tuple */
+    JanetArray *args = janet_array(argc);
+    for (int i = 0; i < argc; i++)
+        janet_array_push(args, janet_cstringv(argv[i]));
+    janet_def(env, "boot/args", janet_wrap_array(args), "Command line arguments.");
+
+    /* Add in options from janetconf.h so boot.janet can configure the image as needed. */
+    JanetTable *opts = janet_table(0);
+#ifdef JANET_NO_DOCSTRINGS
+    janet_table_put(opts, janet_ckeywordv("no-docstrings"), janet_wrap_true());
+#endif
+#ifdef JANET_NO_SOURCEMAPS
+    janet_table_put(opts, janet_ckeywordv("no-sourcemaps"), janet_wrap_true());
+#endif
+    janet_def(env, "boot/config", janet_wrap_table(opts), "Boot options");
+
+    /* Run bootstrap script to generate core image */
+    const char *boot_file;
+#ifdef JANET_NO_SOURCEMAPS
+    boot_file = NULL;
+#else
+    boot_file = "boot.janet";
+#endif
+    status = janet_dobytes(env, janet_gen_boot, janet_gen_boot_size, boot_file, NULL);
+
+    /* Deinitialize vm */
+    janet_deinit();
+
+    return status;
+}

src/boot/buffer_test.c

@@ -20,16 +20,16 @@
 * IN THE SOFTWARE.
 */
 
-#include <janet/janet.h>
+#include <janet.h>
 #include <assert.h>
 
-int main() {
+#include "tests.h"
+
+int buffer_test() {
 
     int i;
     JanetBuffer *buffer1, *buffer2;
 
-    janet_init();
-
     buffer1 = janet_buffer(100);
     buffer2 = janet_buffer(0);
 
@@ -58,7 +58,5 @@ int main() {
         assert(buffer1->data[i] == buffer2->data[i]);
     }
 
-    janet_deinit();
-
     return 0;
 }

src/boot/number_test.c

@@ -20,11 +20,13 @@
 * IN THE SOFTWARE.
 */
 
-#include <janet/janet.h>
+#include <janet.h>
 #include <stdio.h>
 #include <string.h>
 #include <assert.h>
 
+#include "tests.h"
+
 /* Check a subset of numbers against system implementation.
  * Note that this depends on the system implementation being correct,
  * which may not be the case for old or non compliant systems. Also,
@@ -36,14 +38,12 @@ static void test_valid_str(const char *str) {
     double cnum, jnum;
     jnum = 0.0;
     cnum = atof(str);
-    err = janet_scan_number((const uint8_t *) str, strlen(str), &jnum);
+    err = janet_scan_number((const uint8_t *) str, (int32_t) strlen(str), &jnum);
     assert(!err);
     assert(cnum == jnum);
 }
 
-int main() {
-
-    janet_init();
+int number_test() {
 
     test_valid_str("1.0");
     test_valid_str("1");
@@ -63,7 +63,5 @@ int main() {
     test_valid_str("0000000011111111111111111111111111");
     test_valid_str(".112312333333323123123123123123123");
 
-    janet_deinit();
-
     return 0;
 }

src/boot/system_test.c

@@ -20,11 +20,13 @@
 * IN THE SOFTWARE.
 */
 
-#include <janet/janet.h>
+#include <janet.h>
 #include <assert.h>
 #include <stdio.h>
 
-int main() {
+#include "tests.h"
+
+int system_test() {
 
 #ifdef JANET_32
     assert(sizeof(void *) == 4);
@@ -32,8 +34,6 @@ int main() {
     assert(sizeof(void *) == 8);
 #endif
 
-    janet_init();
-
     /* Reflexive testing and nanbox testing */
     assert(janet_equals(janet_wrap_nil(), janet_wrap_nil()));
     assert(janet_equals(janet_wrap_false(), janet_wrap_false()));
@@ -45,10 +45,10 @@ int main() {
     assert(janet_equals(janet_wrap_number(1.4), janet_wrap_number(1.4)));
     assert(janet_equals(janet_wrap_number(3.14159265), janet_wrap_number(3.14159265)));
 
+    assert(NULL != &janet_wrap_nil);
+
     assert(janet_equals(janet_cstringv("a string."), janet_cstringv("a string.")));
     assert(janet_equals(janet_csymbolv("sym"), janet_csymbolv("sym")));
 
-    janet_deinit();
-
     return 0;
 }

src/boot/table_test.c

@@ -20,15 +20,15 @@
 * IN THE SOFTWARE.
 */
 
-#include <janet/janet.h>
+#include <janet.h>
 #include <assert.h>
 
-int main() {
+#include "tests.h"
+
+int table_test() {
 
     JanetTable *t1, *t2;
 
-    janet_init();
-
     t1 = janet_table(10);
     t2 = janet_table(0);
 
@@ -61,7 +61,5 @@ int main() {
     assert(janet_equals(janet_table_get(t2, janet_csymbolv("t2key1")), janet_wrap_integer(10)));
     assert(janet_equals(janet_table_get(t2, janet_csymbolv("t2key2")), janet_wrap_integer(100)));
 
-    janet_deinit();
-
     return 0;
 }

src/boot/tests.h

@@ -0,0 +1,11 @@
+#ifndef TESTS_H_DNMBUYYL
+#define TESTS_H_DNMBUYYL
+
+/* Tests */
+extern int array_test();
+extern int buffer_test();
+extern int number_test();
+extern int system_test();
+extern int table_test();
+
+#endif /* end of include guard: TESTS_H_DNMBUYYL */

src/conf/janetconf.h

@@ -0,0 +1,62 @@
+/*
+* Copyright (c) 2019 Calvin Rose
+*
+* Permission is hereby granted, free of charge, to any person obtaining a copy
+* of this software and associated documentation files (the "Software"), to
+* deal in the Software without restriction, including without limitation the
+* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+* sell copies of the Software, and to permit persons to whom the Software is
+* furnished to do so, subject to the following conditions:
+*
+* The above copyright notice and this permission notice shall be included in
+* all copies or substantial portions of the Software.
+*
+* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+* IN THE SOFTWARE.
+*/
+
+/* This is an example janetconf.h file. This will be usually generated
+ * by the build system. */
+
+#ifndef JANETCONF_H
+#define JANETCONF_H
+
+#define JANET_VERSION_MAJOR 1
+#define JANET_VERSION_MINOR 6
+#define JANET_VERSION_PATCH 0
+#define JANET_VERSION_EXTRA "-dev"
+#define JANET_VERSION "1.6.0-dev"
+
+/* #define JANET_BUILD "local" */
+
+/* These settings all affect linking, so use cautiously. */
+/* #define JANET_SINGLE_THREADED */
+/* #define JANET_NO_DYNAMIC_MODULES */
+/* #define JANET_NO_NANBOX */
+/* #define JANET_API __attribute__((visibility ("default"))) */
+
+/* These settings should be specified before amalgamation is
+ * built. */
+/* #define JANET_NO_DOCSTRINGS */
+/* #define JANET_NO_SOURCEMAPS */
+/* #define JANET_REDUCED_OS */
+
+/* Other settings */
+/* #define JANET_NO_ASSEMBLER */
+/* #define JANET_NO_PEG */
+/* #define JANET_NO_TYPED_ARRAY */
+/* #define JANET_NO_INT_TYPES */
+/* #define JANET_OUT_OF_MEMORY do { printf("janet out of memory\n"); exit(1); } while (0) */
+/* #define JANET_RECURSION_GUARD 1024 */
+/* #define JANET_MAX_PROTO_DEPTH 200 */
+/* #define JANET_MAX_MACRO_EXPAND 200 */
+/* #define JANET_STACK_MAX 16384 */
+/* #define JANET_OS_NAME my-custom-os */
+/* #define JANET_ARCH_NAME pdp-8 */
+
+#endif /* end of include guard: JANETCONF_H */

src/core/abstract.c

@@ -21,16 +21,24 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "gc.h"
 #endif
 
 /* Create new userdata */
-void *janet_abstract(const JanetAbstractType *atype, size_t size) {
-    char *data = janet_gcalloc(JANET_MEMORY_ABSTRACT, sizeof(JanetAbstractHeader) + size);
-    JanetAbstractHeader *header = (JanetAbstractHeader *)data;
-    void *a = data + sizeof(JanetAbstractHeader);
+void *janet_abstract_begin(const JanetAbstractType *atype, size_t size) {
+    JanetAbstractHead *header = janet_gcalloc(JANET_MEMORY_NONE,
+                                sizeof(JanetAbstractHead) + size);
     header->size = size;
     header->type = atype;
-    return a;
+    return (void *) & (header->data);
+}
+
+void *janet_abstract_end(void *x) {
+    janet_gc_settype((void *)(janet_abstract_head(x)), JANET_MEMORY_ABSTRACT);
+    return x;
+}
+
+void *janet_abstract(const JanetAbstractType *atype, size_t size) {
+    return janet_abstract_end(janet_abstract_begin(atype, size));
 }

src/core/array.c

@@ -21,17 +21,20 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "gc.h"
 #include "util.h"
+#include "state.h"
 #endif
 
 #include <string.h>
 
-/* Initializes an array */
-JanetArray *janet_array_init(JanetArray *array, int32_t capacity) {
+/* Creates a new array */
+JanetArray *janet_array(int32_t capacity) {
+    JanetArray *array = janet_gcalloc(JANET_MEMORY_ARRAY, sizeof(JanetArray));
     Janet *data = NULL;
     if (capacity > 0) {
+        janet_vm_next_collection += capacity * sizeof(Janet);
         data = (Janet *) malloc(sizeof(Janet) * capacity);
         if (NULL == data) {
             JANET_OUT_OF_MEMORY;
@@ -43,16 +46,6 @@ JanetArray *janet_array_init(JanetArray *array, int32_t capacity) {
     return array;
 }
 
-void janet_array_deinit(JanetArray *array) {
-    free(array->data);
-}
-
-/* Creates a new array */
-JanetArray *janet_array(int32_t capacity) {
-    JanetArray *array = janet_gcalloc(JANET_MEMORY_ARRAY, sizeof(JanetArray));
-    return janet_array_init(array, capacity);
-}
-
 /* Creates a new array from n elements. */
 JanetArray *janet_array_n(const Janet *elements, int32_t n) {
     JanetArray *array = janet_gcalloc(JANET_MEMORY_ARRAY, sizeof(JanetArray));
@@ -71,11 +64,14 @@ void janet_array_ensure(JanetArray *array, int32_t capacity, int32_t growth) {
     Janet *newData;
     Janet *old = array->data;
     if (capacity <= array->capacity) return;
-    capacity *= growth;
+    int64_t new_capacity = ((int64_t) capacity) * growth;
+    if (new_capacity > INT32_MAX) new_capacity = INT32_MAX;
+    capacity = (int32_t) new_capacity;
     newData = realloc(old, capacity * sizeof(Janet));
     if (NULL == newData) {
         JANET_OUT_OF_MEMORY;
     }
+    janet_vm_next_collection += (capacity - array->capacity) * sizeof(Janet);
     array->data = newData;
     array->capacity = capacity;
 }
@@ -162,10 +158,11 @@ static Janet cfun_array_ensure(int32_t argc, Janet *argv) {
 }
 
 static Janet cfun_array_slice(int32_t argc, Janet *argv) {
-    JanetRange range = janet_getslice(argc, argv);
     JanetView view = janet_getindexed(argv, 0);
+    JanetRange range = janet_getslice(argc, argv);
     JanetArray *array = janet_array(range.end - range.start);
-    memcpy(array->data, view.items + range.start, sizeof(Janet) * (range.end - range.start));
+    if (array->data)
+        memcpy(array->data, view.items + range.start, sizeof(Janet) * (range.end - range.start));
     array->count = range.end - range.start;
     return janet_wrap_array(array);
 }
@@ -180,15 +177,14 @@ static Janet cfun_array_concat(int32_t argc, Janet *argv) {
                 janet_array_push(array, argv[i]);
                 break;
             case JANET_ARRAY:
-            case JANET_TUPLE:
-                {
-                    int32_t j, len;
-                    const Janet *vals;
-                    janet_indexed_view(argv[i], &vals, &len);
-                    for (j = 0; j < len; j++)
-                        janet_array_push(array, vals[j]);
-                }
-                break;
+            case JANET_TUPLE: {
+                int32_t j, len = 0;
+                const Janet *vals = NULL;
+                janet_indexed_view(argv[i], &vals, &len);
+                for (j = 0; j < len; j++)
+                    janet_array_push(array, vals[j]);
+            }
+            break;
         }
     }
     return janet_wrap_array(array);
@@ -212,60 +208,102 @@ static Janet cfun_array_insert(int32_t argc, Janet *argv) {
             restsize);
     memcpy(array->data + at, argv + 2, chunksize);
     array->count += (argc - 2);
-    return janet_wrap_array(array);
+    return argv[0];
+}
+
+static Janet cfun_array_remove(int32_t argc, Janet *argv) {
+    janet_arity(argc, 2, 3);
+    JanetArray *array = janet_getarray(argv, 0);
+    int32_t at = janet_getinteger(argv, 1);
+    int32_t n = 1;
+    if (at < 0) {
+        at = array->count + at + 1;
+    }
+    if (at < 0 || at > array->count)
+        janet_panicf("removal index %d out of range [0,%d]", at, array->count);
+    if (argc == 3) {
+        n = janet_getinteger(argv, 2);
+        if (n < 0)
+            janet_panicf("expected non-negative integer for argument n, got %v", argv[2]);
+    }
+    if (at + n > array->count) {
+        n = array->count - at;
+    }
+    memmove(array->data + at,
+            array->data + at + n,
+            (array->count - at - n) * sizeof(Janet));
+    array->count -= n;
+    return argv[0];
 }
 
 static const JanetReg array_cfuns[] = {
-    {"array/new", cfun_array_new,
+    {
+        "array/new", cfun_array_new,
         JDOC("(array/new capacity)\n\n"
-                "Creates a new empty array with a pre-allocated capacity. The same as "
-                "(array) but can be more efficient if the maximum size of an array is known.")
+             "Creates a new empty array with a pre-allocated capacity. The same as "
+             "(array) but can be more efficient if the maximum size of an array is known.")
     },
-    {"array/pop", cfun_array_pop,
+    {
+        "array/pop", cfun_array_pop,
         JDOC("(array/pop arr)\n\n"
-                "Remove the last element of the array and return it. If the array is empty, will return nil. Modifies "
-                "the input array.")
+             "Remove the last element of the array and return it. If the array is empty, will return nil. Modifies "
+             "the input array.")
     },
-    {"array/peek", cfun_array_peek,
+    {
+        "array/peek", cfun_array_peek,
         JDOC("(array/peek arr)\n\n"
-                "Returns the last element of the array. Does not modify the array.")
+             "Returns the last element of the array. Does not modify the array.")
     },
-    {"array/push", cfun_array_push,
+    {
+        "array/push", cfun_array_push,
         JDOC("(array/push arr x)\n\n"
-                "Insert an element in the end of an array. Modifies the input array and returns it.")
+             "Insert an element in the end of an array. Modifies the input array and returns it.")
     },
-    {"array/ensure", cfun_array_ensure,
-        JDOC("(array/ensure arr capacity)\n\n"
-                "Ensures that the memory backing the array has enough memory for capacity "
-                "items. Capacity must be an integer. If the backing capacity is already enough, "
-                "then this function does nothing. Otherwise, the backing memory will be reallocated "
-                "so that there is enough space.")
+    {
+        "array/ensure", cfun_array_ensure,
+        JDOC("(array/ensure arr capacity growth)\n\n"
+             "Ensures that the memory backing the array is large enough for capacity "
+             "items at the given rate of growth. Capacity and growth must be integers. "
+             "If the backing capacity is already enough, then this function does nothing. "
+             "Otherwise, the backing memory will be reallocated so that there is enough space.")
     },
-    {"array/slice", cfun_array_slice,
-        JDOC("(array/slice arrtup [, start=0 [, end=(length arrtup)]])\n\n"
-                "Takes a slice of array or tuple from start to end. The range is half open, "
-                "[start, end). Indexes can also be negative, indicating indexing from the end of the "
-                "end of the array. By default, start is 0 and end is the length of the array. "
-                "Returns a new array.")
+    {
+        "array/slice", cfun_array_slice,
+        JDOC("(array/slice arrtup &opt start end)\n\n"
+             "Takes a slice of array or tuple from start to end. The range is half open, "
+             "[start, end). Indexes can also be negative, indicating indexing from the end of the "
+             "end of the array. By default, start is 0 and end is the length of the array. "
+             "Note that index -1 is synonymous with index (length arrtup) to allow a full "
+             "negative slice range. Returns a new array.")
     },
-    {"array/concat", cfun_array_concat,
+    {
+        "array/concat", cfun_array_concat,
         JDOC("(array/concat arr & parts)\n\n"
-                "Concatenates a variadic number of arrays (and tuples) into the first argument "
-                "which must an array. If any of the parts are arrays or tuples, their elements will "
-                "be inserted into the array. Otherwise, each part in parts will be appended to arr in order. "
-                "Return the modified array arr.")
+             "Concatenates a variadic number of arrays (and tuples) into the first argument "
+             "which must an array. If any of the parts are arrays or tuples, their elements will "
+             "be inserted into the array. Otherwise, each part in parts will be appended to arr in order. "
+             "Return the modified array arr.")
     },
-    {"array/insert", cfun_array_insert,
+    {
+        "array/insert", cfun_array_insert,
         JDOC("(array/insert arr at & xs)\n\n"
-                "Insert all of xs into array arr at index at. at should be an integer "
-                "0 and the length of the array. A negative value for at will index from "
-                "the end of the array, such that inserting at -1 appends to the array. "
-                "Returns the array.")
+             "Insert all of xs into array arr at index at. at should be an integer "
+             "0 and the length of the array. A negative value for at will index from "
+             "the end of the array, such that inserting at -1 appends to the array. "
+             "Returns the array.")
+    },
+    {
+        "array/remove", cfun_array_remove,
+        JDOC("(array/remove arr at &opt n)\n\n"
+             "Remove up to n elements starting at index at in array arr. at can index from "
+             "the end of the array with a negative index, and n must be a non-negative integer. "
+             "By default, n is 1. "
+             "Returns the array.")
     },
     {NULL, NULL, NULL}
 };
 
 /* Load the array module */
 void janet_lib_array(JanetTable *env) {
-    janet_cfuns(env, NULL, array_cfuns);
+    janet_core_cfuns(env, NULL, array_cfuns);
 }

src/core/asm.c

@@ -21,7 +21,7 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "util.h"
 #endif
 
@@ -85,6 +85,7 @@ static const JanetInstructionDef janet_ops[] = {
     {"gten", JOP_NUMERIC_GREATER_THAN_EQUAL},
     {"gtim", JOP_GREATER_THAN_IMMEDIATE},
     {"gtn", JOP_NUMERIC_GREATER_THAN},
+    {"in", JOP_IN},
     {"jmp", JOP_JUMP},
     {"jmpif", JOP_JUMP_IF},
     {"jmpno", JOP_JUMP_IF_NOT},
@@ -101,6 +102,7 @@ static const JanetInstructionDef janet_ops[] = {
     {"ltim", JOP_LESS_THAN_IMMEDIATE},
     {"ltn", JOP_NUMERIC_LESS_THAN},
     {"mkarr", JOP_MAKE_ARRAY},
+    {"mkbtp", JOP_MAKE_BRACKET_TUPLE},
     {"mkbuf", JOP_MAKE_BUFFER},
     {"mkstr", JOP_MAKE_STRING},
     {"mkstu", JOP_MAKE_STRUCT},
@@ -111,6 +113,7 @@ static const JanetInstructionDef janet_ops[] = {
     {"mul", JOP_MULTIPLY},
     {"mulim", JOP_MULTIPLY_IMMEDIATE},
     {"noop", JOP_NOOP},
+    {"prop", JOP_PROPAGATE},
     {"push", JOP_PUSH},
     {"push2", JOP_PUSH_2},
     {"push3", JOP_PUSH_3},
@@ -147,19 +150,18 @@ static const TypeAlias type_aliases[] = {
     {"callable", JANET_TFLAG_CALLABLE},
     {"cfunction", JANET_TFLAG_CFUNCTION},
     {"dictionary", JANET_TFLAG_DICTIONARY},
-    {"false", JANET_TFLAG_FALSE},
     {"fiber", JANET_TFLAG_FIBER},
     {"function", JANET_TFLAG_FUNCTION},
     {"indexed", JANET_TFLAG_INDEXED},
+    {"keyword", JANET_TFLAG_KEYWORD},
     {"nil", JANET_TFLAG_NIL},
     {"number", JANET_TFLAG_NUMBER},
+    {"pointer", JANET_TFLAG_POINTER},
     {"string", JANET_TFLAG_STRING},
     {"struct", JANET_TFLAG_STRUCT},
     {"symbol", JANET_TFLAG_SYMBOL},
-    {"keyword", JANET_TFLAG_KEYWORD},
-    {"table", JANET_TFLAG_BOOLEAN},
-    {"true", JANET_TFLAG_TRUE},
-    {"tuple", JANET_TFLAG_BOOLEAN}
+    {"table", JANET_TFLAG_TABLE},
+    {"tuple", JANET_TFLAG_TUPLE}
 };
 
 /* Deinitialize an Assembler. Does not deinitialize the parents. */
@@ -224,9 +226,9 @@ static int32_t janet_asm_addenv(JanetAssembler *a, Janet envname) {
 /* Parse an argument to an assembly instruction, and return the result as an
  * integer. This integer will need to be bounds checked. */
 static int32_t doarg_1(
-        JanetAssembler *a,
-        enum JanetOpArgType argtype,
-        Janet x) {
+    JanetAssembler *a,
+    enum JanetOpArgType argtype,
+    Janet x) {
     int32_t ret = -1;
     JanetTable *c;
     switch (argtype) {
@@ -253,8 +255,7 @@ static int32_t doarg_1(
         default:
             goto error;
             break;
-        case JANET_NUMBER:
-        {
+        case JANET_NUMBER: {
             double y = janet_unwrap_number(x);
             if (janet_checkintrange(y)) {
                 ret = (int32_t) y;
@@ -263,8 +264,7 @@ static int32_t doarg_1(
             }
             break;
         }
-        case JANET_TUPLE:
-        {
+        case JANET_TUPLE: {
             const Janet *t = janet_unwrap_tuple(x);
             if (argtype == JANET_OAT_TYPE) {
                 int32_t i = 0;
@@ -277,8 +277,7 @@ static int32_t doarg_1(
             }
             break;
         }
-        case JANET_KEYWORD:
-        {
+        case JANET_KEYWORD: {
             if (NULL != c && argtype == JANET_OAT_LABEL) {
                 Janet result = janet_table_get(c, x);
                 if (janet_checktype(result, JANET_NUMBER)) {
@@ -288,10 +287,10 @@ static int32_t doarg_1(
                 }
             } else if (argtype == JANET_OAT_TYPE || argtype == JANET_OAT_SIMPLETYPE) {
                 const TypeAlias *alias = janet_strbinsearch(
-                            &type_aliases,
-                            sizeof(type_aliases)/sizeof(TypeAlias),
-                            sizeof(TypeAlias),
-                            janet_unwrap_keyword(x));
+                                             &type_aliases,
+                                             sizeof(type_aliases) / sizeof(TypeAlias),
+                                             sizeof(TypeAlias),
+                                             janet_unwrap_keyword(x));
                 if (alias) {
                     ret = alias->mask;
                 } else {
@@ -302,8 +301,7 @@ static int32_t doarg_1(
             }
             break;
         }
-        case JANET_SYMBOL:
-        {
+        case JANET_SYMBOL: {
             if (NULL != c) {
                 Janet result = janet_table_get(c, x);
                 if (janet_checktype(result, JANET_NUMBER)) {
@@ -328,7 +326,7 @@ static int32_t doarg_1(
         a->def->slotcount = (int32_t) ret + 1;
     return ret;
 
-    error:
+error:
     janet_asm_errorv(a, janet_formatc("error parsing instruction argument %v", x));
     return 0;
 }
@@ -336,12 +334,12 @@ static int32_t doarg_1(
 /* Parse a single argument to an instruction. Trims it as well as
  * try to convert arguments to bit patterns */
 static uint32_t doarg(
-        JanetAssembler *a,
-        enum JanetOpArgType argtype,
-        int nth,
-        int nbytes,
-        int hassign,
-        Janet x) {
+    JanetAssembler *a,
+    enum JanetOpArgType argtype,
+    int nth,
+    int nbytes,
+    int hassign,
+    Janet x) {
     int32_t arg = doarg_1(a, argtype, x);
     /* Calculate the min and max values that can be stored given
      * nbytes, and whether or not the storage is signed */
@@ -349,59 +347,53 @@ static uint32_t doarg(
     int32_t min = hassign ? -max - 1 : 0;
     if (arg < min)
         janet_asm_errorv(a, janet_formatc("instruction argument %v is too small, must be %d byte%s",
-                    x, nbytes, nbytes > 1 ? "s" : ""));
+                                          x, nbytes, nbytes > 1 ? "s" : ""));
     if (arg > max)
         janet_asm_errorv(a, janet_formatc("instruction argument %v is too large, must be %d byte%s",
-                    x, nbytes, nbytes > 1 ? "s" : ""));
+                                          x, nbytes, nbytes > 1 ? "s" : ""));
     return ((uint32_t) arg) << (nth << 3);
 }
 
 /* Provide parsing methods for the different kinds of arguments */
 static uint32_t read_instruction(
-        JanetAssembler *a,
-        const JanetInstructionDef *idef,
-        const Janet *argt) {
+    JanetAssembler *a,
+    const JanetInstructionDef *idef,
+    const Janet *argt) {
     uint32_t instr = idef->opcode;
     enum JanetInstructionType type = janet_instructions[idef->opcode];
     switch (type) {
-        case JINT_0:
-        {
+        case JINT_0: {
             if (janet_tuple_length(argt) != 1)
                 janet_asm_error(a, "expected 0 arguments: (op)");
             break;
         }
-        case JINT_S:
-        {
+        case JINT_S: {
             if (janet_tuple_length(argt) != 2)
                 janet_asm_error(a, "expected 1 argument: (op, slot)");
             instr |= doarg(a, JANET_OAT_SLOT, 1, 2, 0, argt[1]);
             break;
         }
-        case JINT_L:
-        {
+        case JINT_L: {
             if (janet_tuple_length(argt) != 2)
                 janet_asm_error(a, "expected 1 argument: (op, label)");
             instr |= doarg(a, JANET_OAT_LABEL, 1, 3, 1, argt[1]);
             break;
         }
-        case JINT_SS:
-        {
+        case JINT_SS: {
             if (janet_tuple_length(argt) != 3)
                 janet_asm_error(a, "expected 2 arguments: (op, slot, slot)");
             instr |= doarg(a, JANET_OAT_SLOT, 1, 1, 0, argt[1]);
             instr |= doarg(a, JANET_OAT_SLOT, 2, 2, 0, argt[2]);
             break;
         }
-        case JINT_SL:
-        {
+        case JINT_SL: {
             if (janet_tuple_length(argt) != 3)
                 janet_asm_error(a, "expected 2 arguments: (op, slot, label)");
             instr |= doarg(a, JANET_OAT_SLOT, 1, 1, 0, argt[1]);
             instr |= doarg(a, JANET_OAT_LABEL, 2, 2, 1, argt[2]);
             break;
         }
-        case JINT_ST:
-        {
+        case JINT_ST: {
             if (janet_tuple_length(argt) != 3)
                 janet_asm_error(a, "expected 2 arguments: (op, slot, type)");
             instr |= doarg(a, JANET_OAT_SLOT, 1, 1, 0, argt[1]);
@@ -409,24 +401,21 @@ static uint32_t read_instruction(
             break;
         }
         case JINT_SI:
-        case JINT_SU:
-        {
+        case JINT_SU: {
             if (janet_tuple_length(argt) != 3)
                 janet_asm_error(a, "expected 2 arguments: (op, slot, integer)");
             instr |= doarg(a, JANET_OAT_SLOT, 1, 1, 0, argt[1]);
             instr |= doarg(a, JANET_OAT_INTEGER, 2, 2, type == JINT_SI, argt[2]);
             break;
         }
-        case JINT_SD:
-        {
+        case JINT_SD: {
             if (janet_tuple_length(argt) != 3)
                 janet_asm_error(a, "expected 2 arguments: (op, slot, funcdef)");
             instr |= doarg(a, JANET_OAT_SLOT, 1, 1, 0, argt[1]);
             instr |= doarg(a, JANET_OAT_FUNCDEF, 2, 2, 0, argt[2]);
             break;
         }
-        case JINT_SSS:
-        {
+        case JINT_SSS: {
             if (janet_tuple_length(argt) != 4)
                 janet_asm_error(a, "expected 3 arguments: (op, slot, slot, slot)");
             instr |= doarg(a, JANET_OAT_SLOT, 1, 1, 0, argt[1]);
@@ -435,8 +424,7 @@ static uint32_t read_instruction(
             break;
         }
         case JINT_SSI:
-        case JINT_SSU:
-        {
+        case JINT_SSU: {
             if (janet_tuple_length(argt) != 4)
                 janet_asm_error(a, "expected 3 arguments: (op, slot, slot, integer)");
             instr |= doarg(a, JANET_OAT_SLOT, 1, 1, 0, argt[1]);
@@ -444,8 +432,7 @@ static uint32_t read_instruction(
             instr |= doarg(a, JANET_OAT_INTEGER, 3, 1, type == JINT_SSI, argt[3]);
             break;
         }
-        case JINT_SES:
-        {
+        case JINT_SES: {
             JanetAssembler *b = a;
             uint32_t env;
             if (janet_tuple_length(argt) != 4)
@@ -461,8 +448,7 @@ static uint32_t read_instruction(
             instr |= doarg(b, JANET_OAT_SLOT, 3, 1, 0, argt[3]);
             break;
         }
-        case JINT_SC:
-        {
+        case JINT_SC: {
             if (janet_tuple_length(argt) != 3)
                 janet_asm_error(a, "expected 2 arguments: (op, slot, constant)");
             instr |= doarg(a, JANET_OAT_SLOT, 1, 1, 0, argt[1]);
@@ -528,9 +514,9 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
     }
 
     janet_asm_assert(&a,
-            janet_checktype(s, JANET_STRUCT) ||
-            janet_checktype(s, JANET_TABLE),
-            "expected struct or table for assembly source");
+                     janet_checktype(s, JANET_STRUCT) ||
+                     janet_checktype(s, JANET_TABLE),
+                     "expected struct or table for assembly source");
 
     /* Check for function name */
     a.name = janet_get1(s, janet_csymbolv("name"));
@@ -541,15 +527,20 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
     /* Set function arity */
     x = janet_get1(s, janet_csymbolv("arity"));
     def->arity = janet_checkint(x) ? janet_unwrap_integer(x) : 0;
+    janet_asm_assert(&a, def->arity >= 0, "arity must be non-negative");
+
+    x = janet_get1(s, janet_csymbolv("max-arity"));
+    def->max_arity = janet_checkint(x) ? janet_unwrap_integer(x) : def->arity;
+    janet_asm_assert(&a, def->max_arity >= def->arity, "max-arity must be greater than or equal to arity");
+
+    x = janet_get1(s, janet_csymbolv("min-arity"));
+    def->min_arity = janet_checkint(x) ? janet_unwrap_integer(x) : def->arity;
+    janet_asm_assert(&a, def->min_arity <= def->arity, "min-arity must be less than or equal to arity");
 
     /* Check vararg */
     x = janet_get1(s, janet_csymbolv("vararg"));
     if (janet_truthy(x)) def->flags |= JANET_FUNCDEF_FLAG_VARARG;
 
-    /* Check strict arity */
-    x = janet_get1(s, janet_csymbolv("fix-arity"));
-    if (janet_truthy(x)) def->flags |= JANET_FUNCDEF_FLAG_FIXARITY;
-
     /* Check source */
     x = janet_get1(s, janet_csymbolv("source"));
     if (janet_checktype(x, JANET_STRING)) def->source = janet_unwrap_string(x);
@@ -586,16 +577,16 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
         for (i = 0; i < count; i++) {
             Janet ct = arr[i];
             if (janet_checktype(ct, JANET_TUPLE) &&
-                janet_tuple_length(janet_unwrap_tuple(ct)) > 1 &&
-                janet_checktype(janet_unwrap_tuple(ct)[0], JANET_SYMBOL)) {
+                    janet_tuple_length(janet_unwrap_tuple(ct)) > 1 &&
+                    janet_checktype(janet_unwrap_tuple(ct)[0], JANET_SYMBOL)) {
                 const Janet *t = janet_unwrap_tuple(ct);
                 int32_t tcount = janet_tuple_length(t);
                 const uint8_t *macro = janet_unwrap_symbol(t[0]);
                 if (0 == janet_cstrcmp(macro, "quote")) {
                     def->constants[i] = t[1];
                 } else if (tcount == 3 &&
-                        janet_checktype(t[1], JANET_SYMBOL) &&
-                        0 == janet_cstrcmp(macro, "def")) {
+                           janet_checktype(t[1], JANET_SYMBOL) &&
+                           0 == janet_cstrcmp(macro, "def")) {
                     def->constants[i] = t[2];
                     janet_table_put(&a.constants, t[1], janet_wrap_integer(i));
                 } else {
@@ -658,7 +649,7 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
         }
         /* Allocate bytecode array */
         def->bytecode_length = blength;
-        def->bytecode = malloc(sizeof(int32_t) * blength);
+        def->bytecode = malloc(sizeof(uint32_t) * blength);
         if (NULL == def->bytecode) {
             JANET_OUT_OF_MEMORY;
         }
@@ -678,12 +669,12 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
                     op = 0;
                 } else {
                     janet_asm_assert(&a, janet_checktype(t[0], JANET_SYMBOL),
-                            "expected symbol in assembly instruction");
+                                     "expected symbol in assembly instruction");
                     idef = janet_strbinsearch(
-                            &janet_ops,
-                            sizeof(janet_ops)/sizeof(JanetInstructionDef),
-                            sizeof(JanetInstructionDef),
-                            janet_unwrap_symbol(t[0]));
+                               &janet_ops,
+                               sizeof(janet_ops) / sizeof(JanetInstructionDef),
+                               sizeof(JanetInstructionDef),
+                               janet_unwrap_symbol(t[0]));
                     if (NULL == idef)
                         janet_asm_errorv(&a, janet_formatc("unknown instruction %v", t[0]));
                     op = read_instruction(&a, idef, t);
@@ -715,8 +706,8 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
             if (!janet_checkint(tup[1])) {
                 janet_asm_error(&a, "expected integer");
             }
-            mapping.start = janet_unwrap_integer(tup[0]);
-            mapping.end = janet_unwrap_integer(tup[1]);
+            mapping.line = janet_unwrap_integer(tup[0]);
+            mapping.column = janet_unwrap_integer(tup[1]);
             def->sourcemap[i] = mapping;
         }
     }
@@ -750,7 +741,7 @@ JanetAssembleResult janet_asm(Janet source, int flags) {
 static const JanetInstructionDef *janet_asm_reverse_lookup(uint32_t instr) {
     size_t i;
     uint32_t opcode = instr & 0x7F;
-    for (i = 0; i < sizeof(janet_ops)/sizeof(JanetInstructionDef); i++) {
+    for (i = 0; i < sizeof(janet_ops) / sizeof(JanetInstructionDef); i++) {
         const JanetInstructionDef *def = janet_ops + i;
         if (def->opcode == opcode)
             return def;
@@ -759,31 +750,31 @@ static const JanetInstructionDef *janet_asm_reverse_lookup(uint32_t instr) {
 }
 
 /* Create some constant sized tuples */
-static Janet tup1(Janet x) {
+static const Janet *tup1(Janet x) {
     Janet *tup = janet_tuple_begin(1);
     tup[0] = x;
-    return janet_wrap_tuple(janet_tuple_end(tup));
+    return janet_tuple_end(tup);
 }
-static Janet tup2(Janet x, Janet y) {
+static const Janet *tup2(Janet x, Janet y) {
     Janet *tup = janet_tuple_begin(2);
     tup[0] = x;
     tup[1] = y;
-    return janet_wrap_tuple(janet_tuple_end(tup));
+    return janet_tuple_end(tup);
 }
-static Janet tup3(Janet x, Janet y, Janet z) {
+static const Janet *tup3(Janet x, Janet y, Janet z) {
     Janet *tup = janet_tuple_begin(3);
     tup[0] = x;
     tup[1] = y;
     tup[2] = z;
-    return janet_wrap_tuple(janet_tuple_end(tup));
+    return janet_tuple_end(tup);
 }
-static Janet tup4(Janet w, Janet x, Janet y, Janet z) {
+static const Janet *tup4(Janet w, Janet x, Janet y, Janet z) {
     Janet *tup = janet_tuple_begin(4);
     tup[0] = w;
     tup[1] = x;
     tup[2] = y;
     tup[3] = z;
-    return janet_wrap_tuple(janet_tuple_end(tup));
+    return janet_tuple_end(tup);
 }
 
 /* Given an argument, convert it to the appropriate integer or symbol */
@@ -794,41 +785,56 @@ Janet janet_asm_decode_instruction(uint32_t instr) {
         return janet_wrap_integer((int32_t)instr);
     }
     name = janet_csymbolv(def->name);
+    const Janet *ret = NULL;
 #define oparg(shift, mask) ((instr >> ((shift) << 3)) & (mask))
     switch (janet_instructions[def->opcode]) {
         case JINT_0:
-            return tup1(name);
+            ret = tup1(name);
+            break;
         case JINT_S:
-            return tup2(name, janet_wrap_integer(oparg(1, 0xFFFFFF)));
+            ret = tup2(name, janet_wrap_integer(oparg(1, 0xFFFFFF)));
+            break;
         case JINT_L:
-            return tup2(name, janet_wrap_integer((int32_t)instr >> 8));
+            ret = tup2(name, janet_wrap_integer((int32_t)instr >> 8));
+            break;
         case JINT_SS:
         case JINT_ST:
         case JINT_SC:
         case JINT_SU:
         case JINT_SD:
-            return tup3(name,
-                    janet_wrap_integer(oparg(1, 0xFF)),
-                    janet_wrap_integer(oparg(2, 0xFFFF)));
+            ret = tup3(name,
+                       janet_wrap_integer(oparg(1, 0xFF)),
+                       janet_wrap_integer(oparg(2, 0xFFFF)));
+            break;
         case JINT_SI:
         case JINT_SL:
-            return tup3(name,
-                    janet_wrap_integer(oparg(1, 0xFF)),
-                    janet_wrap_integer((int32_t)instr >> 16));
+            ret =  tup3(name,
+                        janet_wrap_integer(oparg(1, 0xFF)),
+                        janet_wrap_integer((int32_t)instr >> 16));
+            break;
         case JINT_SSS:
         case JINT_SES:
         case JINT_SSU:
-            return tup4(name,
-                    janet_wrap_integer(oparg(1, 0xFF)),
-                    janet_wrap_integer(oparg(2, 0xFF)),
-                    janet_wrap_integer(oparg(3, 0xFF)));
+            ret = tup4(name,
+                       janet_wrap_integer(oparg(1, 0xFF)),
+                       janet_wrap_integer(oparg(2, 0xFF)),
+                       janet_wrap_integer(oparg(3, 0xFF)));
+            break;
         case JINT_SSI:
-            return tup4(name,
-                    janet_wrap_integer(oparg(1, 0xFF)),
-                    janet_wrap_integer(oparg(2, 0xFF)),
-                    janet_wrap_integer((int32_t)instr >> 24));
+            ret = tup4(name,
+                       janet_wrap_integer(oparg(1, 0xFF)),
+                       janet_wrap_integer(oparg(2, 0xFF)),
+                       janet_wrap_integer((int32_t)instr >> 24));
+            break;
     }
 #undef oparg
+    if (ret) {
+        /* Check if break point set */
+        if (instr & 0x80) {
+            janet_tuple_flag(ret) |= JANET_TUPLE_FLAG_BRACKETCTOR;
+        }
+        return janet_wrap_tuple(ret);
+    }
     return janet_wrap_nil();
 }
 
@@ -838,6 +844,8 @@ Janet janet_disasm(JanetFuncDef *def) {
     JanetArray *constants;
     JanetTable *ret = janet_table(10);
     janet_table_put(ret, janet_csymbolv("arity"), janet_wrap_integer(def->arity));
+    janet_table_put(ret, janet_csymbolv("min-arity"), janet_wrap_integer(def->min_arity));
+    janet_table_put(ret, janet_csymbolv("max-arity"), janet_wrap_integer(def->max_arity));
     janet_table_put(ret, janet_csymbolv("bytecode"), janet_wrap_array(bcode));
     if (NULL != def->source) {
         janet_table_put(ret, janet_csymbolv("source"), janet_wrap_string(def->source));
@@ -845,9 +853,6 @@ Janet janet_disasm(JanetFuncDef *def) {
     if (def->flags & JANET_FUNCDEF_FLAG_VARARG) {
         janet_table_put(ret, janet_csymbolv("vararg"), janet_wrap_true());
     }
-    if (def->flags & JANET_FUNCDEF_FLAG_FIXARITY) {
-        janet_table_put(ret, janet_csymbolv("fix-arity"), janet_wrap_true());
-    }
     if (NULL != def->name) {
         janet_table_put(ret, janet_csymbolv("name"), janet_wrap_string(def->name));
     }
@@ -860,7 +865,7 @@ Janet janet_disasm(JanetFuncDef *def) {
             Janet src = def->constants[i];
             Janet dest;
             if (janet_checktype(src, JANET_TUPLE)) {
-                dest = tup2(janet_csymbolv("quote"), src);
+                dest = janet_wrap_tuple(tup2(janet_csymbolv("quote"), src));
             } else {
                 dest = src;
             }
@@ -881,8 +886,8 @@ Janet janet_disasm(JanetFuncDef *def) {
         for (i = 0; i < def->bytecode_length; i++) {
             Janet *t = janet_tuple_begin(2);
             JanetSourceMapping mapping = def->sourcemap[i];
-            t[0] = janet_wrap_integer(mapping.start);
-            t[1] = janet_wrap_integer(mapping.end);
+            t[0] = janet_wrap_integer(mapping.line);
+            t[1] = janet_wrap_integer(mapping.column);
             sourcemap->data[i] = janet_wrap_tuple(janet_tuple_end(t));
         }
         sourcemap->count = def->bytecode_length;
@@ -934,24 +939,26 @@ static Janet cfun_disasm(int32_t argc, Janet *argv) {
 }
 
 static const JanetReg asm_cfuns[] = {
-    {"asm", cfun_asm,
+    {
+        "asm", cfun_asm,
         JDOC("(asm assembly)\n\n"
-                "Returns a new function that is the compiled result of the assembly.\n"
-                "The syntax for the assembly can be found on the janet wiki. Will throw an\n"
-                "error on invalid assembly.")
+             "Returns a new function that is the compiled result of the assembly.\n"
+             "The syntax for the assembly can be found on the janet wiki. Will throw an\n"
+             "error on invalid assembly.")
     },
-    {"disasm", cfun_disasm,
+    {
+        "disasm", cfun_disasm,
         JDOC("(disasm func)\n\n"
-                "Returns assembly that could be used be compile the given function.\n"
-                "func must be a function, not a c function. Will throw on error on a badly\n"
-                "typed argument.")
+             "Returns assembly that could be used be compile the given function.\n"
+             "func must be a function, not a c function. Will throw on error on a badly\n"
+             "typed argument.")
     },
     {NULL, NULL, NULL}
 };
 
 /* Load the library */
 void janet_lib_asm(JanetTable *env) {
-    janet_cfuns(env, NULL, asm_cfuns);
+    janet_core_cfuns(env, NULL, asm_cfuns);
 }
 
 #endif

src/core/buffer.c

@@ -21,15 +21,17 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "gc.h"
 #include "util.h"
+#include "state.h"
 #endif
 
 /* Initialize a buffer */
 JanetBuffer *janet_buffer_init(JanetBuffer *buffer, int32_t capacity) {
     uint8_t *data = NULL;
     if (capacity > 0) {
+        janet_vm_next_collection += capacity;
         data = malloc(sizeof(uint8_t) * capacity);
         if (NULL == data) {
             JANET_OUT_OF_MEMORY;
@@ -57,8 +59,9 @@ void janet_buffer_ensure(JanetBuffer *buffer, int32_t capacity, int32_t growth)
     uint8_t *new_data;
     uint8_t *old = buffer->data;
     if (capacity <= buffer->capacity) return;
-    int64_t big_capacity = capacity * growth;
+    int64_t big_capacity = ((int64_t) capacity) * growth;
     capacity = big_capacity > INT32_MAX ? INT32_MAX : (int32_t) big_capacity;
+    janet_vm_next_collection += capacity - buffer->capacity;
     new_data = realloc(old, capacity * sizeof(uint8_t));
     if (NULL == new_data) {
         JANET_OUT_OF_MEMORY;
@@ -90,6 +93,7 @@ void janet_buffer_extra(JanetBuffer *buffer, int32_t n) {
     if (new_size > buffer->capacity) {
         int32_t new_capacity = new_size * 2;
         uint8_t *new_data = realloc(buffer->data, new_capacity * sizeof(uint8_t));
+        janet_vm_next_collection += new_capacity - buffer->capacity;
         if (NULL == new_data) {
             JANET_OUT_OF_MEMORY;
         }
@@ -172,7 +176,8 @@ static Janet cfun_buffer_new_filled(int32_t argc, Janet *argv) {
         byte = janet_getinteger(argv, 1) & 0xFF;
     }
     JanetBuffer *buffer = janet_buffer(count);
-    memset(buffer->data, byte, count);
+    if (buffer->data)
+        memset(buffer->data, byte, count);
     buffer->count = count;
     return janet_wrap_buffer(buffer);
 }
@@ -182,7 +187,7 @@ static Janet cfun_buffer_u8(int32_t argc, Janet *argv) {
     janet_arity(argc, 1, -1);
     JanetBuffer *buffer = janet_getbuffer(argv, 0);
     for (i = 1; i < argc; i++) {
-        janet_buffer_push_u8(buffer, (uint8_t) (janet_getinteger(argv, i) & 0xFF));
+        janet_buffer_push_u8(buffer, (uint8_t)(janet_getinteger(argv, i) & 0xFF));
     }
     return argv[0];
 }
@@ -207,6 +212,10 @@ static Janet cfun_buffer_chars(int32_t argc, Janet *argv) {
     JanetBuffer *buffer = janet_getbuffer(argv, 0);
     for (i = 1; i < argc; i++) {
         JanetByteView view = janet_getbytes(argv, i);
+        if (view.bytes == buffer->data) {
+            janet_buffer_ensure(buffer, buffer->count + view.len, 2);
+            view.bytes = buffer->data;
+        }
         janet_buffer_push_bytes(buffer, view.bytes, view.len);
     }
     return argv[0];
@@ -233,10 +242,11 @@ static Janet cfun_buffer_popn(int32_t argc, Janet *argv) {
 }
 
 static Janet cfun_buffer_slice(int32_t argc, Janet *argv) {
-    JanetRange range = janet_getslice(argc, argv);
     JanetByteView view = janet_getbytes(argv, 0);
+    JanetRange range = janet_getslice(argc, argv);
     JanetBuffer *buffer = janet_buffer(range.end - range.start);
-    memcpy(buffer->data, view.bytes + range.start, range.end - range.start);
+    if (buffer->data)
+        memcpy(buffer->data, view.bytes + range.start, range.end - range.start);
     buffer->count = range.end - range.start;
     return janet_wrap_buffer(buffer);
 }
@@ -294,6 +304,7 @@ static Janet cfun_buffer_blit(int32_t argc, Janet *argv) {
     janet_arity(argc, 2, 5);
     JanetBuffer *dest = janet_getbuffer(argv, 0);
     JanetByteView src = janet_getbytes(argv, 1);
+    int same_buf = src.bytes == dest->data;
     int32_t offset_dest = 0;
     int32_t offset_src = 0;
     if (argc > 2)
@@ -313,79 +324,111 @@ static Janet cfun_buffer_blit(int32_t argc, Janet *argv) {
         janet_panic("buffer blit out of range");
     janet_buffer_ensure(dest, (int32_t) last, 2);
     if (last > dest->count) dest->count = (int32_t) last;
-    memcpy(dest->data + offset_dest, src.bytes + offset_src, length_src);
+    if (same_buf) {
+        src.bytes = dest->data;
+        memmove(dest->data + offset_dest, src.bytes + offset_src, length_src);
+    } else {
+        memcpy(dest->data + offset_dest, src.bytes + offset_src, length_src);
+    }
+    return argv[0];
+}
+
+static Janet cfun_buffer_format(int32_t argc, Janet *argv) {
+    janet_arity(argc, 2, -1);
+    JanetBuffer *buffer = janet_getbuffer(argv, 0);
+    const char *strfrmt = (const char *) janet_getstring(argv, 1);
+    janet_buffer_format(buffer, strfrmt, 1, argc, argv);
     return argv[0];
 }
 
 static const JanetReg buffer_cfuns[] = {
-    {"buffer/new", cfun_buffer_new,
+    {
+        "buffer/new", cfun_buffer_new,
         JDOC("(buffer/new capacity)\n\n"
-                "Creates a new, empty buffer with enough memory for capacity bytes. "
-                "Returns a new buffer.")
+             "Creates a new, empty buffer with enough memory for capacity bytes. "
+             "Returns a new buffer.")
     },
-    {"buffer/new-filled", cfun_buffer_new_filled,
-        JDOC("(buffer/new-filled count [, byte=0])\n\n"
-                "Creates a new buffer of length count filled with byte. "
-                "Returns the new buffer.")
+    {
+        "buffer/new-filled", cfun_buffer_new_filled,
+        JDOC("(buffer/new-filled count &opt byte)\n\n"
+             "Creates a new buffer of length count filled with byte. By default, byte is 0. "
+             "Returns the new buffer.")
     },
-    {"buffer/push-byte", cfun_buffer_u8,
+    {
+        "buffer/push-byte", cfun_buffer_u8,
         JDOC("(buffer/push-byte buffer x)\n\n"
-                "Append a byte to a buffer. Will expand the buffer as necessary. "
-                "Returns the modified buffer. Will throw an error if the buffer overflows.")
+             "Append a byte to a buffer. Will expand the buffer as necessary. "
+             "Returns the modified buffer. Will throw an error if the buffer overflows.")
     },
-    {"buffer/push-word", cfun_buffer_word,
+    {
+        "buffer/push-word", cfun_buffer_word,
         JDOC("(buffer/push-word buffer x)\n\n"
-                "Append a machine word to a buffer. The 4 bytes of the integer are appended "
-                "in twos complement, big endian order, unsigned. Returns the modified buffer. Will "
-                "throw an error if the buffer overflows.")
+             "Append a machine word to a buffer. The 4 bytes of the integer are appended "
+             "in twos complement, big endian order, unsigned. Returns the modified buffer. Will "
+             "throw an error if the buffer overflows.")
     },
-    {"buffer/push-string", cfun_buffer_chars,
+    {
+        "buffer/push-string", cfun_buffer_chars,
         JDOC("(buffer/push-string buffer str)\n\n"
-                "Push a string onto the end of a buffer. Non string values will be converted "
-                "to strings before being pushed. Returns the modified buffer. "
-                "Will throw an error if the buffer overflows.")
+             "Push a string onto the end of a buffer. Non string values will be converted "
+             "to strings before being pushed. Returns the modified buffer. "
+             "Will throw an error if the buffer overflows.")
     },
-    {"buffer/popn", cfun_buffer_popn,
+    {
+        "buffer/popn", cfun_buffer_popn,
         JDOC("(buffer/popn buffer n)\n\n"
-                "Removes the last n bytes from the buffer. Returns the modified buffer.")
+             "Removes the last n bytes from the buffer. Returns the modified buffer.")
     },
-    {"buffer/clear", cfun_buffer_clear,
+    {
+        "buffer/clear", cfun_buffer_clear,
         JDOC("(buffer/clear buffer)\n\n"
-                "Sets the size of a buffer to 0 and empties it. The buffer retains "
-                "its memory so it can be efficiently refilled. Returns the modified buffer.")
+             "Sets the size of a buffer to 0 and empties it. The buffer retains "
+             "its memory so it can be efficiently refilled. Returns the modified buffer.")
     },
-    {"buffer/slice", cfun_buffer_slice,
-        JDOC("(buffer/slice bytes [, start=0 [, end=(length bytes)]])\n\n"
-                "Takes a slice of a byte sequence from start to end. The range is half open, "
-                "[start, end). Indexes can also be negative, indicating indexing from the end of the "
-                "end of the array. By default, start is 0 and end is the length of the buffer. "
-                "Returns a new buffer.")
+    {
+        "buffer/slice", cfun_buffer_slice,
+        JDOC("(buffer/slice bytes &opt start end)\n\n"
+             "Takes a slice of a byte sequence from start to end. The range is half open, "
+             "[start, end). Indexes can also be negative, indicating indexing from the end of the "
+             "end of the array. By default, start is 0 and end is the length of the buffer. "
+             "Returns a new buffer.")
     },
-    {"buffer/bit-set", cfun_buffer_bitset,
+    {
+        "buffer/bit-set", cfun_buffer_bitset,
         JDOC("(buffer/bit-set buffer index)\n\n"
-                "Sets the bit at the given bit-index. Returns the buffer.")
+             "Sets the bit at the given bit-index. Returns the buffer.")
     },
-    {"buffer/bit-clear", cfun_buffer_bitclear,
+    {
+        "buffer/bit-clear", cfun_buffer_bitclear,
         JDOC("(buffer/bit-clear buffer index)\n\n"
-                "Clears the bit at the given bit-index. Returns the buffer.")
+             "Clears the bit at the given bit-index. Returns the buffer.")
     },
-    {"buffer/bit", cfun_buffer_bitget,
+    {
+        "buffer/bit", cfun_buffer_bitget,
         JDOC("(buffer/bit buffer index)\n\n"
-                "Gets the bit at the given bit-index. Returns true if the bit is set, false if not.")
+             "Gets the bit at the given bit-index. Returns true if the bit is set, false if not.")
     },
-    {"buffer/bit-toggle", cfun_buffer_bittoggle,
+    {
+        "buffer/bit-toggle", cfun_buffer_bittoggle,
         JDOC("(buffer/bit-toggle buffer index)\n\n"
-                "Toggles the bit at the given bit index in buffer. Returns the buffer.")
+             "Toggles the bit at the given bit index in buffer. Returns the buffer.")
     },
-    {"buffer/blit", cfun_buffer_blit,
-        JDOC("(buffer/blit dest src [, dest-start=0 [, src-start=0 [, src-end=-1]]])\n\n"
-                "Insert the contents of src into dest. Can optionally take indices that "
-                "indicate which part of src to copy into which part of dest. Indices can be "
-                "negative to index from the end of src or dest. Returns dest.")
+    {
+        "buffer/blit", cfun_buffer_blit,
+        JDOC("(buffer/blit dest src & opt dest-start src-start src-end)\n\n"
+             "Insert the contents of src into dest. Can optionally take indices that "
+             "indicate which part of src to copy into which part of dest. Indices can be "
+             "negative to index from the end of src or dest. Returns dest.")
+    },
+    {
+        "buffer/format", cfun_buffer_format,
+        JDOC("(buffer/format buffer format & args)\n\n"
+             "Snprintf like functionality for printing values into a buffer. Returns "
+             " the modified buffer.")
     },
     {NULL, NULL, NULL}
 };
 
 void janet_lib_buffer(JanetTable *env) {
-    janet_cfuns(env, NULL, buffer_cfuns);
+    janet_core_cfuns(env, NULL, buffer_cfuns);
 }

src/core/bytecode.c

@@ -21,8 +21,9 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "gc.h"
+#include "util.h"
 #endif
 
 /* Look up table for instructions */
@@ -78,6 +79,8 @@ enum JanetInstructionType janet_instructions[JOP_INSTRUCTION_COUNT] = {
     JINT_S, /* JOP_TAILCALL, */
     JINT_SSS, /* JOP_RESUME, */
     JINT_SSU, /* JOP_SIGNAL, */
+    JINT_SSS, /* JOP_PROPAGATE */
+    JINT_SSS, /* JOP_IN, */
     JINT_SSS, /* JOP_GET, */
     JINT_SSS, /* JOP_PUT, */
     JINT_SSU, /* JOP_GET_INDEX, */
@@ -85,10 +88,11 @@ enum JanetInstructionType janet_instructions[JOP_INSTRUCTION_COUNT] = {
     JINT_SS, /* JOP_LENGTH */
     JINT_S, /* JOP_MAKE_ARRAY */
     JINT_S, /* JOP_MAKE_BUFFER */
-    JINT_S, /* JOP_MAKE_TUPLE */
+    JINT_S, /* JOP_MAKE_STRING */
     JINT_S, /* JOP_MAKE_STRUCT */
     JINT_S, /* JOP_MAKE_TABLE */
-    JINT_S, /* JOP_MAKE_STRING */
+    JINT_S, /* JOP_MAKE_TUPLE */
+    JINT_S, /* JOP_MAKE_BRACKET_TUPLE */
     JINT_SSS, /* JOP_NUMERIC_LESS_THAN */
     JINT_SSS, /* JOP_NUMERIC_LESS_THAN_EQUAL */
     JINT_SSS, /* JOP_NUMERIC_GREATER_THAN */
@@ -118,72 +122,62 @@ int32_t janet_verify(JanetFuncDef *def) {
         switch (type) {
             case JINT_0:
                 continue;
-            case JINT_S:
-                {
-                    if ((int32_t)(instr >> 8) >= sc) return 4;
-                    continue;
-                }
+            case JINT_S: {
+                if ((int32_t)(instr >> 8) >= sc) return 4;
+                continue;
+            }
             case JINT_SI:
             case JINT_SU:
-            case JINT_ST:
-                {
-                    if ((int32_t)((instr >> 8) & 0xFF) >= sc) return 4;
-                    continue;
-                }
-            case JINT_L:
-                {
-                    int32_t jumpdest = i + (((int32_t)instr) >> 8);
-                    if (jumpdest < 0 || jumpdest >= def->bytecode_length) return 5;
-                    continue;
-                }
-            case JINT_SS:
-                {
-                    if ((int32_t)((instr >> 8) & 0xFF) >= sc ||
+            case JINT_ST: {
+                if ((int32_t)((instr >> 8) & 0xFF) >= sc) return 4;
+                continue;
+            }
+            case JINT_L: {
+                int32_t jumpdest = i + (((int32_t)instr) >> 8);
+                if (jumpdest < 0 || jumpdest >= def->bytecode_length) return 5;
+                continue;
+            }
+            case JINT_SS: {
+                if ((int32_t)((instr >> 8) & 0xFF) >= sc ||
                         (int32_t)(instr >> 16) >= sc) return 4;
-                    continue;
-                }
+                continue;
+            }
             case JINT_SSI:
-            case JINT_SSU:
-                {
-                    if ((int32_t)((instr >> 8) & 0xFF) >= sc ||
+            case JINT_SSU: {
+                if ((int32_t)((instr >> 8) & 0xFF) >= sc ||
                         (int32_t)((instr >> 16) & 0xFF) >= sc) return 4;
-                    continue;
-                }
-            case JINT_SL:
-                {
-                    int32_t jumpdest = i + (((int32_t)instr) >> 16);
-                    if ((int32_t)((instr >> 8) & 0xFF) >= sc) return 4;
-                    if (jumpdest < 0 || jumpdest >= def->bytecode_length) return 5;
-                    continue;
-                }
-            case JINT_SSS:
-                {
-                    if (((int32_t)(instr >> 8) & 0xFF) >= sc ||
+                continue;
+            }
+            case JINT_SL: {
+                int32_t jumpdest = i + (((int32_t)instr) >> 16);
+                if ((int32_t)((instr >> 8) & 0xFF) >= sc) return 4;
+                if (jumpdest < 0 || jumpdest >= def->bytecode_length) return 5;
+                continue;
+            }
+            case JINT_SSS: {
+                if (((int32_t)(instr >> 8) & 0xFF) >= sc ||
                         ((int32_t)(instr >> 16) & 0xFF) >= sc ||
                         ((int32_t)(instr >> 24) & 0xFF) >= sc) return 4;
-                    continue;
-                }
-            case JINT_SD:
-                {
-                    if ((int32_t)((instr >> 8) & 0xFF) >= sc) return 4;
-                    if ((int32_t)(instr >> 16) >= def->defs_length) return 6;
-                    continue;
-                }
-            case JINT_SC:
-                {
-                    if ((int32_t)((instr >> 8) & 0xFF) >= sc) return 4;
-                    if ((int32_t)(instr >> 16) >= def->constants_length) return 7;
-                    continue;
-                }
-            case JINT_SES:
-                {
-                    /* How can we check the last slot index? We need info parent funcdefs. Resort
-                     * to runtime checks for now. Maybe invalid upvalue references could be defaulted
-                     * to nil? (don't commit to this in the long term, though) */
-                    if ((int32_t)((instr >> 8) & 0xFF) >= sc) return 4;
-                    if ((int32_t)((instr >> 16) & 0xFF) >= def->environments_length) return 8;
-                    continue;
-                }
+                continue;
+            }
+            case JINT_SD: {
+                if ((int32_t)((instr >> 8) & 0xFF) >= sc) return 4;
+                if ((int32_t)(instr >> 16) >= def->defs_length) return 6;
+                continue;
+            }
+            case JINT_SC: {
+                if ((int32_t)((instr >> 8) & 0xFF) >= sc) return 4;
+                if ((int32_t)(instr >> 16) >= def->constants_length) return 7;
+                continue;
+            }
+            case JINT_SES: {
+                /* How can we check the last slot index? We need info parent funcdefs. Resort
+                 * to runtime checks for now. Maybe invalid upvalue references could be defaulted
+                 * to nil? (don't commit to this in the long term, though) */
+                if ((int32_t)((instr >> 8) & 0xFF) >= sc) return 4;
+                if ((int32_t)((instr >> 16) & 0xFF) >= def->environments_length) return 8;
+                continue;
+            }
         }
     }
 
@@ -210,7 +204,7 @@ int32_t janet_verify(JanetFuncDef *def) {
 
 /* Allocate an empty funcdef. This function may have added functionality
  * as commonalities between asm and compile arise. */
-JanetFuncDef *janet_funcdef_alloc() {
+JanetFuncDef *janet_funcdef_alloc(void) {
     JanetFuncDef *def = janet_gcalloc(JANET_MEMORY_FUNCDEF, sizeof(JanetFuncDef));
     def->environments = NULL;
     def->constants = NULL;
@@ -218,6 +212,8 @@ JanetFuncDef *janet_funcdef_alloc() {
     def->flags = 0;
     def->slotcount = 0;
     def->arity = 0;
+    def->min_arity = 0;
+    def->max_arity = INT32_MAX;
     def->source = NULL;
     def->sourcemap = NULL;
     def->name = NULL;

src/core/capi.c

@@ -21,21 +21,36 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "state.h"
 #include "fiber.h"
 #endif
 
 void janet_panicv(Janet message) {
-    if (janet_vm_fiber != NULL) {
-        janet_fiber_push(janet_vm_fiber, message);
-        longjmp(janet_vm_fiber->buf, 1);
+    if (janet_vm_return_reg != NULL) {
+        *janet_vm_return_reg = message;
+        longjmp(*janet_vm_jmp_buf, 1);
     } else {
         fputs((const char *)janet_formatc("janet top level panic - %v\n", message), stdout);
         exit(1);
     }
 }
 
+void janet_panicf(const char *format, ...) {
+    va_list args;
+    const uint8_t *ret;
+    JanetBuffer buffer;
+    int32_t len = 0;
+    while (format[len]) len++;
+    janet_buffer_init(&buffer, len);
+    va_start(args, format);
+    janet_formatb(&buffer, format, args);
+    va_end(args);
+    ret = janet_string(buffer.data, buffer.count);
+    janet_buffer_deinit(&buffer);
+    janet_panics(ret);
+}
+
 void janet_panic(const char *message) {
     janet_panicv(janet_cstringv(message));
 }
@@ -73,6 +88,32 @@ type janet_get##name(const Janet *argv, int32_t n) { \
     return janet_unwrap_##name(x); \
 }
 
+#define DEFINE_OPT(name, NAME, type) \
+type janet_opt##name(const Janet *argv, int32_t argc, int32_t n, type dflt) { \
+    if (n >= argc) return dflt; \
+    if (janet_checktype(argv[n], JANET_NIL)) return dflt; \
+    return janet_get##name(argv, n); \
+}
+
+#define DEFINE_OPTLEN(name, NAME, type) \
+type janet_opt##name(const Janet *argv, int32_t argc, int32_t n, int32_t dflt_len) { \
+    if (n >= argc || janet_checktype(argv[n], JANET_NIL)) {\
+        return janet_##name(dflt_len); \
+    }\
+    return janet_get##name(argv, n); \
+}
+
+int janet_getmethod(const uint8_t *method, const JanetMethod *methods, Janet *out) {
+    while (methods->name) {
+        if (!janet_cstrcmp(method, methods->name)) {
+            *out = janet_wrap_cfunction(methods->cfun);
+            return 1;
+        }
+        methods++;
+    }
+    return 0;
+}
+
 DEFINE_GETTER(number, NUMBER, double)
 DEFINE_GETTER(array, ARRAY, JanetArray *)
 DEFINE_GETTER(tuple, TUPLE, const Janet *)
@@ -85,21 +126,52 @@ DEFINE_GETTER(buffer, BUFFER, JanetBuffer *)
 DEFINE_GETTER(fiber, FIBER, JanetFiber *)
 DEFINE_GETTER(function, FUNCTION, JanetFunction *)
 DEFINE_GETTER(cfunction, CFUNCTION, JanetCFunction)
+DEFINE_GETTER(boolean, BOOLEAN, int)
+DEFINE_GETTER(pointer, POINTER, void *)
 
-int janet_getboolean(const Janet *argv, int32_t n) {
-    Janet x = argv[n];
-    if (janet_checktype(x, JANET_TRUE)) {
-        return 1;
-    } else if (!janet_checktype(x, JANET_FALSE)) {
-        janet_panicf("bad slot #%d, expected boolean, got %v", n, x);
+DEFINE_OPT(number, NUMBER, double)
+DEFINE_OPT(tuple, TUPLE, const Janet *)
+DEFINE_OPT(struct, STRUCT, const JanetKV *)
+DEFINE_OPT(string, STRING, const uint8_t *)
+DEFINE_OPT(keyword, KEYWORD, const uint8_t *)
+DEFINE_OPT(symbol, SYMBOL, const uint8_t *)
+DEFINE_OPT(fiber, FIBER, JanetFiber *)
+DEFINE_OPT(function, FUNCTION, JanetFunction *)
+DEFINE_OPT(cfunction, CFUNCTION, JanetCFunction)
+DEFINE_OPT(boolean, BOOLEAN, int)
+DEFINE_OPT(pointer, POINTER, void *)
+
+DEFINE_OPTLEN(buffer, BUFFER, JanetBuffer *)
+DEFINE_OPTLEN(table, TABLE, JanetTable *)
+DEFINE_OPTLEN(array, ARRAY, JanetArray *)
+
+#undef DEFINE_GETTER
+#undef DEFINE_OPT
+#undef DEFINE_OPTLEN
+
+const char *janet_getcstring(const Janet *argv, int32_t n) {
+    const uint8_t *jstr = janet_getstring(argv, n);
+    const char *cstr = (const char *)jstr;
+    if (strlen(cstr) != (size_t) janet_string_length(jstr)) {
+        janet_panicf("string %v contains embedded 0s");
     }
-    return 0;
+    return cstr;
+}
+
+int32_t janet_getnat(const Janet *argv, int32_t n) {
+    Janet x = argv[n];
+    if (!janet_checkint(x)) goto bad;
+    int32_t ret = janet_unwrap_integer(x);
+    if (ret < 0) goto bad;
+    return ret;
+bad:
+    janet_panicf("bad slot #%d, expected non-negative 32 bit signed integer, got %v", n, x);
 }
 
 int32_t janet_getinteger(const Janet *argv, int32_t n) {
     Janet x = argv[n];
     if (!janet_checkint(x)) {
-        janet_panicf("bad slot #%d, expected integer, got %v", n, x);
+        janet_panicf("bad slot #%d, expected 32 bit signed integer, got %v", n, x);
     }
     return janet_unwrap_integer(x);
 }
@@ -107,11 +179,19 @@ int32_t janet_getinteger(const Janet *argv, int32_t n) {
 int64_t janet_getinteger64(const Janet *argv, int32_t n) {
     Janet x = argv[n];
     if (!janet_checkint64(x)) {
-        janet_panicf("bad slot #%d, expected 64 bit integer, got %v", n, x);
+        janet_panicf("bad slot #%d, expected 64 bit signed integer, got %v", n, x);
     }
     return (int64_t) janet_unwrap_number(x);
 }
 
+size_t janet_getsize(const Janet *argv, int32_t n) {
+    Janet x = argv[n];
+    if (!janet_checksize(x)) {
+        janet_panicf("bad slot #%d, expected size, got %v", n, x);
+    }
+    return (size_t) janet_unwrap_number(x);
+}
+
 int32_t janet_gethalfrange(const Janet *argv, int32_t n, int32_t length, const char *which) {
     int32_t raw = janet_getinteger(argv, n);
     if (raw < 0) raw += length + 1;
@@ -175,13 +255,106 @@ JanetRange janet_getslice(int32_t argc, const Janet *argv) {
         range.start = 0;
         range.end = length;
     } else if (argc == 2) {
-        range.start = janet_gethalfrange(argv, 1, length, "start");
+        range.start = janet_checktype(argv[1], JANET_NIL)
+                      ? 0
+                      : janet_gethalfrange(argv, 1, length, "start");
         range.end = length;
     } else {
-        range.start = janet_gethalfrange(argv, 1, length, "start");
-        range.end = janet_gethalfrange(argv, 2, length, "end");
+        range.start = janet_checktype(argv[1], JANET_NIL)
+                      ? 0
+                      : janet_gethalfrange(argv, 1, length, "start");
+        range.end = janet_checktype(argv[2], JANET_NIL)
+                    ? length
+                    : janet_gethalfrange(argv, 2, length, "end");
         if (range.end < range.start)
             range.end = range.start;
     }
     return range;
 }
+
+Janet janet_dyn(const char *name) {
+    if (!janet_vm_fiber) return janet_wrap_nil();
+    if (janet_vm_fiber->env) {
+        return janet_table_get(janet_vm_fiber->env, janet_ckeywordv(name));
+    } else {
+        return janet_wrap_nil();
+    }
+}
+
+void janet_setdyn(const char *name, Janet value) {
+    if (!janet_vm_fiber) return;
+    if (!janet_vm_fiber->env) {
+        janet_vm_fiber->env = janet_table(1);
+    }
+    janet_table_put(janet_vm_fiber->env, janet_ckeywordv(name), value);
+}
+
+uint64_t janet_getflags(const Janet *argv, int32_t n, const char *flags) {
+    uint64_t ret = 0;
+    const uint8_t *keyw = janet_getkeyword(argv, n);
+    int32_t klen = janet_string_length(keyw);
+    int32_t flen = (int32_t) strlen(flags);
+    if (flen > 64) {
+        flen = 64;
+    }
+    for (int32_t j = 0; j < klen; j++) {
+        for (int32_t i = 0; i < flen; i++) {
+            if (((uint8_t) flags[i]) == keyw[j]) {
+                ret |= 1ULL << i;
+                goto found;
+            }
+        }
+        janet_panicf("unexpected flag %c, expected one of \"%s\"", (char) keyw[j], flags);
+    found:
+        ;
+    }
+    return ret;
+}
+
+int32_t janet_optnat(const Janet *argv, int32_t argc, int32_t n, int32_t dflt) {
+    if (argc <= n) return dflt;
+    if (janet_checktype(argv[n], JANET_NIL)) return dflt;
+    return janet_getnat(argv, n);
+}
+
+int32_t janet_optinteger(const Janet *argv, int32_t argc, int32_t n, int32_t dflt) {
+    if (argc <= n) return dflt;
+    if (janet_checktype(argv[n], JANET_NIL)) return dflt;
+    return janet_getinteger(argv, n);
+}
+
+int64_t janet_optinteger64(const Janet *argv, int32_t argc, int32_t n, int64_t dflt) {
+    if (argc <= n) return dflt;
+    if (janet_checktype(argv[n], JANET_NIL)) return dflt;
+    return janet_getinteger64(argv, n);
+}
+
+size_t janet_optsize(const Janet *argv, int32_t argc, int32_t n, size_t dflt) {
+    if (argc <= n) return dflt;
+    if (janet_checktype(argv[n], JANET_NIL)) return dflt;
+    return janet_getsize(argv, n);
+}
+
+void *janet_optabstract(const Janet *argv, int32_t argc, int32_t n, const JanetAbstractType *at, void *dflt) {
+    if (argc <= n) return dflt;
+    if (janet_checktype(argv[n], JANET_NIL)) return dflt;
+    return janet_getabstract(argv, n, at);
+}
+
+/* Some definitions for function-like macros */
+
+JANET_API JanetStructHead *(janet_struct_head)(const JanetKV *st) {
+    return janet_struct_head(st);
+}
+
+JANET_API JanetAbstractHead *(janet_abstract_head)(const void *abstract) {
+    return janet_abstract_head(abstract);
+}
+
+JANET_API JanetStringHead *(janet_string_head)(const uint8_t *s) {
+    return janet_string_head(s);
+}
+
+JANET_API JanetTupleHead *(janet_tuple_head)(const Janet *tuple) {
+    return janet_tuple_head(tuple);
+}

src/core/cfuns.c

@@ -21,20 +21,20 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "compile.h"
 #include "emit.h"
 #include "vector.h"
 #endif
 
-static int fixarity0(JanetFopts opts, JanetSlot *args) {
-    (void) opts;
-    return janet_v_count(args) == 0;
-}
 static int fixarity1(JanetFopts opts, JanetSlot *args) {
     (void) opts;
     return janet_v_count(args) == 1;
 }
+static int maxarity1(JanetFopts opts, JanetSlot *args) {
+    (void) opts;
+    return janet_v_count(args) <= 1;
+}
 static int minarity2(JanetFopts opts, JanetSlot *args) {
     (void) opts;
     return janet_v_count(args) >= 2;
@@ -64,10 +64,10 @@ static JanetSlot genericSSI(JanetFopts opts, int op, JanetSlot s, int32_t imm) {
 
 /* Emit a series of instructions instead of a function call to a math op */
 static JanetSlot opreduce(
-        JanetFopts opts,
-        JanetSlot *args,
-        int op,
-        Janet nullary) {
+    JanetFopts opts,
+    JanetSlot *args,
+    int op,
+    Janet nullary) {
     JanetCompiler *c = opts.compiler;
     int32_t i, len;
     len = janet_v_count(args);
@@ -88,27 +88,49 @@ static JanetSlot opreduce(
 
 /* Function optimizers */
 
+static JanetSlot do_propagate(JanetFopts opts, JanetSlot *args) {
+    return opreduce(opts, args, JOP_PROPAGATE, janet_wrap_nil());
+}
 static JanetSlot do_error(JanetFopts opts, JanetSlot *args) {
     janetc_emit_s(opts.compiler, JOP_ERROR, args[0], 0);
     return janetc_cslot(janet_wrap_nil());
 }
 static JanetSlot do_debug(JanetFopts opts, JanetSlot *args) {
     (void)args;
-    janetc_emit(opts.compiler, JOP_SIGNAL | (2 << 24));
-    return janetc_cslot(janet_wrap_nil());
+    int32_t len = janet_v_count(args);
+    JanetSlot t = janetc_gettarget(opts);
+    janetc_emit_ssu(opts.compiler, JOP_SIGNAL, t,
+                    (len == 1) ? args[0] : janetc_cslot(janet_wrap_nil()),
+                    JANET_SIGNAL_DEBUG,
+                    1);
+    return t;
+}
+static JanetSlot do_in(JanetFopts opts, JanetSlot *args) {
+    return opreduce(opts, args, JOP_IN, janet_wrap_nil());
 }
 static JanetSlot do_get(JanetFopts opts, JanetSlot *args) {
     return opreduce(opts, args, JOP_GET, janet_wrap_nil());
 }
 static JanetSlot do_put(JanetFopts opts, JanetSlot *args) {
-    janetc_emit_sss(opts.compiler, JOP_PUT, args[0], args[1], args[2], 0);
-    return args[0];
+    if (opts.flags & JANET_FOPTS_DROP) {
+        janetc_emit_sss(opts.compiler, JOP_PUT, args[0], args[1], args[2], 0);
+        return janetc_cslot(janet_wrap_nil());
+    } else {
+        JanetSlot t = janetc_gettarget(opts);
+        janetc_copy(opts.compiler, t, args[0]);
+        janetc_emit_sss(opts.compiler, JOP_PUT, t, args[1], args[2], 0);
+        return t;
+    }
 }
 static JanetSlot do_length(JanetFopts opts, JanetSlot *args) {
     return genericSS(opts, JOP_LENGTH, args[0]);
 }
 static JanetSlot do_yield(JanetFopts opts, JanetSlot *args) {
-    return genericSSI(opts, JOP_SIGNAL, args[0], 3);
+    if (janet_v_count(args) == 0) {
+        return genericSSI(opts, JOP_SIGNAL, janetc_cslot(janet_wrap_nil()), 3);
+    } else {
+        return genericSSI(opts, JOP_SIGNAL, args[0], 3);
+    }
 }
 static JanetSlot do_resume(JanetFopts opts, JanetSlot *args) {
     return opreduce(opts, args, JOP_RESUME, janet_wrap_nil());
@@ -118,9 +140,9 @@ static JanetSlot do_apply(JanetFopts opts, JanetSlot *args) {
     JanetCompiler *c = opts.compiler;
     int32_t i;
     for (i = 1; i < janet_v_count(args) - 3; i += 3)
-        janetc_emit_sss(c, JOP_PUSH_3, args[i], args[i+1], args[i+2], 0);
+        janetc_emit_sss(c, JOP_PUSH_3, args[i], args[i + 1], args[i + 2], 0);
     if (i == janet_v_count(args) - 3)
-        janetc_emit_ss(c, JOP_PUSH_2, args[i], args[i+1], 0);
+        janetc_emit_ss(c, JOP_PUSH_2, args[i], args[i + 1], 0);
     else if (i == janet_v_count(args) - 2)
         janetc_emit_s(c, JOP_PUSH, args[i], 0);
     /* Push array phase */
@@ -176,10 +198,10 @@ static JanetSlot do_bnot(JanetFopts opts, JanetSlot *args) {
 
 /* Specialization for comparators */
 static JanetSlot compreduce(
-        JanetFopts opts,
-        JanetSlot *args,
-        int op,
-        int invert) {
+    JanetFopts opts,
+    JanetSlot *args,
+    int op,
+    int invert) {
     JanetCompiler *c = opts.compiler;
     int32_t i, len;
     len = janet_v_count(args);
@@ -187,8 +209,8 @@ static JanetSlot compreduce(
     JanetSlot t;
     if (len < 2) {
         return invert
-            ? janetc_cslot(janet_wrap_false())
-            : janetc_cslot(janet_wrap_true());
+               ? janetc_cslot(janet_wrap_false())
+               : janetc_cslot(janet_wrap_true());
     }
     t = janetc_gettarget(opts);
     for (i = 1; i < len; i++) {
@@ -252,12 +274,12 @@ static JanetSlot do_neq(JanetFopts opts, JanetSlot *args) {
 
 /* Arranged by tag */
 static const JanetFunOptimizer optimizers[] = {
-    {fixarity0, do_debug},
+    {maxarity1, do_debug},
     {fixarity1, do_error},
     {minarity2, do_apply},
-    {fixarity1, do_yield},
+    {maxarity1, do_yield},
     {fixarity2, do_resume},
-    {fixarity2, do_get},
+    {fixarity2, do_in},
     {fixarity3, do_put},
     {fixarity1, do_length},
     {NULL, do_add},
@@ -282,7 +304,9 @@ static const JanetFunOptimizer optimizers[] = {
     {NULL, do_gte},
     {NULL, do_lte},
     {NULL, do_eq},
-    {NULL, do_neq}
+    {NULL, do_neq},
+    {fixarity2, do_propagate},
+    {fixarity2, do_get}
 };
 
 const JanetFunOptimizer *janetc_funopt(uint32_t flags) {
@@ -290,7 +314,7 @@ const JanetFunOptimizer *janetc_funopt(uint32_t flags) {
     if (tag == 0)
         return NULL;
     uint32_t index = tag - 1;
-    if (index >= (sizeof(optimizers)/sizeof(optimizers[0])))
+    if (index >= (sizeof(optimizers) / sizeof(optimizers[0])))
         return NULL;
     return optimizers + index;
 }

src/core/compile.c

@@ -21,11 +21,12 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "compile.h"
 #include "emit.h"
 #include "vector.h"
 #include "util.h"
+#include "state.h"
 #endif
 
 JanetFopts janetc_fopts_default(JanetCompiler *c) {
@@ -97,7 +98,6 @@ void janetc_scope(JanetScope *s, JanetCompiler *c, int flags, const char *name)
     scope.syms = NULL;
     scope.envs = NULL;
     scope.defs = NULL;
-    scope.selfconst = -1;
     scope.bytecode_start = janet_v_count(c->buffer);
     scope.flags = flags;
     scope.parent = c->scope;
@@ -166,8 +166,8 @@ void janetc_popscope_keepslot(JanetCompiler *c, JanetSlot retslot) {
 
 /* Allow searching for symbols. Return information about the symbol */
 JanetSlot janetc_resolve(
-        JanetCompiler *c,
-        const uint8_t *sym) {
+    JanetCompiler *c,
+    const uint8_t *sym) {
 
     JanetSlot ret = janetc_cslot(janet_wrap_nil());
     JanetScope *scope = c->scope;
@@ -206,8 +206,7 @@ JanetSlot janetc_resolve(
             case JANET_BINDING_DEF:
             case JANET_BINDING_MACRO: /* Macro should function like defs when not in calling pos */
                 return janetc_cslot(check);
-            case JANET_BINDING_VAR:
-            {
+            case JANET_BINDING_VAR: {
                 JanetSlot ret = janetc_cslot(check);
                 /* TODO save type info */
                 ret.flags |= JANET_SLOT_REF | JANET_SLOT_NAMED | JANET_SLOT_MUTABLE | JANET_SLOTTYPE_ANY;
@@ -218,7 +217,7 @@ JanetSlot janetc_resolve(
     }
 
     /* Symbol was found */
-    found:
+found:
 
     /* Constants can be returned immediately (they are stateless) */
     if (ret.flags & (JANET_SLOT_CONSTANT | JANET_SLOT_REF))
@@ -283,8 +282,8 @@ JanetSlot janetc_return(JanetCompiler *c, JanetSlot s) {
 JanetSlot janetc_gettarget(JanetFopts opts) {
     JanetSlot slot;
     if ((opts.flags & JANET_FOPTS_HINT) &&
-        (opts.hint.envindex < 0) &&
-        (opts.hint.index >= 0 && opts.hint.index <= 0xFF)) {
+            (opts.hint.envindex < 0) &&
+            (opts.hint.index >= 0 && opts.hint.index <= 0xFF)) {
         slot = opts.hint;
     } else {
         slot.envindex = -1;
@@ -311,9 +310,9 @@ JanetSlot *janetc_toslotskv(JanetCompiler *c, Janet ds) {
     JanetSlot *ret = NULL;
     JanetFopts subopts = janetc_fopts_default(c);
     const JanetKV *kvs = NULL;
-    int32_t cap, i, len;
+    int32_t cap = 0, len = 0;
     janet_dictionary_view(ds, &kvs, &len, &cap);
-    for (i = 0; i < cap; i++) {
+    for (int32_t i = 0; i < cap; i++) {
         if (janet_checktype(kvs[i].key, JANET_NIL)) continue;
         janet_v_push(ret, janetc_value(subopts, kvs[i].key));
         janet_v_push(ret, janetc_value(subopts, kvs[i].value));
@@ -321,33 +320,46 @@ JanetSlot *janetc_toslotskv(JanetCompiler *c, Janet ds) {
     return ret;
 }
 
-/* Push slots load via janetc_toslots. */
-void janetc_pushslots(JanetCompiler *c, JanetSlot *slots) {
+/* Push slots loaded via janetc_toslots. Return the minimum number of slots pushed,
+ * or -1 - min_arity if there is a splice. (if there is no splice, min_arity is also
+ * the maximum possible arity). */
+int32_t janetc_pushslots(JanetCompiler *c, JanetSlot *slots) {
     int32_t i;
     int32_t count = janet_v_count(slots);
+    int32_t min_arity = 0;
+    int has_splice = 0;
     for (i = 0; i < count;) {
         if (slots[i].flags & JANET_SLOT_SPLICED) {
             janetc_emit_s(c, JOP_PUSH_ARRAY, slots[i], 0);
             i++;
+            has_splice = 1;
         } else if (i + 1 == count) {
             janetc_emit_s(c, JOP_PUSH, slots[i], 0);
             i++;
+            min_arity++;
         } else if (slots[i + 1].flags & JANET_SLOT_SPLICED) {
             janetc_emit_s(c, JOP_PUSH, slots[i], 0);
-            janetc_emit_s(c, JOP_PUSH_ARRAY, slots[i+1], 0);
+            janetc_emit_s(c, JOP_PUSH_ARRAY, slots[i + 1], 0);
             i += 2;
+            min_arity++;
+            has_splice = 1;
         } else if (i + 2 == count) {
-            janetc_emit_ss(c, JOP_PUSH_2, slots[i], slots[i+1], 0);
+            janetc_emit_ss(c, JOP_PUSH_2, slots[i], slots[i + 1], 0);
             i += 2;
+            min_arity += 2;
         } else if (slots[i + 2].flags & JANET_SLOT_SPLICED) {
-            janetc_emit_ss(c, JOP_PUSH_2, slots[i], slots[i+1], 0);
-            janetc_emit_s(c, JOP_PUSH_ARRAY, slots[i+2], 0);
+            janetc_emit_ss(c, JOP_PUSH_2, slots[i], slots[i + 1], 0);
+            janetc_emit_s(c, JOP_PUSH_ARRAY, slots[i + 2], 0);
             i += 3;
+            min_arity += 2;
+            has_splice = 1;
         } else {
-            janetc_emit_sss(c, JOP_PUSH_3, slots[i], slots[i+1], slots[i+2], 0);
+            janetc_emit_sss(c, JOP_PUSH_3, slots[i], slots[i + 1], slots[i + 2], 0);
             i += 3;
+            min_arity += 3;
         }
     }
+    return has_splice ? (-1 - min_arity) : min_arity;
 }
 
 /* Check if a list of slots has any spliced slots */
@@ -404,7 +416,67 @@ static JanetSlot janetc_call(JanetFopts opts, JanetSlot *slots, JanetSlot fun) {
         /* TODO janet function inlining (no c functions)*/
     }
     if (!specialized) {
-        janetc_pushslots(c, slots);
+        int32_t min_arity = janetc_pushslots(c, slots);
+        /* Check for provably incorrect function calls */
+        if (fun.flags & JANET_SLOT_CONSTANT) {
+
+            /* Check for bad arity type if fun is a constant */
+            switch (janet_type(fun.constant)) {
+                case JANET_FUNCTION: {
+                    JanetFunction *f = janet_unwrap_function(fun.constant);
+                    int32_t min = f->def->min_arity;
+                    int32_t max = f->def->max_arity;
+                    if (min_arity < 0) {
+                        /* Call has splices */
+                        min_arity = -1 - min_arity;
+                        if (min_arity > max && max >= 0) {
+                            const uint8_t *es = janet_formatc(
+                                                    "%v expects at most %d argument, got at least %d",
+                                                    fun.constant, max, min_arity);
+                            janetc_error(c, es);
+                        }
+                    } else {
+                        /* Call has no splices */
+                        if (min_arity > max && max >= 0) {
+                            const uint8_t *es = janet_formatc(
+                                                    "%v expects at most %d argument, got %d",
+                                                    fun.constant, max, min_arity);
+                            janetc_error(c, es);
+                        }
+                        if (min_arity < min) {
+                            const uint8_t *es = janet_formatc(
+                                                    "%v expects at least %d argument, got %d",
+                                                    fun.constant, min, min_arity);
+                            janetc_error(c, es);
+                        }
+                    }
+                }
+                break;
+                case JANET_CFUNCTION:
+                case JANET_ABSTRACT:
+                    break;
+                case JANET_KEYWORD:
+                    if (min_arity == 0) {
+                        const uint8_t *es = janet_formatc("%v expects at least 1 argument, got 0",
+                                                          fun.constant);
+                        janetc_error(c, es);
+                    }
+                    break;
+                default:
+                    if (min_arity > 1 || min_arity == 0) {
+                        const uint8_t *es = janet_formatc("%v expects 1 argument, got %d",
+                                                          fun.constant, min_arity);
+                        janetc_error(c, es);
+                    }
+                    if (min_arity < -2) {
+                        const uint8_t *es = janet_formatc("%v expects 1 argument, got at least %d",
+                                                          fun.constant, -1 - min_arity);
+                        janetc_error(c, es);
+                    }
+                    break;
+            }
+        }
+
         if ((opts.flags & JANET_FOPTS_TAIL) &&
                 /* Prevent top level tail calls for better errors */
                 !(c->scope->flags & JANET_SCOPE_TOP)) {
@@ -434,15 +506,23 @@ static JanetSlot janetc_array(JanetFopts opts, Janet x) {
     JanetCompiler *c = opts.compiler;
     JanetArray *a = janet_unwrap_array(x);
     return janetc_maker(opts,
-            janetc_toslots(c, a->data, a->count),
-            JOP_MAKE_ARRAY);
+                        janetc_toslots(c, a->data, a->count),
+                        JOP_MAKE_ARRAY);
+}
+
+static JanetSlot janetc_tuple(JanetFopts opts, Janet x) {
+    JanetCompiler *c = opts.compiler;
+    const Janet *t = janet_unwrap_tuple(x);
+    return janetc_maker(opts,
+                        janetc_toslots(c, t, janet_tuple_length(t)),
+                        JOP_MAKE_TUPLE);
 }
 
 static JanetSlot janetc_tablector(JanetFopts opts, Janet x, int op) {
     JanetCompiler *c = opts.compiler;
     return janetc_maker(opts,
-            janetc_toslotskv(c, x),
-            op);
+                        janetc_toslotskv(c, x),
+                        op);
 }
 
 static JanetSlot janetc_bufferctor(JanetFopts opts, Janet x) {
@@ -450,27 +530,30 @@ static JanetSlot janetc_bufferctor(JanetFopts opts, Janet x) {
     JanetBuffer *b = janet_unwrap_buffer(x);
     Janet onearg = janet_stringv(b->data, b->count);
     return janetc_maker(opts,
-            janetc_toslots(c, &onearg, 1),
-            JOP_MAKE_BUFFER);
+                        janetc_toslots(c, &onearg, 1),
+                        JOP_MAKE_BUFFER);
 }
 
 /* Expand a macro one time. Also get the special form compiler if we
  * find that instead. */
 static int macroexpand1(
-        JanetCompiler *c,
-        Janet x,
-        Janet *out,
-        const JanetSpecial **spec) {
+    JanetCompiler *c,
+    Janet x,
+    Janet *out,
+    const JanetSpecial **spec) {
     if (!janet_checktype(x, JANET_TUPLE))
         return 0;
     const Janet *form = janet_unwrap_tuple(x);
     if (janet_tuple_length(form) == 0)
         return 0;
     /* Source map - only set when we get a tuple */
-    if (janet_tuple_sm_start(form) >= 0) {
-        c->current_mapping.start = janet_tuple_sm_start(form);
-        c->current_mapping.end = janet_tuple_sm_end(form);
+    if (janet_tuple_sm_line(form) >= 0) {
+        c->current_mapping.line = janet_tuple_sm_line(form);
+        c->current_mapping.column = janet_tuple_sm_column(form);
     }
+    /* Bracketed tuples are not specials or macros! */
+    if (janet_tuple_flag(form) & JANET_TUPLE_FLAG_BRACKETCTOR)
+        return 0;
     if (!janet_checktype(form[0], JANET_SYMBOL))
         return 0;
     const uint8_t *name = janet_unwrap_symbol(form[0]);
@@ -486,15 +569,24 @@ static int macroexpand1(
         return 0;
 
     /* Evaluate macro */
-    JanetFiber *fiberp;
     JanetFunction *macro = janet_unwrap_function(macroval);
+    int32_t arity = janet_tuple_length(form) - 1;
+    JanetFiber *fiberp = janet_fiber(macro, 64, arity, form + 1);
+    if (NULL == fiberp) {
+        int32_t minar = macro->def->min_arity;
+        int32_t maxar = macro->def->max_arity;
+        const uint8_t *es = NULL;
+        if (minar >= 0 && arity < minar)
+            es = janet_formatc("macro arity mismatch, expected at least %d, got %d", minar, arity);
+        if (maxar >= 0 && arity > maxar)
+            es = janet_formatc("macro arity mismatch, expected at most %d, got %d", maxar, arity);
+        c->result.macrofiber = NULL;
+        janetc_error(c, es);
+    }
+    /* Set env */
+    fiberp->env = c->env;
     int lock = janet_gclock();
-    JanetSignal status = janet_pcall(
-            macro,
-            janet_tuple_length(form) - 1,
-            form + 1,
-            &x,
-            &fiberp);
+    JanetSignal status = janet_continue(fiberp, janet_wrap_nil(), &x);
     janet_gcunlock(lock);
     if (status != JANET_SIGNAL_OK) {
         const uint8_t *es = janet_formatc("(macro) %V", x);
@@ -540,22 +632,23 @@ JanetSlot janetc_value(JanetFopts opts, Janet x) {
         ret = spec->compile(opts, janet_tuple_length(tup) - 1, tup + 1);
     } else {
         switch (janet_type(x)) {
-            case JANET_TUPLE:
-                {
-                    JanetFopts subopts = janetc_fopts_default(c);
-                    const Janet *tup = janet_unwrap_tuple(x);
-                    /* Empty tuple is tuple literal */
-                    if (janet_tuple_length(tup) == 0) {
-                        ret = janetc_cslot(x);
-                    } else {
-                        JanetSlot head = janetc_value(subopts, tup[0]);
-                        subopts.flags = JANET_FUNCTION | JANET_CFUNCTION;
-                        ret = janetc_call(opts, janetc_toslots(c, tup + 1, janet_tuple_length(tup) - 1), head);
-                        janetc_freeslot(c, head);
-                    }
-                    ret.flags &= ~JANET_SLOT_SPLICED;
+            case JANET_TUPLE: {
+                JanetFopts subopts = janetc_fopts_default(c);
+                const Janet *tup = janet_unwrap_tuple(x);
+                /* Empty tuple is tuple literal */
+                if (janet_tuple_length(tup) == 0) {
+                    ret = janetc_cslot(janet_wrap_tuple(janet_tuple_n(NULL, 0)));
+                } else if (janet_tuple_flag(tup) & JANET_TUPLE_FLAG_BRACKETCTOR) { /* [] tuples are not function call */
+                    ret = janetc_tuple(opts, x);
+                } else {
+                    JanetSlot head = janetc_value(subopts, tup[0]);
+                    subopts.flags = JANET_FUNCTION | JANET_CFUNCTION;
+                    ret = janetc_call(opts, janetc_toslots(c, tup + 1, janet_tuple_length(tup) - 1), head);
+                    janetc_freeslot(c, head);
                 }
-                break;
+                ret.flags &= ~JANET_SLOT_SPLICED;
+            }
+            break;
             case JANET_SYMBOL:
                 ret = janetc_resolve(c, janet_unwrap_symbol(x));
                 break;
@@ -618,7 +711,7 @@ JanetFuncDef *janetc_pop_funcdef(JanetCompiler *c) {
         }
         memcpy(def->bytecode, c->buffer + scope->bytecode_start, s);
         janet_v__cnt(c->buffer) = scope->bytecode_start;
-        if (NULL != c->mapbuffer) {
+        if (NULL != c->mapbuffer && c->source) {
             size_t s = sizeof(JanetSourceMapping) * def->bytecode_length;
             def->sourcemap = malloc(s);
             if (NULL == def->sourcemap) {
@@ -633,6 +726,7 @@ JanetFuncDef *janetc_pop_funcdef(JanetCompiler *c) {
     def->source = c->source;
 
     def->arity = 0;
+    def->min_arity = 0;
     def->flags = 0;
     if (scope->flags & JANET_SCOPE_ENV) {
         def->flags |= JANET_FUNCDEF_FLAG_NEEDSENV;
@@ -652,15 +746,15 @@ static void janetc_init(JanetCompiler *c, JanetTable *env, const uint8_t *where)
     c->recursion_guard = JANET_RECURSION_GUARD;
     c->env = env;
     c->source = where;
-    c->current_mapping.start = -1;
-    c->current_mapping.end = -1;
+    c->current_mapping.line = -1;
+    c->current_mapping.column = -1;
     /* Init result */
     c->result.error = NULL;
     c->result.status = JANET_COMPILE_OK;
     c->result.funcdef = NULL;
     c->result.macrofiber = NULL;
-    c->result.error_mapping.start = -1;
-    c->result.error_mapping.end = -1;
+    c->result.error_mapping.line = -1;
+    c->result.error_mapping.column = -1;
 }
 
 /* Deinitialize a compiler struct */
@@ -705,8 +799,12 @@ JanetCompileResult janet_compile(Janet source, JanetTable *env, const uint8_t *w
 
 /* C Function for compiling */
 static Janet cfun(int32_t argc, Janet *argv) {
-    janet_arity(argc, 2, 3);
-    JanetTable *env = janet_gettable(argv, 1);
+    janet_arity(argc, 1, 3);
+    JanetTable *env = argc > 1 ? janet_gettable(argv, 1) : janet_vm_fiber->env;
+    if (NULL == env) {
+        env = janet_table(0);
+        janet_vm_fiber->env = env;
+    }
     const uint8_t *source = NULL;
     if (argc == 3) {
         source = janet_getstring(argv, 2);
@@ -717,8 +815,8 @@ static Janet cfun(int32_t argc, Janet *argv) {
     } else {
         JanetTable *t = janet_table(4);
         janet_table_put(t, janet_ckeywordv("error"), janet_wrap_string(res.error));
-        janet_table_put(t, janet_ckeywordv("start"), janet_wrap_integer(res.error_mapping.start));
-        janet_table_put(t, janet_ckeywordv("end"), janet_wrap_integer(res.error_mapping.end));
+        janet_table_put(t, janet_ckeywordv("line"), janet_wrap_integer(res.error_mapping.line));
+        janet_table_put(t, janet_ckeywordv("column"), janet_wrap_integer(res.error_mapping.column));
         if (res.macrofiber) {
             janet_table_put(t, janet_ckeywordv("fiber"), janet_wrap_fiber(res.macrofiber));
         }
@@ -727,16 +825,17 @@ static Janet cfun(int32_t argc, Janet *argv) {
 }
 
 static const JanetReg compile_cfuns[] = {
-    {"compile", cfun,
-        JDOC("(compile ast env [, source])\n\n"
-                "Compiles an Abstract Syntax Tree (ast) into a janet function. "
-                "Pair the compile function with parsing functionality to implement "
-                "eval. Returns a janet function and does not modify ast. Throws an "
-                "error if the ast cannot be compiled.")
+    {
+        "compile", cfun,
+        JDOC("(compile ast &opt env source)\n\n"
+             "Compiles an Abstract Syntax Tree (ast) into a janet function. "
+             "Pair the compile function with parsing functionality to implement "
+             "eval. Returns a janet function and does not modify ast. Throws an "
+             "error if the ast cannot be compiled.")
     },
     {NULL, NULL, NULL}
 };
 
 void janet_lib_compile(JanetTable *env) {
-    janet_cfuns(env, NULL, compile_cfuns);
+    janet_core_cfuns(env, NULL, compile_cfuns);
 }

src/core/compile.h

@@ -24,7 +24,7 @@
 #define JANET_COMPILE_H
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "regalloc.h"
 #endif
 
@@ -34,7 +34,7 @@
 #define JANET_FUN_APPLY 3
 #define JANET_FUN_YIELD 4
 #define JANET_FUN_RESUME 5
-#define JANET_FUN_GET 6
+#define JANET_FUN_IN 6
 #define JANET_FUN_PUT 7
 #define JANET_FUN_LENGTH 8
 #define JANET_FUN_ADD 9
@@ -60,6 +60,8 @@
 #define JANET_FUN_LTE 29
 #define JANET_FUN_EQ 30
 #define JANET_FUN_NEQ 31
+#define JANET_FUN_PROP 32
+#define JANET_FUN_GET 33
 
 /* Compiler typedefs */
 typedef struct JanetCompiler JanetCompiler;
@@ -96,6 +98,7 @@ struct JanetSlot {
 #define JANET_SCOPE_TOP 4
 #define JANET_SCOPE_UNUSED 8
 #define JANET_SCOPE_CLOSURE 16
+#define JANET_SCOPE_WHILE 32
 
 /* A symbol and slot pair */
 typedef struct SymPair {
@@ -131,9 +134,6 @@ struct JanetScope {
      * that corresponds to the direct parent's stack will always have value 0. */
     int32_t *envs;
 
-    /* Where to add reference to self in constants */
-    int32_t selfconst;
-
     int32_t bytecode_start;
     int flags;
 };
@@ -180,13 +180,13 @@ JanetFopts janetc_fopts_default(JanetCompiler *c);
 /* For optimizing builtin normal functions. */
 struct JanetFunOptimizer {
     int (*can_optimize)(JanetFopts opts, JanetSlot *args);
-    JanetSlot (*optimize)(JanetFopts opts, JanetSlot *args);
+    JanetSlot(*optimize)(JanetFopts opts, JanetSlot *args);
 };
 
 /* A grouping of a named special and the corresponding compiler fragment */
 struct JanetSpecial {
     const char *name;
-    JanetSlot (*compile)(JanetFopts opts, int32_t argn, const Janet *argv);
+    JanetSlot(*compile)(JanetFopts opts, int32_t argn, const Janet *argv);
 };
 
 /****************************************************/
@@ -215,7 +215,7 @@ JanetSlot *janetc_toslots(JanetCompiler *c, const Janet *vals, int32_t len);
 JanetSlot *janetc_toslotskv(JanetCompiler *c, Janet ds);
 
 /* Push slots load via janetc_toslots. */
-void janetc_pushslots(JanetCompiler *c, JanetSlot *slots);
+int32_t janetc_pushslots(JanetCompiler *c, JanetSlot *slots);
 
 /* Free slots loaded via janetc_toslots */
 void janetc_freeslots(JanetCompiler *c, JanetSlot *slots);

src/core/debug.c

@@ -21,10 +21,11 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "gc.h"
 #include "state.h"
 #include "util.h"
+#include "vector.h"
 #endif
 
 /* Implements functionality to build a debugger from within janet.
@@ -50,32 +51,36 @@ void janet_debug_unbreak(JanetFuncDef *def, int32_t pc) {
  * location.
  */
 void janet_debug_find(
-        JanetFuncDef **def_out, int32_t *pc_out,
-        const uint8_t *source, int32_t offset) {
+    JanetFuncDef **def_out, int32_t *pc_out,
+    const uint8_t *source, int32_t sourceLine, int32_t sourceColumn) {
     /* Scan the heap for right func def */
-    JanetGCMemoryHeader *current = janet_vm_blocks;
+    JanetGCObject *current = janet_vm_blocks;
     /* Keep track of the best source mapping we have seen so far */
     int32_t besti = -1;
-    int32_t best_range = INT32_MAX;
+    int32_t best_line = -1;
+    int32_t best_column = -1;
     JanetFuncDef *best_def = NULL;
     while (NULL != current) {
         if ((current->flags & JANET_MEM_TYPEBITS) == JANET_MEMORY_FUNCDEF) {
-            JanetFuncDef *def = (JanetFuncDef *)(current + 1);
+            JanetFuncDef *def = (JanetFuncDef *)(current);
             if (def->sourcemap &&
                     def->source &&
                     !janet_string_compare(source, def->source)) {
                 /* Correct source file, check mappings. The chosen
-                 * pc index is the first match with the smallest range. */
+                 * pc index is the instruction closest to the given line column, but
+                 * not after. */
                 int32_t i;
                 for (i = 0; i < def->bytecode_length; i++) {
-                    int32_t start = def->sourcemap[i].start;
-                    int32_t end = def->sourcemap[i].end;
-                    if (end - start < best_range &&
-                            start <= offset &&
-                            end >= offset) {
-                        best_range = end - start;
-                        besti = i;
-                        best_def = def;
+                    int32_t line = def->sourcemap[i].line;
+                    int32_t column = def->sourcemap[i].column;
+                    if (line <= sourceLine && line >= best_line) {
+                        if (column <= sourceColumn &&
+                                (line > best_line || column > best_column)) {
+                            best_line = line;
+                            best_column = column;
+                            besti = i;
+                            best_def = def;
+                        }
                     }
                 }
             }
@@ -90,6 +95,79 @@ void janet_debug_find(
     }
 }
 
+/* Error reporting. This can be emulated from within Janet, but for
+ * consitency with the top level code it is defined once. */
+void janet_stacktrace(JanetFiber *fiber, Janet err) {
+    int32_t fi;
+    const char *errstr = (const char *)janet_to_string(err);
+    JanetFiber **fibers = NULL;
+    int wrote_error = 0;
+
+    int print_color = janet_truthy(janet_dyn("err-color"));
+    if (print_color) janet_eprintf("\x1b[31m");
+
+    while (fiber) {
+        janet_v_push(fibers, fiber);
+        fiber = fiber->child;
+    }
+
+    for (fi = janet_v_count(fibers) - 1; fi >= 0; fi--) {
+        fiber = fibers[fi];
+        int32_t i = fiber->frame;
+        while (i > 0) {
+            JanetStackFrame *frame = (JanetStackFrame *)(fiber->data + i - JANET_FRAME_SIZE);
+            JanetFuncDef *def = NULL;
+            i = frame->prevframe;
+
+            /* Print prelude to stack frame */
+            if (!wrote_error) {
+                JanetFiberStatus status = janet_fiber_status(fiber);
+                const char *prefix = status == JANET_STATUS_ERROR ? "" : "status ";
+                janet_eprintf("%s%s: %s\n",
+                              prefix,
+                              janet_status_names[status],
+                              errstr);
+                wrote_error = 1;
+            }
+
+            janet_eprintf("  in");
+
+            if (frame->func) {
+                def = frame->func->def;
+                janet_eprintf(" %s", def->name ? (const char *)def->name : "<anonymous>");
+                if (def->source) {
+                    janet_eprintf(" [%s]", (const char *)def->source);
+                }
+            } else {
+                JanetCFunction cfun = (JanetCFunction)(frame->pc);
+                if (cfun) {
+                    Janet name = janet_table_get(janet_vm_registry, janet_wrap_cfunction(cfun));
+                    if (!janet_checktype(name, JANET_NIL))
+                        janet_eprintf(" %s", (const char *)janet_to_string(name));
+                    else
+                        janet_eprintf(" <cfunction>");
+                }
+            }
+            if (frame->flags & JANET_STACKFRAME_TAILCALL)
+                janet_eprintf(" (tailcall)");
+            if (frame->func && frame->pc) {
+                int32_t off = (int32_t)(frame->pc - def->bytecode);
+                if (def->sourcemap) {
+                    JanetSourceMapping mapping = def->sourcemap[off];
+                    janet_eprintf(" on line %d, column %d", mapping.line, mapping.column);
+                } else {
+                    janet_eprintf(" pc=%d", off);
+                }
+            }
+            janet_eprintf("\n");
+        }
+    }
+
+    if (print_color) janet_eprintf("\x1b[0m");
+
+    janet_v_free(fibers);
+}
+
 /*
  * CFuns
  */
@@ -97,10 +175,11 @@ void janet_debug_find(
 /* Helper to find funcdef and bytecode offset to insert or remove breakpoints.
  * Takes a source file name and byte offset. */
 static void helper_find(int32_t argc, Janet *argv, JanetFuncDef **def, int32_t *bytecode_offset) {
-    janet_fixarity(argc, 2);
+    janet_fixarity(argc, 3);
     const uint8_t *source = janet_getstring(argv, 0);
-    int32_t source_offset = janet_getinteger(argv, 1);
-    janet_debug_find(def, bytecode_offset, source, source_offset);
+    int32_t line = janet_getinteger(argv, 1);
+    int32_t col = janet_getinteger(argv, 2);
+    janet_debug_find(def, bytecode_offset, source, line, col);
 }
 
 /* Helper to find funcdef and bytecode offset to insert or remove breakpoints.
@@ -123,7 +202,7 @@ static Janet cfun_debug_break(int32_t argc, Janet *argv) {
 
 static Janet cfun_debug_unbreak(int32_t argc, Janet *argv) {
     JanetFuncDef *def;
-    int32_t offset;
+    int32_t offset = 0;
     helper_find(argc, argv, &def, &offset);
     janet_debug_unbreak(def, offset);
     return janet_wrap_nil();
@@ -131,7 +210,7 @@ static Janet cfun_debug_unbreak(int32_t argc, Janet *argv) {
 
 static Janet cfun_debug_fbreak(int32_t argc, Janet *argv) {
     JanetFuncDef *def;
-    int32_t offset;
+    int32_t offset = 0;
     helper_find_fun(argc, argv, &def, &offset);
     janet_debug_break(def, offset);
     return janet_wrap_nil();
@@ -183,12 +262,12 @@ static Janet doframe(JanetStackFrame *frame) {
     if (frame->func && frame->pc) {
         Janet *stack = (Janet *)frame + JANET_FRAME_SIZE;
         JanetArray *slots;
-        off = (int32_t) (frame->pc - def->bytecode);
+        off = (int32_t)(frame->pc - def->bytecode);
         janet_table_put(t, janet_ckeywordv("pc"), janet_wrap_integer(off));
         if (def->sourcemap) {
             JanetSourceMapping mapping = def->sourcemap[off];
-            janet_table_put(t, janet_ckeywordv("source-start"), janet_wrap_integer(mapping.start));
-            janet_table_put(t, janet_ckeywordv("source-end"), janet_wrap_integer(mapping.end));
+            janet_table_put(t, janet_ckeywordv("source-line"), janet_wrap_integer(mapping.line));
+            janet_table_put(t, janet_ckeywordv("source-column"), janet_wrap_integer(mapping.column));
         }
         if (def->source) {
             janet_table_put(t, janet_ckeywordv("source"), janet_wrap_string(def->source));
@@ -218,6 +297,13 @@ static Janet cfun_debug_stack(int32_t argc, Janet *argv) {
     return janet_wrap_array(array);
 }
 
+static Janet cfun_debug_stacktrace(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 2);
+    JanetFiber *fiber = janet_getfiber(argv, 0);
+    janet_stacktrace(fiber, argv[1]);
+    return argv[0];
+}
+
 static Janet cfun_debug_argstack(int32_t argc, Janet *argv) {
     janet_fixarity(argc, 1);
     JanetFiber *fiber = janet_getfiber(argv, 0);
@@ -227,71 +313,93 @@ static Janet cfun_debug_argstack(int32_t argc, Janet *argv) {
     return janet_wrap_array(array);
 }
 
+static Janet cfun_debug_step(int32_t argc, Janet *argv) {
+    janet_arity(argc, 1, 2);
+    JanetFiber *fiber = janet_getfiber(argv, 0);
+    Janet out = janet_wrap_nil();
+    janet_step(fiber, argc == 1 ? janet_wrap_nil() : argv[1], &out);
+    return out;
+}
+
 static const JanetReg debug_cfuns[] = {
     {
         "debug/break", cfun_debug_break,
         JDOC("(debug/break source byte-offset)\n\n"
-                "Sets a breakpoint with source a key at a given byte offset. An offset "
-                "of 0 is the first byte in a file. Will throw an error if the breakpoint location "
-                "cannot be found. For example\n\n"
-                "\t(debug/break \"core.janet\" 1000)\n\n"
-                "wil set a breakpoint at the 1000th byte of the file core.janet.")
+             "Sets a breakpoint with source a key at a given line and column. "
+             "Will throw an error if the breakpoint location "
+             "cannot be found. For example\n\n"
+             "\t(debug/break \"core.janet\" 1000)\n\n"
+             "wil set a breakpoint at the 1000th byte of the file core.janet.")
     },
     {
         "debug/unbreak", cfun_debug_unbreak,
-        JDOC("(debug/unbreak source byte-offset)\n\n"
-                "Remove a breakpoint with a source key at a given byte offset. An offset "
-                "of 0 is the first byte in a file. Will throw an error if the breakpoint "
-                "cannot be found.")
+        JDOC("(debug/unbreak source line column)\n\n"
+             "Remove a breakpoint with a source key at a given line and column. "
+             "Will throw an error if the breakpoint "
+             "cannot be found.")
     },
     {
         "debug/fbreak", cfun_debug_fbreak,
-        JDOC("(debug/fbreak fun [,pc=0])\n\n"
-                "Set a breakpoint in a given function. pc is an optional offset, which "
-                "is in bytecode instructions. fun is a function value. Will throw an error "
-                "if the offset is too large or negative.")
+        JDOC("(debug/fbreak fun &opt pc)\n\n"
+             "Set a breakpoint in a given function. pc is an optional offset, which "
+             "is in bytecode instructions. fun is a function value. Will throw an error "
+             "if the offset is too large or negative.")
     },
     {
         "debug/unfbreak", cfun_debug_unfbreak,
-        JDOC("(debug/unfbreak fun [,pc=0])\n\n"
-                "Unset a breakpoint set with debug/fbreak.")
+        JDOC("(debug/unfbreak fun &opt pc)\n\n"
+             "Unset a breakpoint set with debug/fbreak.")
     },
     {
         "debug/arg-stack", cfun_debug_argstack,
         JDOC("(debug/arg-stack fiber)\n\n"
-                "Gets all values currently on the fiber's argument stack. Normally, "
-                "this should be empty unless the fiber signals while pushing arguments "
-                "to make a function call. Returns a new array.")
+             "Gets all values currently on the fiber's argument stack. Normally, "
+             "this should be empty unless the fiber signals while pushing arguments "
+             "to make a function call. Returns a new array.")
     },
     {
         "debug/stack", cfun_debug_stack,
         JDOC("(debug/stack fib)\n\n"
-                "Gets information about the stack as an array of tables. Each table "
-                "in the array contains information about a stack frame. The top most, current "
-                "stack frame is the first table in the array, and the bottom most stack frame "
-                "is the last value. Each stack frame contains some of the following attributes:\n\n"
-                "\t:c - true if the stack frame is a c function invocation\n"
-                "\t:column - the current source column of the stack frame\n"
-                "\t:function - the function that the stack frame represents\n"
-                "\t:line - the current source line of the stack frame\n"
-                "\t:name - the human friendly name of the function\n"
-                "\t:pc - integer indicating the location of the program counter\n"
-                "\t:source - string with the file path or other identifier for the source code\n"
-                "\t:slots - array of all values in each slot\n"
-                "\t:tail - boolean indicating a tail call")
+             "Gets information about the stack as an array of tables. Each table "
+             "in the array contains information about a stack frame. The top most, current "
+             "stack frame is the first table in the array, and the bottom most stack frame "
+             "is the last value. Each stack frame contains some of the following attributes:\n\n"
+             "\t:c - true if the stack frame is a c function invocation\n"
+             "\t:column - the current source column of the stack frame\n"
+             "\t:function - the function that the stack frame represents\n"
+             "\t:line - the current source line of the stack frame\n"
+             "\t:name - the human friendly name of the function\n"
+             "\t:pc - integer indicating the location of the program counter\n"
+             "\t:source - string with the file path or other identifier for the source code\n"
+             "\t:slots - array of all values in each slot\n"
+             "\t:tail - boolean indicating a tail call")
+    },
+    {
+        "debug/stacktrace", cfun_debug_stacktrace,
+        JDOC("(debug/stacktrace fiber err)\n\n"
+             "Prints a nice looking stacktrace for a fiber. The error message "
+             "err must be passed to the function as fiber's do not keep track of "
+             "the last error they have thrown. Returns the fiber.")
     },
     {
         "debug/lineage", cfun_debug_lineage,
         JDOC("(debug/lineage fib)\n\n"
-                "Returns an array of all child fibers from a root fiber. This function "
-                "is useful when a fiber signals or errors to an ancestor fiber. Using this function, "
-                "the fiber handling the error can see which fiber raised the signal. This function should "
-                "be used mostly for debugging purposes.")
+             "Returns an array of all child fibers from a root fiber. This function "
+             "is useful when a fiber signals or errors to an ancestor fiber. Using this function, "
+             "the fiber handling the error can see which fiber raised the signal. This function should "
+             "be used mostly for debugging purposes.")
+    },
+    {
+        "debug/step", cfun_debug_step,
+        JDOC("(debug/step fiber &opt x)\n\n"
+             "Run a fiber for one virtual instruction of the Janet machine. Can optionally "
+             "pass in a value that will be passed as the resuming value. Returns the signal value, "
+             "which will usually be nil, as breakpoints raise nil signals.")
     },
     {NULL, NULL, NULL}
 };
 
 /* Module entry point */
 void janet_lib_debug(JanetTable *env) {
-    janet_cfuns(env, NULL, debug_cfuns);
+    janet_core_cfuns(env, NULL, debug_cfuns);
 }

src/core/emit.c

@@ -21,7 +21,7 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "emit.h"
 #include "vector.h"
 #include "regalloc.h"
@@ -78,32 +78,31 @@ static void janetc_loadconst(JanetCompiler *c, Janet k, int32_t reg) {
         case JANET_NIL:
             janetc_emit(c, (reg << 8) | JOP_LOAD_NIL);
             break;
-        case JANET_TRUE:
-            janetc_emit(c, (reg << 8) | JOP_LOAD_TRUE);
+        case JANET_BOOLEAN:
+            janetc_emit(c, (reg << 8) |
+                        (janet_unwrap_boolean(k) ? JOP_LOAD_TRUE : JOP_LOAD_FALSE));
             break;
-        case JANET_FALSE:
-            janetc_emit(c, (reg << 8) | JOP_LOAD_FALSE);
-            break;
-        case JANET_NUMBER:
-            {
-                double dval = janet_unwrap_number(k);
-                int32_t i = (int32_t) dval;
-                if (dval != i || !(dval >= INT16_MIN && dval <= INT16_MAX))
-                    goto do_constant;
-                janetc_emit(c,
-                        (i << 16) |
+        case JANET_NUMBER: {
+            double dval = janet_unwrap_number(k);
+            if (dval < INT16_MIN || dval > INT16_MAX)
+                goto do_constant;
+            int32_t i = (int32_t) dval;
+            if (dval != i)
+                goto do_constant;
+            uint32_t iu = (uint32_t)i;
+            janetc_emit(c,
+                        (iu << 16) |
                         (reg << 8) |
                         JOP_LOAD_INTEGER);
-                break;
-            }
+            break;
+        }
         default:
-        do_constant:
-            {
+        do_constant: {
                 int32_t cindex = janetc_const(c, k);
                 janetc_emit(c,
-                        (cindex << 16) |
-                        (reg << 8) |
-                        JOP_LOAD_CONSTANT);
+                            (cindex << 16) |
+                            (reg << 8) |
+                            JOP_LOAD_CONSTANT);
                 break;
             }
     }
@@ -111,53 +110,53 @@ static void janetc_loadconst(JanetCompiler *c, Janet k, int32_t reg) {
 
 /* Move a slot to a near register */
 static void janetc_movenear(JanetCompiler *c,
-        int32_t dest,
-        JanetSlot src) {
+                            int32_t dest,
+                            JanetSlot src) {
     if (src.flags & (JANET_SLOT_CONSTANT | JANET_SLOT_REF)) {
         janetc_loadconst(c, src.constant, dest);
         /* If we also are a reference, deref the one element array */
         if (src.flags & JANET_SLOT_REF) {
             janetc_emit(c,
-                    (dest << 16) |
-                    (dest << 8) |
-                    JOP_GET_INDEX);
+                        (dest << 16) |
+                        (dest << 8) |
+                        JOP_GET_INDEX);
         }
     } else if (src.envindex >= 0) {
         janetc_emit(c,
-                ((uint32_t)(src.index) << 24) |
-                ((uint32_t)(src.envindex) << 16) |
-                ((uint32_t)(dest) << 8) |
-                JOP_LOAD_UPVALUE);
+                    ((uint32_t)(src.index) << 24) |
+                    ((uint32_t)(src.envindex) << 16) |
+                    ((uint32_t)(dest) << 8) |
+                    JOP_LOAD_UPVALUE);
     } else if (src.index > 0xFF || src.index != dest) {
         janetc_emit(c,
-                ((uint32_t)(src.index) << 16) |
-                ((uint32_t)(dest) << 8) |
+                    ((uint32_t)(src.index) << 16) |
+                    ((uint32_t)(dest) << 8) |
                     JOP_MOVE_NEAR);
     }
 }
 
 /* Move a near register to a Slot. */
 static void janetc_moveback(JanetCompiler *c,
-        JanetSlot dest,
-        int32_t src) {
+                            JanetSlot dest,
+                            int32_t src) {
     if (dest.flags & JANET_SLOT_REF) {
         int32_t refreg = janetc_regalloc_temp(&c->scope->ra, JANETC_REGTEMP_5);
         janetc_loadconst(c, dest.constant, refreg);
         janetc_emit(c,
-               (src << 16) |
-               (refreg << 8) |
-               JOP_PUT_INDEX);
+                    (src << 16) |
+                    (refreg << 8) |
+                    JOP_PUT_INDEX);
         janetc_regalloc_freetemp(&c->scope->ra, refreg, JANETC_REGTEMP_5);
     } else if (dest.envindex >= 0) {
         janetc_emit(c,
-                ((uint32_t)(dest.index) << 24) |
-                ((uint32_t)(dest.envindex) << 16) |
-                ((uint32_t)(src) << 8) |
-                JOP_SET_UPVALUE);
+                    ((uint32_t)(dest.index) << 24) |
+                    ((uint32_t)(dest.envindex) << 16) |
+                    ((uint32_t)(src) << 8) |
+                    JOP_SET_UPVALUE);
     } else if (dest.index != src) {
         janetc_emit(c,
-                ((uint32_t)(dest.index) << 16) |
-                ((uint32_t)(src) << 8) |
+                    ((uint32_t)(dest.index) << 16) |
+                    ((uint32_t)(src) << 8) |
                     JOP_MOVE_FAR);
     }
 }
@@ -221,9 +220,9 @@ static int janetc_sequal(JanetSlot lhs, JanetSlot rhs) {
 /* Move values from one slot to another. The destination must
  * be writeable (not a literal). */
 void janetc_copy(
-        JanetCompiler *c,
-        JanetSlot dest,
-        JanetSlot src) {
+    JanetCompiler *c,
+    JanetSlot dest,
+    JanetSlot src) {
     if (dest.flags & JANET_SLOT_CONSTANT) {
         janetc_cerror(c, "cannot write to constant");
         return;
@@ -240,11 +239,11 @@ void janetc_copy(
         return;
     }
     /* Process: src -> near -> dest */
-    int32_t near = janetc_allocnear(c, JANETC_REGTEMP_3);
-    janetc_movenear(c, near, src);
-    janetc_moveback(c, dest, near);
+    int32_t nearreg = janetc_allocnear(c, JANETC_REGTEMP_3);
+    janetc_movenear(c, nearreg, src);
+    janetc_moveback(c, dest, nearreg);
     /* Cleanup */
-    janetc_regalloc_freetemp(&c->scope->ra, near, JANETC_REGTEMP_3);
+    janetc_regalloc_freetemp(&c->scope->ra, nearreg, JANETC_REGTEMP_3);
 
 }
 /* Instruction templated emitters */
@@ -252,7 +251,7 @@ void janetc_copy(
 static int32_t emit1s(JanetCompiler *c, uint8_t op, JanetSlot s, int32_t rest, int wr) {
     int32_t reg = janetc_regnear(c, s, JANETC_REGTEMP_0);
     int32_t label = janet_v_count(c->buffer);
-    janetc_emit(c, op | (reg << 8) | (rest << 16));
+    janetc_emit(c, op | (reg << 8) | ((uint32_t)rest << 16));
     if (wr)
         janetc_moveback(c, s, reg);
     janetc_free_regnear(c, s, reg, JANETC_REGTEMP_0);
@@ -294,7 +293,7 @@ static int32_t emit2s(JanetCompiler *c, uint8_t op, JanetSlot s1, JanetSlot s2,
     int32_t reg1 = janetc_regnear(c, s1, JANETC_REGTEMP_0);
     int32_t reg2 = janetc_regnear(c, s2, JANETC_REGTEMP_1);
     int32_t label = janet_v_count(c->buffer);
-    janetc_emit(c, op | (reg1 << 8) | (reg2 << 16) | (rest << 24));
+    janetc_emit(c, op | (reg1 << 8) | (reg2 << 16) | ((uint32_t)rest << 24));
     janetc_free_regnear(c, s2, reg2, JANETC_REGTEMP_1);
     if (wr)
         janetc_moveback(c, s1, reg1);
@@ -327,7 +326,7 @@ int32_t janetc_emit_sss(JanetCompiler *c, uint8_t op, JanetSlot s1, JanetSlot s2
     int32_t reg2 = janetc_regnear(c, s2, JANETC_REGTEMP_1);
     int32_t reg3 = janetc_regnear(c, s3, JANETC_REGTEMP_2);
     int32_t label = janet_v_count(c->buffer);
-    janetc_emit(c, op | (reg1 << 8) | (reg2 << 16) | (reg3 << 24));
+    janetc_emit(c, op | (reg1 << 8) | (reg2 << 16) | ((uint32_t)reg3 << 24));
     janetc_free_regnear(c, s2, reg2, JANETC_REGTEMP_1);
     janetc_free_regnear(c, s3, reg3, JANETC_REGTEMP_2);
     if (wr)

src/core/fiber.c

@@ -21,7 +21,7 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "fiber.h"
 #include "state.h"
 #include "gc.h"
@@ -35,6 +35,7 @@ static void fiber_reset(JanetFiber *fiber) {
     fiber->stacktop = JANET_FRAME_SIZE;
     fiber->child = NULL;
     fiber->flags = JANET_FIBER_MASK_YIELD;
+    fiber->env = NULL;
     janet_fiber_set_status(fiber, JANET_STATUS_NEW);
 }
 
@@ -49,6 +50,7 @@ static JanetFiber *fiber_alloc(int32_t capacity) {
     if (NULL == data) {
         JANET_OUT_OF_MEMORY;
     }
+    janet_vm_next_collection += sizeof(Janet) * capacity;
     fiber->data = data;
     return fiber;
 }
@@ -66,6 +68,7 @@ JanetFiber *janet_fiber_reset(JanetFiber *fiber, JanetFunction *callee, int32_t
         fiber->stacktop = newstacktop;
     }
     if (janet_fiber_funcframe(fiber, callee)) return NULL;
+    janet_fiber_frame(fiber)->flags |= JANET_STACKFRAME_ENTRANCE;
     return fiber;
 }
 
@@ -84,19 +87,27 @@ void janet_fiber_setcapacity(JanetFiber *fiber, int32_t n) {
     fiber->capacity = n;
 }
 
+/* Grow fiber if needed */
+static void janet_fiber_grow(JanetFiber *fiber, int32_t needed) {
+    int32_t cap = needed > (INT32_MAX / 2) ? INT32_MAX : 2 * needed;
+    janet_fiber_setcapacity(fiber, cap);
+}
+
 /* Push a value on the next stack frame */
 void janet_fiber_push(JanetFiber *fiber, Janet x) {
+    if (fiber->stacktop == INT32_MAX) janet_panic("stack overflow");
     if (fiber->stacktop >= fiber->capacity) {
-        janet_fiber_setcapacity(fiber, 2 * fiber->stacktop);
+        janet_fiber_grow(fiber, fiber->stacktop);
     }
     fiber->data[fiber->stacktop++] = x;
 }
 
 /* Push 2 values on the next stack frame */
 void janet_fiber_push2(JanetFiber *fiber, Janet x, Janet y) {
+    if (fiber->stacktop >= INT32_MAX - 1) janet_panic("stack overflow");
     int32_t newtop = fiber->stacktop + 2;
     if (newtop > fiber->capacity) {
-        janet_fiber_setcapacity(fiber, 2 * newtop);
+        janet_fiber_grow(fiber, newtop);
     }
     fiber->data[fiber->stacktop] = x;
     fiber->data[fiber->stacktop + 1] = y;
@@ -105,9 +116,10 @@ void janet_fiber_push2(JanetFiber *fiber, Janet x, Janet y) {
 
 /* Push 3 values on the next stack frame */
 void janet_fiber_push3(JanetFiber *fiber, Janet x, Janet y, Janet z) {
+    if (fiber->stacktop >= INT32_MAX - 2) janet_panic("stack overflow");
     int32_t newtop = fiber->stacktop + 3;
     if (newtop > fiber->capacity) {
-        janet_fiber_setcapacity(fiber, 2 * newtop);
+        janet_fiber_grow(fiber, newtop);
     }
     fiber->data[fiber->stacktop] = x;
     fiber->data[fiber->stacktop + 1] = y;
@@ -117,14 +129,25 @@ void janet_fiber_push3(JanetFiber *fiber, Janet x, Janet y, Janet z) {
 
 /* Push an array on the next stack frame */
 void janet_fiber_pushn(JanetFiber *fiber, const Janet *arr, int32_t n) {
+    if (fiber->stacktop > INT32_MAX - n) janet_panic("stack overflow");
     int32_t newtop = fiber->stacktop + n;
     if (newtop > fiber->capacity) {
-        janet_fiber_setcapacity(fiber, 2 * newtop);
+        janet_fiber_grow(fiber, newtop);
     }
     memcpy(fiber->data + fiber->stacktop, arr, n * sizeof(Janet));
     fiber->stacktop = newtop;
 }
 
+/* Create a struct with n values. If n is odd, the last value is ignored. */
+static Janet make_struct_n(const Janet *args, int32_t n) {
+    int32_t i = 0;
+    JanetKV *st = janet_struct_begin(n & (~1));
+    for (; i < n; i += 2) {
+        janet_struct_put(st, args[i], args[i + 1]);
+    }
+    return janet_wrap_struct(janet_struct_end(st));
+}
+
 /* Push a stack frame to a fiber */
 int janet_fiber_funcframe(JanetFiber *fiber, JanetFunction *func) {
     JanetStackFrame *newframe;
@@ -137,11 +160,8 @@ int janet_fiber_funcframe(JanetFiber *fiber, JanetFunction *func) {
     int32_t next_arity = fiber->stacktop - fiber->stackstart;
 
     /* Check strict arity before messing with state */
-    if (func->def->flags & JANET_FUNCDEF_FLAG_FIXARITY) {
-        if (func->def->arity != next_arity) {
-            return 1;
-        }
-    }
+    if (next_arity < func->def->min_arity) return 1;
+    if (next_arity > func->def->max_arity) return 1;
 
     if (fiber->capacity < nextstacktop) {
         janet_fiber_setcapacity(fiber, 2 * nextstacktop);
@@ -165,12 +185,19 @@ int janet_fiber_funcframe(JanetFiber *fiber, JanetFunction *func) {
     /* Check varargs */
     if (func->def->flags & JANET_FUNCDEF_FLAG_VARARG) {
         int32_t tuplehead = fiber->frame + func->def->arity;
+        int st = func->def->flags & JANET_FUNCDEF_FLAG_STRUCTARG;
         if (tuplehead >= oldtop) {
-            fiber->data[tuplehead] = janet_wrap_tuple(janet_tuple_n(NULL, 0));
+            fiber->data[tuplehead] = st
+                                     ? make_struct_n(NULL, 0)
+                                     : janet_wrap_tuple(janet_tuple_n(NULL, 0));
         } else {
-            fiber->data[tuplehead] = janet_wrap_tuple(janet_tuple_n(
-                fiber->data + tuplehead,
-                oldtop - tuplehead));
+            fiber->data[tuplehead] = st
+                                     ? make_struct_n(
+                                         fiber->data + tuplehead,
+                                         oldtop - tuplehead)
+                                     : janet_wrap_tuple(janet_tuple_n(
+                                                 fiber->data + tuplehead,
+                                                 oldtop - tuplehead));
         }
     }
 
@@ -185,6 +212,7 @@ static void janet_env_detach(JanetFuncEnv *env) {
     if (env) {
         size_t s = sizeof(Janet) * env->length;
         Janet *vmem = malloc(s);
+        janet_vm_next_collection += (uint32_t) s;
         if (NULL == vmem) {
             JANET_OUT_OF_MEMORY;
         }
@@ -203,11 +231,8 @@ int janet_fiber_funcframe_tail(JanetFiber *fiber, JanetFunction *func) {
     int32_t stacksize;
 
     /* Check strict arity before messing with state */
-    if (func->def->flags & JANET_FUNCDEF_FLAG_FIXARITY) {
-        if (func->def->arity != next_arity) {
-            return 1;
-        }
-    }
+    if (next_arity < func->def->min_arity) return 1;
+    if (next_arity > func->def->max_arity) return 1;
 
     if (fiber->capacity < nextstacktop) {
         janet_fiber_setcapacity(fiber, 2 * nextstacktop);
@@ -224,14 +249,21 @@ int janet_fiber_funcframe_tail(JanetFiber *fiber, JanetFunction *func) {
     /* Check varargs */
     if (func->def->flags & JANET_FUNCDEF_FLAG_VARARG) {
         int32_t tuplehead = fiber->stackstart + func->def->arity;
+        int st = func->def->flags & JANET_FUNCDEF_FLAG_STRUCTARG;
         if (tuplehead >= fiber->stacktop) {
             if (tuplehead >= fiber->capacity) janet_fiber_setcapacity(fiber, 2 * (tuplehead + 1));
             for (i = fiber->stacktop; i < tuplehead; ++i) fiber->data[i] = janet_wrap_nil();
-            fiber->data[tuplehead] = janet_wrap_tuple(janet_tuple_n(NULL, 0));
+            fiber->data[tuplehead] = st
+                                     ? make_struct_n(NULL, 0)
+                                     : janet_wrap_tuple(janet_tuple_n(NULL, 0));
         } else {
-            fiber->data[tuplehead] = janet_wrap_tuple(janet_tuple_n(
-                fiber->data + tuplehead,
-                fiber->stacktop - tuplehead));
+            fiber->data[tuplehead] = st
+                                     ? make_struct_n(
+                                         fiber->data + tuplehead,
+                                         fiber->stacktop - tuplehead)
+                                     : janet_wrap_tuple(janet_tuple_n(
+                                                 fiber->data + tuplehead,
+                                                 fiber->stacktop - tuplehead));
         }
         stacksize = tuplehead - fiber->stackstart + 1;
     } else {
@@ -296,16 +328,41 @@ void janet_fiber_popframe(JanetFiber *fiber) {
     fiber->frame = frame->prevframe;
 }
 
+JanetFiberStatus janet_fiber_status(JanetFiber *f) {
+    return ((f)->flags & JANET_FIBER_STATUS_MASK) >> JANET_FIBER_STATUS_OFFSET;
+}
+
+JanetFiber *janet_current_fiber(void) {
+    return janet_vm_fiber;
+}
+
 /* CFuns */
 
+static Janet cfun_fiber_getenv(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    JanetFiber *fiber = janet_getfiber(argv, 0);
+    return fiber->env ?
+           janet_wrap_table(fiber->env) :
+           janet_wrap_nil();
+}
+
+static Janet cfun_fiber_setenv(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 2);
+    JanetFiber *fiber = janet_getfiber(argv, 0);
+    if (janet_checktype(argv[1], JANET_NIL)) {
+        fiber->env = NULL;
+    } else {
+        fiber->env = janet_gettable(argv, 1);
+    }
+    return argv[0];
+}
+
 static Janet cfun_fiber_new(int32_t argc, Janet *argv) {
     janet_arity(argc, 1, 2);
     JanetFunction *func = janet_getfunction(argv, 0);
     JanetFiber *fiber;
-    if (func->def->flags & JANET_FUNCDEF_FLAG_FIXARITY) {
-        if (func->def->arity != 0) {
-            janet_panic("expected nullary function in fiber constructor");
-        }
+    if (func->def->min_arity != 0) {
+        janet_panic("expected nullary function in fiber constructor");
     }
     fiber = janet_fiber(func, 64, 0, NULL);
     if (argc == 2) {
@@ -340,6 +397,19 @@ static Janet cfun_fiber_new(int32_t argc, Janet *argv) {
                     case 'y':
                         fiber->flags |= JANET_FIBER_MASK_YIELD;
                         break;
+                    case 'i':
+                        if (!janet_vm_fiber->env) {
+                            janet_vm_fiber->env = janet_table(0);
+                        }
+                        fiber->env = janet_vm_fiber->env;
+                        break;
+                    case 'p':
+                        if (!janet_vm_fiber->env) {
+                            janet_vm_fiber->env = janet_table(0);
+                        }
+                        fiber->env = janet_table(0);
+                        fiber->env->proto = janet_vm_fiber->env;
+                        break;
                 }
             }
         }
@@ -350,8 +420,7 @@ static Janet cfun_fiber_new(int32_t argc, Janet *argv) {
 static Janet cfun_fiber_status(int32_t argc, Janet *argv) {
     janet_fixarity(argc, 1);
     JanetFiber *fiber = janet_getfiber(argv, 0);
-    uint32_t s = (fiber->flags & JANET_FIBER_STATUS_MASK) >>
-        JANET_FIBER_STATUS_OFFSET;
+    uint32_t s = janet_fiber_status(fiber);
     return janet_ckeywordv(janet_status_names[s]);
 }
 
@@ -381,56 +450,72 @@ static Janet cfun_fiber_setmaxstack(int32_t argc, Janet *argv) {
 static const JanetReg fiber_cfuns[] = {
     {
         "fiber/new", cfun_fiber_new,
-        JDOC("(fiber/new func [,sigmask])\n\n"
-                "Create a new fiber with function body func. Can optionally "
-                "take a set of signals to block from the current parent fiber "
-                "when called. The mask is specified as a keyword where each character "
-                "is used to indicate a signal to block. The default sigmask is :y. "
-                "For example, \n\n"
-                "\t(fiber/new myfun :e123)\n\n"
-                "blocks error signals and user signals 1, 2 and 3. The signals are "
-                "as follows: \n\n"
-                "\ta - block all signals\n"
-                "\td - block debug signals\n"
-                "\te - block error signals\n"
-                "\tu - block user signals\n"
-                "\ty - block yield signals\n"
-                "\t0-9 - block a specific user signal")
+        JDOC("(fiber/new func &opt sigmask)\n\n"
+             "Create a new fiber with function body func. Can optionally "
+             "take a set of signals to block from the current parent fiber "
+             "when called. The mask is specified as a keyword where each character "
+             "is used to indicate a signal to block. The default sigmask is :y. "
+             "For example, \n\n"
+             "\t(fiber/new myfun :e123)\n\n"
+             "blocks error signals and user signals 1, 2 and 3. The signals are "
+             "as follows: \n\n"
+             "\ta - block all signals\n"
+             "\td - block debug signals\n"
+             "\te - block error signals\n"
+             "\tu - block user signals\n"
+             "\ty - block yield signals\n"
+             "\t0-9 - block a specific user signal\n\n"
+             "The sigmask argument also can take environment flags. If any mutually "
+             "exclusive flags are present, the last flag takes precedence.\n\n"
+             "\ti - inherit the environment from the current fiber\n"
+             "\tp - the environment table's prototype is the current environment table")
     },
     {
         "fiber/status", cfun_fiber_status,
         JDOC("(fiber/status fib)\n\n"
-                "Get the status of a fiber. The status will be one of:\n\n"
-                "\t:dead - the fiber has finished\n"
-                "\t:error - the fiber has errored out\n"
-                "\t:debug - the fiber is suspended in debug mode\n"
-                "\t:pending - the fiber has been yielded\n"
-                "\t:user(0-9) - the fiber is suspended by a user signal\n"
-                "\t:alive - the fiber is currently running and cannot be resumed\n"
-                "\t:new - the fiber has just been created and not yet run")
+             "Get the status of a fiber. The status will be one of:\n\n"
+             "\t:dead - the fiber has finished\n"
+             "\t:error - the fiber has errored out\n"
+             "\t:debug - the fiber is suspended in debug mode\n"
+             "\t:pending - the fiber has been yielded\n"
+             "\t:user(0-9) - the fiber is suspended by a user signal\n"
+             "\t:alive - the fiber is currently running and cannot be resumed\n"
+             "\t:new - the fiber has just been created and not yet run")
     },
     {
         "fiber/current", cfun_fiber_current,
         JDOC("(fiber/current)\n\n"
-                "Returns the currently running fiber.")
+             "Returns the currently running fiber.")
     },
     {
         "fiber/maxstack", cfun_fiber_maxstack,
         JDOC("(fiber/maxstack fib)\n\n"
-                "Gets the maximum stack size in janet values allowed for a fiber. While memory for "
-                "the fiber's stack is not allocated up front, the fiber will not allocated more "
-                "than this amount and will throw a stack-overflow error if more memory is needed. ")
+             "Gets the maximum stack size in janet values allowed for a fiber. While memory for "
+             "the fiber's stack is not allocated up front, the fiber will not allocated more "
+             "than this amount and will throw a stack-overflow error if more memory is needed. ")
     },
     {
         "fiber/setmaxstack", cfun_fiber_setmaxstack,
         JDOC("(fiber/setmaxstack fib maxstack)\n\n"
-                "Sets the maximum stack size in janet values for a fiber. By default, the "
-                "maximum stack size is usually 8192.")
+             "Sets the maximum stack size in janet values for a fiber. By default, the "
+             "maximum stack size is usually 8192.")
+    },
+    {
+        "fiber/getenv", cfun_fiber_getenv,
+        JDOC("(fiber/getenv fiber)\n\n"
+             "Gets the environment for a fiber. Returns nil if no such table is "
+             "set yet.")
+    },
+    {
+        "fiber/setenv", cfun_fiber_setenv,
+        JDOC("(fiber/setenv fiber table)\n\n"
+             "Sets the environment table for a fiber. Set to nil to remove the current "
+             "environment.")
     },
     {NULL, NULL, NULL}
 };
 
 /* Module entry point */
 void janet_lib_fiber(JanetTable *env) {
-    janet_cfuns(env, NULL, fiber_cfuns);
+    janet_core_cfuns(env, NULL, fiber_cfuns);
 }

src/core/fiber.h

@@ -24,7 +24,7 @@
 #define JANET_FIBER_H_defined
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #endif
 
 extern JANET_THREAD_LOCAL JanetFiber *janet_vm_fiber;

src/core/gc.c

@@ -21,10 +21,11 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "state.h"
 #include "symcache.h"
 #include "gc.h"
+#include "util.h"
 #endif
 
 /* GC State */
@@ -38,6 +39,18 @@ JANET_THREAD_LOCAL Janet *janet_vm_roots;
 JANET_THREAD_LOCAL uint32_t janet_vm_root_count;
 JANET_THREAD_LOCAL uint32_t janet_vm_root_capacity;
 
+/* Scratch Memory */
+#ifdef JANET_64
+#define SCRATCH_HDR_SIZE 16 /* smalloc must guarantee 16 byte alignment. */
+#elif JANET_32
+#define SCRATCH_HDR_SIZE 8  /* smalloc must guarantee 8 byte alignment. */
+#else
+#error "unknown scratch alignment"
+#endif
+JANET_THREAD_LOCAL void **janet_scratch_mem;
+JANET_THREAD_LOCAL size_t janet_scratch_cap;
+JANET_THREAD_LOCAL size_t janet_scratch_len;
+
 /* Helpers for marking the various gc types */
 static void janet_mark_funcenv(JanetFuncEnv *env);
 static void janet_mark_funcdef(JanetFuncDef *def);
@@ -60,18 +73,37 @@ void janet_mark(Janet x) {
     if (depth) {
         depth--;
         switch (janet_type(x)) {
-            default: break;
+            default:
+                break;
             case JANET_STRING:
             case JANET_KEYWORD:
-            case JANET_SYMBOL: janet_mark_string(janet_unwrap_string(x)); break;
-            case JANET_FUNCTION: janet_mark_function(janet_unwrap_function(x)); break;
-            case JANET_ARRAY: janet_mark_array(janet_unwrap_array(x)); break;
-            case JANET_TABLE: janet_mark_table(janet_unwrap_table(x)); break;
-            case JANET_STRUCT: janet_mark_struct(janet_unwrap_struct(x)); break;
-            case JANET_TUPLE: janet_mark_tuple(janet_unwrap_tuple(x)); break;
-            case JANET_BUFFER: janet_mark_buffer(janet_unwrap_buffer(x)); break;
-            case JANET_FIBER: janet_mark_fiber(janet_unwrap_fiber(x)); break;
-            case JANET_ABSTRACT: janet_mark_abstract(janet_unwrap_abstract(x)); break;
+            case JANET_SYMBOL:
+                janet_mark_string(janet_unwrap_string(x));
+                break;
+            case JANET_FUNCTION:
+                janet_mark_function(janet_unwrap_function(x));
+                break;
+            case JANET_ARRAY:
+                janet_mark_array(janet_unwrap_array(x));
+                break;
+            case JANET_TABLE:
+                janet_mark_table(janet_unwrap_table(x));
+                break;
+            case JANET_STRUCT:
+                janet_mark_struct(janet_unwrap_struct(x));
+                break;
+            case JANET_TUPLE:
+                janet_mark_tuple(janet_unwrap_tuple(x));
+                break;
+            case JANET_BUFFER:
+                janet_mark_buffer(janet_unwrap_buffer(x));
+                break;
+            case JANET_FIBER:
+                janet_mark_fiber(janet_unwrap_fiber(x));
+                break;
+            case JANET_ABSTRACT:
+                janet_mark_abstract(janet_unwrap_abstract(x));
+                break;
         }
         depth++;
     } else {
@@ -80,7 +112,7 @@ void janet_mark(Janet x) {
 }
 
 static void janet_mark_string(const uint8_t *str) {
-    janet_gc_mark(janet_string_raw(str));
+    janet_gc_mark(janet_string_head(str));
 }
 
 static void janet_mark_buffer(JanetBuffer *buffer) {
@@ -88,11 +120,11 @@ static void janet_mark_buffer(JanetBuffer *buffer) {
 }
 
 static void janet_mark_abstract(void *adata) {
-    if (janet_gc_reachable(janet_abstract_header(adata)))
+    if (janet_gc_reachable(janet_abstract_head(adata)))
         return;
-    janet_gc_mark(janet_abstract_header(adata));
-    if (janet_abstract_header(adata)->type->gcmark) {
-        janet_abstract_header(adata)->type->gcmark(adata, janet_abstract_size(adata));
+    janet_gc_mark(janet_abstract_head(adata));
+    if (janet_abstract_head(adata)->type->gcmark) {
+        janet_abstract_head(adata)->type->gcmark(adata, janet_abstract_size(adata));
     }
 }
 
@@ -123,7 +155,7 @@ static void janet_mark_array(JanetArray *array) {
 }
 
 static void janet_mark_table(JanetTable *table) {
-    recur: /* Manual tail recursion */
+recur: /* Manual tail recursion */
     if (janet_gc_reachable(table))
         return;
     janet_gc_mark(table);
@@ -135,16 +167,16 @@ static void janet_mark_table(JanetTable *table) {
 }
 
 static void janet_mark_struct(const JanetKV *st) {
-    if (janet_gc_reachable(janet_struct_raw(st)))
+    if (janet_gc_reachable(janet_struct_head(st)))
         return;
-    janet_gc_mark(janet_struct_raw(st));
+    janet_gc_mark(janet_struct_head(st));
     janet_mark_kvs(st, janet_struct_capacity(st));
 }
 
 static void janet_mark_tuple(const Janet *tuple) {
-    if (janet_gc_reachable(janet_tuple_raw(tuple)))
+    if (janet_gc_reachable(janet_tuple_head(tuple)))
         return;
-    janet_gc_mark(janet_tuple_raw(tuple));
+    janet_gc_mark(janet_tuple_head(tuple));
     janet_mark_many(tuple, janet_tuple_length(tuple));
 }
 
@@ -201,7 +233,7 @@ recur:
 
     /* Mark values on the argument stack */
     janet_mark_many(fiber->data + fiber->stackstart,
-            fiber->stacktop - fiber->stackstart);
+                    fiber->stacktop - fiber->stackstart);
 
     i = fiber->frame;
     j = fiber->stackstart - JANET_FRAME_SIZE;
@@ -217,6 +249,9 @@ recur:
         i = frame->prevframe;
     }
 
+    if (fiber->env)
+        janet_mark_table(fiber->env);
+
     /* Explicit tail recursion */
     if (fiber->child) {
         fiber = fiber->child;
@@ -225,21 +260,19 @@ recur:
 }
 
 /* Deinitialize a block of memory */
-static void janet_deinit_block(JanetGCMemoryHeader *block) {
-    void *mem = ((char *)(block + 1));
-    JanetAbstractHeader *h = (JanetAbstractHeader *)mem;
-    switch (block->flags & JANET_MEM_TYPEBITS) {
+static void janet_deinit_block(JanetGCObject *mem) {
+    switch (mem->flags & JANET_MEM_TYPEBITS) {
         default:
         case JANET_MEMORY_FUNCTION:
             break; /* Do nothing for non gc types */
         case JANET_MEMORY_SYMBOL:
-            janet_symbol_deinit((const uint8_t *)mem + 2 * sizeof(int32_t));
+            janet_symbol_deinit(((JanetStringHead *) mem)->data);
             break;
         case JANET_MEMORY_ARRAY:
-            janet_array_deinit((JanetArray*) mem);
+            free(((JanetArray *) mem)->data);
             break;
         case JANET_MEMORY_TABLE:
-            janet_table_deinit((JanetTable*) mem);
+            free(((JanetTable *) mem)->data);
             break;
         case JANET_MEMORY_FIBER:
             free(((JanetFiber *)mem)->data);
@@ -247,38 +280,38 @@ static void janet_deinit_block(JanetGCMemoryHeader *block) {
         case JANET_MEMORY_BUFFER:
             janet_buffer_deinit((JanetBuffer *) mem);
             break;
-        case JANET_MEMORY_ABSTRACT:
-            if (h->type->gc) {
-                janet_assert(!h->type->gc((void *)(h + 1), h->size), "finalizer failed");
+        case JANET_MEMORY_ABSTRACT: {
+            JanetAbstractHead *head = (JanetAbstractHead *)mem;
+            if (head->type->gc) {
+                janet_assert(!head->type->gc(head->data, head->size), "finalizer failed");
             }
-            break;
-        case JANET_MEMORY_FUNCENV:
-            {
-                JanetFuncEnv *env = (JanetFuncEnv *)mem;
-                if (0 == env->offset)
-                    free(env->as.values);
-            }
-            break;
-        case JANET_MEMORY_FUNCDEF:
-            {
-                JanetFuncDef *def = (JanetFuncDef *)mem;
-                /* TODO - get this all with one alloc and one free */
-                free(def->defs);
-                free(def->environments);
-                free(def->constants);
-                free(def->bytecode);
-                free(def->sourcemap);
-            }
-            break;
+        }
+        break;
+        case JANET_MEMORY_FUNCENV: {
+            JanetFuncEnv *env = (JanetFuncEnv *)mem;
+            if (0 == env->offset)
+                free(env->as.values);
+        }
+        break;
+        case JANET_MEMORY_FUNCDEF: {
+            JanetFuncDef *def = (JanetFuncDef *)mem;
+            /* TODO - get this all with one alloc and one free */
+            free(def->defs);
+            free(def->environments);
+            free(def->constants);
+            free(def->bytecode);
+            free(def->sourcemap);
+        }
+        break;
     }
 }
 
 /* Iterate over all allocated memory, and free memory that is not
  * marked as reachable. Flip the gc color flag for next sweep. */
 void janet_sweep() {
-    JanetGCMemoryHeader *previous = NULL;
-    JanetGCMemoryHeader *current = janet_vm_blocks;
-    JanetGCMemoryHeader *next;
+    JanetGCObject *previous = NULL;
+    JanetGCObject *current = janet_vm_blocks;
+    JanetGCObject *next;
     while (NULL != current) {
         next = current->next;
         if (current->flags & (JANET_MEM_REACHABLE | JANET_MEM_DISABLED)) {
@@ -299,29 +332,41 @@ void janet_sweep() {
 
 /* Allocate some memory that is tracked for garbage collection */
 void *janet_gcalloc(enum JanetMemoryType type, size_t size) {
-    JanetGCMemoryHeader *mdata;
-    size_t total = size + sizeof(JanetGCMemoryHeader);
+    JanetGCObject *mem;
 
     /* Make sure everything is inited */
     janet_assert(NULL != janet_vm_cache, "please initialize janet before use");
-    void *mem = malloc(total);
+    mem = malloc(size);
 
     /* Check for bad malloc */
     if (NULL == mem) {
         JANET_OUT_OF_MEMORY;
     }
 
-    mdata = (JanetGCMemoryHeader *)mem;
-
     /* Configure block */
-    mdata->flags = type;
+    mem->flags = type;
 
     /* Prepend block to heap list */
     janet_vm_next_collection += (int32_t) size;
-    mdata->next = janet_vm_blocks;
-    janet_vm_blocks = mdata;
+    mem->next = janet_vm_blocks;
+    janet_vm_blocks = mem;
 
-    return (char *) mem + sizeof(JanetGCMemoryHeader);
+    return (void *)mem;
+}
+
+static void free_one_scratch(void *mem) {
+    ScratchFinalizer finalize = *(ScratchFinalizer *)mem;
+    if (finalize)
+        finalize((char *)mem + SCRATCH_HDR_SIZE);
+    free(mem);
+}
+
+/* Free all allocated scratch memory */
+static void janet_free_all_scratch(void) {
+    for (size_t i = 0; i < janet_scratch_len; i++) {
+        free_one_scratch(janet_scratch_mem[i]);
+    }
+    janet_scratch_len = 0;
 }
 
 /* Run garbage collection */
@@ -338,6 +383,7 @@ void janet_collect(void) {
     }
     janet_sweep();
     janet_vm_next_collection = 0;
+    janet_free_all_scratch();
 }
 
 /* Add a root value to the GC. This prevents the GC from removing a value
@@ -362,8 +408,7 @@ static int janet_gc_idequals(Janet lhs, Janet rhs) {
     if (janet_type(lhs) != janet_type(rhs))
         return 0;
     switch (janet_type(lhs)) {
-        case JANET_TRUE:
-        case JANET_FALSE:
+        case JANET_BOOLEAN:
         case JANET_NIL:
         case JANET_NUMBER:
             /* These values don't really matter to the gc so returning 1 all the time is fine. */
@@ -377,9 +422,8 @@ static int janet_gc_idequals(Janet lhs, Janet rhs) {
  * a value and all its children. */
 int janet_gcunroot(Janet root) {
     Janet *vtop = janet_vm_roots + janet_vm_root_count;
-    Janet *v = janet_vm_roots;
     /* Search from top to bottom as access is most likely LIFO */
-    for (v = janet_vm_roots; v < vtop; v++) {
+    for (Janet *v = janet_vm_roots; v < vtop; v++) {
         if (janet_gc_idequals(root, *v)) {
             *v = janet_vm_roots[--janet_vm_root_count];
             return 1;
@@ -391,10 +435,9 @@ int janet_gcunroot(Janet root) {
 /* Remove a root value from the GC. This sets the effective reference count to 0. */
 int janet_gcunrootall(Janet root) {
     Janet *vtop = janet_vm_roots + janet_vm_root_count;
-    Janet *v = janet_vm_roots;
     int ret = 0;
     /* Search from top to bottom as access is most likely LIFO */
-    for (v = janet_vm_roots; v < vtop; v++) {
+    for (Janet *v = janet_vm_roots; v < vtop; v++) {
         if (janet_gc_idequals(root, *v)) {
             *v = janet_vm_roots[--janet_vm_root_count];
             vtop--;
@@ -406,16 +449,83 @@ int janet_gcunrootall(Janet root) {
 
 /* Free all allocated memory */
 void janet_clear_memory(void) {
-    JanetGCMemoryHeader *current = janet_vm_blocks;
+    JanetGCObject *current = janet_vm_blocks;
     while (NULL != current) {
         janet_deinit_block(current);
-        JanetGCMemoryHeader *next = current->next;
+        JanetGCObject *next = current->next;
         free(current);
         current = next;
     }
     janet_vm_blocks = NULL;
+    janet_free_all_scratch();
+    free(janet_scratch_mem);
 }
 
 /* Primitives for suspending GC. */
-int janet_gclock(void) { return janet_vm_gc_suspend++; }
-void janet_gcunlock(int handle) { janet_vm_gc_suspend = handle; }
+int janet_gclock(void) {
+    return janet_vm_gc_suspend++;
+}
+void janet_gcunlock(int handle) {
+    janet_vm_gc_suspend = handle;
+}
+
+/* Scratch memory API */
+
+void *janet_smalloc(size_t size) {
+    void *mem = malloc(SCRATCH_HDR_SIZE + size);
+    if (NULL == mem) {
+        JANET_OUT_OF_MEMORY;
+    }
+    *(ScratchFinalizer *)mem = NULL;
+    if (janet_scratch_len == janet_scratch_cap) {
+        size_t newcap = 2 * janet_scratch_cap + 2;
+        void **newmem = (void **) realloc(janet_scratch_mem, newcap * sizeof(void *));
+        if (NULL == newmem) {
+            JANET_OUT_OF_MEMORY;
+        }
+        janet_scratch_cap = newcap;
+        janet_scratch_mem = newmem;
+    }
+    janet_scratch_mem[janet_scratch_len++] = mem;
+    return (char *)mem + SCRATCH_HDR_SIZE;
+}
+
+void *janet_srealloc(void *mem, size_t size) {
+    if (NULL == mem) return janet_smalloc(size);
+    mem = (char *)mem - SCRATCH_HDR_SIZE;
+    if (janet_scratch_len) {
+        for (size_t i = janet_scratch_len - 1; ; i--) {
+            if (janet_scratch_mem[i] == mem) {
+                void *newmem = realloc(mem, size + SCRATCH_HDR_SIZE);
+                if (NULL == newmem) {
+                    JANET_OUT_OF_MEMORY;
+                }
+                janet_scratch_mem[i] = newmem;
+                return (char *)newmem + SCRATCH_HDR_SIZE;
+            }
+            if (i == 0) break;
+        }
+    }
+    janet_exit("invalid janet_srealloc");
+}
+
+void janet_sfinalizer(void *mem, ScratchFinalizer finalizer) {
+    mem = (char *)mem - SCRATCH_HDR_SIZE;
+    *(ScratchFinalizer *)mem = finalizer;
+}
+
+void janet_sfree(void *mem) {
+    if (NULL == mem) return;
+    mem = (char *)mem - SCRATCH_HDR_SIZE;
+    if (janet_scratch_len) {
+        for (size_t i = janet_scratch_len - 1; ; i--) {
+            if (janet_scratch_mem[i] == mem) {
+                janet_scratch_mem[i] = janet_scratch_mem[--janet_scratch_len];
+                free_one_scratch(mem);
+                return;
+            }
+            if (i == 0) break;
+        }
+    }
+    janet_exit("invalid janet_sfree");
+}

src/core/gc.h

@@ -24,11 +24,11 @@
 #define JANET_GC_H
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #endif
 
 /* The metadata header associated with an allocated block of memory */
-#define janet_gc_header(mem) ((JanetGCMemoryHeader *)(mem) - 1)
+#define janet_gc_header(mem) ((JanetGCObject *)(mem))
 
 #define JANET_MEM_TYPEBITS 0xFF
 #define JANET_MEM_REACHABLE 0x100
@@ -40,13 +40,6 @@
 #define janet_gc_mark(m) (janet_gc_header(m)->flags |= JANET_MEM_REACHABLE)
 #define janet_gc_reachable(m) (janet_gc_header(m)->flags & JANET_MEM_REACHABLE)
 
-/* Memory header struct. Node of a linked list of memory blocks. */
-typedef struct JanetGCMemoryHeader JanetGCMemoryHeader;
-struct JanetGCMemoryHeader {
-    JanetGCMemoryHeader *next;
-    uint32_t flags;
-};
-
 /* Memory types for the GC. Different from JanetType to include funcenv and funcdef. */
 enum JanetMemoryType {
     JANET_MEMORY_NONE,

src/core/inttypes.c

@@ -0,0 +1,389 @@
+/*
+* Copyright (c) 2019 Calvin Rose & contributors
+*
+* Permission is hereby granted, free of charge, to any person obtaining a copy
+* of this software and associated documentation files (the "Software"), to
+* deal in the Software without restriction, including without limitation the
+* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+* sell copies of the Software, and to permit persons to whom the Software is
+* furnished to do so, subject to the following conditions:
+*
+* The above copyright notice and this permission notice shall be included in
+* all copies or substantial portions of the Software.
+*
+* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+* IN THE SOFTWARE.
+*/
+
+#include <errno.h>
+#include <stdlib.h>
+#include <limits.h>
+#include <inttypes.h>
+#include <math.h>
+
+#ifndef JANET_AMALG
+#include <janet.h>
+#include "util.h"
+#endif
+
+/* Conditional compilation */
+#ifdef JANET_INT_TYPES
+
+#define MAX_INT_IN_DBL 9007199254740992ULL /* 2^53 */
+
+static int it_s64_get(void *p, Janet key, Janet *out);
+static int it_u64_get(void *p, Janet key, Janet *out);
+
+static void int64_marshal(void *p, JanetMarshalContext *ctx) {
+    janet_marshal_abstract(ctx, p);
+    janet_marshal_int64(ctx, *((int64_t *)p));
+}
+
+static void *int64_unmarshal(JanetMarshalContext *ctx) {
+    int64_t *p = janet_unmarshal_abstract(ctx, sizeof(int64_t));
+    p[0] = janet_unmarshal_int64(ctx);
+    return p;
+}
+
+static void it_s64_tostring(void *p, JanetBuffer *buffer) {
+    char str[32];
+    sprintf(str, "%" PRId64, *((int64_t *)p));
+    janet_buffer_push_cstring(buffer, str);
+}
+
+static void it_u64_tostring(void *p, JanetBuffer *buffer) {
+    char str[32];
+    sprintf(str, "%" PRIu64, *((uint64_t *)p));
+    janet_buffer_push_cstring(buffer, str);
+}
+
+static const JanetAbstractType it_s64_type = {
+    "core/s64",
+    NULL,
+    NULL,
+    it_s64_get,
+    NULL,
+    int64_marshal,
+    int64_unmarshal,
+    it_s64_tostring
+};
+
+static const JanetAbstractType it_u64_type = {
+    "core/u64",
+    NULL,
+    NULL,
+    it_u64_get,
+    NULL,
+    int64_marshal,
+    int64_unmarshal,
+    it_u64_tostring
+};
+
+int64_t janet_unwrap_s64(Janet x) {
+    switch (janet_type(x)) {
+        default:
+            break;
+        case JANET_NUMBER : {
+            double dbl = janet_unwrap_number(x);
+            if (fabs(dbl) <=  MAX_INT_IN_DBL)
+                return (int64_t)dbl;
+            break;
+        }
+        case JANET_STRING: {
+            int64_t value;
+            const uint8_t *str = janet_unwrap_string(x);
+            if (janet_scan_int64(str, janet_string_length(str), &value))
+                return value;
+            break;
+        }
+        case JANET_ABSTRACT: {
+            void *abst = janet_unwrap_abstract(x);
+            if (janet_abstract_type(abst) == &it_s64_type ||
+                    (janet_abstract_type(abst) == &it_u64_type))
+                return *(int64_t *)abst;
+            break;
+        }
+    }
+    janet_panic("bad s64 initializer");
+    return 0;
+}
+
+uint64_t janet_unwrap_u64(Janet x) {
+    switch (janet_type(x)) {
+        default:
+            break;
+        case JANET_NUMBER : {
+            double dbl = janet_unwrap_number(x);
+            if ((dbl >= 0) && (dbl <= MAX_INT_IN_DBL))
+                return (uint64_t)dbl;
+            break;
+        }
+        case JANET_STRING: {
+            uint64_t value;
+            const uint8_t *str = janet_unwrap_string(x);
+            if (janet_scan_uint64(str, janet_string_length(str), &value))
+                return value;
+            break;
+        }
+        case JANET_ABSTRACT: {
+            void *abst = janet_unwrap_abstract(x);
+            if (janet_abstract_type(abst) == &it_s64_type ||
+                    (janet_abstract_type(abst) == &it_u64_type))
+                return *(uint64_t *)abst;
+            break;
+        }
+    }
+    janet_panic("bad u64 initializer");
+    return 0;
+}
+
+JanetIntType janet_is_int(Janet x) {
+    if (!janet_checktype(x, JANET_ABSTRACT)) return JANET_INT_NONE;
+    const JanetAbstractType *at = janet_abstract_type(janet_unwrap_abstract(x));
+    return (at == &it_s64_type) ? JANET_INT_S64 :
+           ((at == &it_u64_type) ? JANET_INT_U64 :
+            JANET_INT_NONE);
+}
+
+Janet janet_wrap_s64(int64_t x) {
+    int64_t *box = janet_abstract(&it_s64_type, sizeof(int64_t));
+    *box = (int64_t)x;
+    return janet_wrap_abstract(box);
+}
+
+Janet janet_wrap_u64(uint64_t x) {
+    uint64_t *box = janet_abstract(&it_u64_type, sizeof(uint64_t));
+    *box = (uint64_t)x;
+    return janet_wrap_abstract(box);
+}
+
+static Janet cfun_it_s64_new(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    return janet_wrap_s64(janet_unwrap_s64(argv[0]));
+}
+
+static Janet cfun_it_u64_new(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    return janet_wrap_u64(janet_unwrap_u64(argv[0]));
+}
+
+#define OPMETHOD(T, type, name, oper) \
+static Janet cfun_it_##type##_##name(int32_t argc, Janet *argv) { \
+    janet_arity(argc, 2, -1); \
+    T *box = janet_abstract(&it_##type##_type, sizeof(T)); \
+    *box = janet_unwrap_##type(argv[0]); \
+    for (int i = 1; i < argc; i++) \
+        *box oper##= janet_unwrap_##type(argv[i]); \
+    return janet_wrap_abstract(box); \
+} \
+ \
+static Janet cfun_it_##type##_##name##_mut(int32_t argc, Janet *argv) { \
+    janet_arity(argc, 2, -1); \
+    T *box = janet_getabstract(argv,0,&it_##type##_type); \
+    for (int i = 1; i < argc; i++) \
+        *box oper##= janet_unwrap_##type(argv[i]); \
+    return janet_wrap_abstract(box); \
+}
+
+#define DIVMETHOD(T, type, name, oper) \
+static Janet cfun_it_##type##_##name(int32_t argc, Janet *argv) { \
+    janet_arity(argc, 2, -1);                       \
+    T *box = janet_abstract(&it_##type##_type, sizeof(T)); \
+    *box = janet_unwrap_##type(argv[0]); \
+    for (int i = 1; i < argc; i++) { \
+      T value = janet_unwrap_##type(argv[i]); \
+      if (value == 0) janet_panic("division by zero"); \
+      *box oper##= value; \
+    } \
+    return janet_wrap_abstract(box); \
+} \
+ \
+static Janet cfun_it_##type##_##name##_mut(int32_t argc, Janet *argv) { \
+    janet_arity(argc, 2, -1); \
+    T *box = janet_getabstract(argv,0,&it_##type##_type); \
+    for (int i = 1; i < argc; i++) { \
+      T value =  janet_unwrap_##type(argv[i]); \
+      if (value == 0) janet_panic("division by zero"); \
+      *box oper##= value; \
+    } \
+    return janet_wrap_abstract(box); \
+}
+
+#define DIVMETHOD_SIGNED(T, type, name, oper) \
+static Janet cfun_it_##type##_##name(int32_t argc, Janet *argv) { \
+    janet_arity(argc, 2, -1);                       \
+    T *box = janet_abstract(&it_##type##_type, sizeof(T)); \
+    *box = janet_unwrap_##type(argv[0]); \
+    for (int i = 1; i < argc; i++) { \
+      T value = janet_unwrap_##type(argv[i]); \
+      if (value == 0) janet_panic("division by zero"); \
+      if ((value == -1) && (*box == INT64_MIN)) janet_panic("INT64_MIN divided by -1"); \
+      *box oper##= value; \
+    } \
+    return janet_wrap_abstract(box); \
+} \
+ \
+static Janet cfun_it_##type##_##name##_mut(int32_t argc, Janet *argv) { \
+    janet_arity(argc, 2, -1); \
+    T *box = janet_getabstract(argv,0,&it_##type##_type); \
+    for (int i = 1; i < argc; i++) { \
+      T value = janet_unwrap_##type(argv[i]); \
+      if (value == 0) janet_panic("division by zero"); \
+      if ((value == -1) && (*box == INT64_MIN)) janet_panic("INT64_MIN divided by -1"); \
+      *box oper##= value; \
+    } \
+    return janet_wrap_abstract(box); \
+}
+
+#define COMPMETHOD(T, type, name, oper) \
+static Janet cfun_it_##type##_##name(int32_t argc, Janet *argv) { \
+    janet_fixarity(argc, 2); \
+    T v1 = janet_unwrap_##type(argv[0]); \
+    T v2 = janet_unwrap_##type(argv[1]); \
+    return janet_wrap_boolean(v1 oper v2); \
+}
+
+OPMETHOD(int64_t, s64, add, +)
+OPMETHOD(int64_t, s64, sub, -)
+OPMETHOD(int64_t, s64, mul, *)
+DIVMETHOD_SIGNED(int64_t, s64, div, /)
+DIVMETHOD_SIGNED(int64_t, s64, mod, %)
+OPMETHOD(int64_t, s64, and, &)
+OPMETHOD(int64_t, s64, or, |)
+OPMETHOD(int64_t, s64, xor, ^)
+OPMETHOD(int64_t, s64, lshift, <<)
+OPMETHOD(int64_t, s64, rshift, >>)
+COMPMETHOD(int64_t, s64, lt, <)
+COMPMETHOD(int64_t, s64, gt, >)
+COMPMETHOD(int64_t, s64, le, <=)
+COMPMETHOD(int64_t, s64, ge, >=)
+COMPMETHOD(int64_t, s64, eq, ==)
+COMPMETHOD(int64_t, s64, ne, !=)
+
+OPMETHOD(uint64_t, u64, add, +)
+OPMETHOD(uint64_t, u64, sub, -)
+OPMETHOD(uint64_t, u64, mul, *)
+DIVMETHOD(uint64_t, u64, div, /)
+DIVMETHOD(uint64_t, u64, mod, %)
+OPMETHOD(uint64_t, u64, and, &)
+OPMETHOD(uint64_t, u64, or, |)
+OPMETHOD(uint64_t, u64, xor, ^)
+OPMETHOD(uint64_t, u64, lshift, <<)
+OPMETHOD(uint64_t, u64, rshift, >>)
+COMPMETHOD(uint64_t, u64, lt, <)
+COMPMETHOD(uint64_t, u64, gt, >)
+COMPMETHOD(uint64_t, u64, le, <=)
+COMPMETHOD(uint64_t, u64, ge, >=)
+COMPMETHOD(uint64_t, u64, eq, ==)
+COMPMETHOD(uint64_t, u64, ne, !=)
+
+#undef OPMETHOD
+#undef DIVMETHOD
+#undef DIVMETHOD_SIGNED
+#undef COMPMETHOD
+
+static JanetMethod it_s64_methods[] = {
+    {"+", cfun_it_s64_add},
+    {"-", cfun_it_s64_sub},
+    {"*", cfun_it_s64_mul},
+    {"/", cfun_it_s64_div},
+    {"%", cfun_it_s64_mod},
+    {"<", cfun_it_s64_lt},
+    {">", cfun_it_s64_gt},
+    {"<=", cfun_it_s64_le},
+    {">=", cfun_it_s64_ge},
+    {"==", cfun_it_s64_eq},
+    {"!=", cfun_it_s64_ne},
+    {"&", cfun_it_s64_and},
+    {"|", cfun_it_s64_or},
+    {"^", cfun_it_s64_xor},
+    {"<<", cfun_it_s64_lshift},
+    {">>", cfun_it_s64_rshift},
+
+    {"+!", cfun_it_s64_add_mut},
+    {"-!", cfun_it_s64_sub_mut},
+    {"*!", cfun_it_s64_mul_mut},
+    {"/!", cfun_it_s64_div_mut},
+    {"%!", cfun_it_s64_mod_mut},
+    {"&!", cfun_it_s64_and_mut},
+    {"|!", cfun_it_s64_or_mut},
+    {"^!", cfun_it_s64_xor_mut},
+    {"<<!", cfun_it_s64_lshift_mut},
+    {">>!", cfun_it_s64_rshift_mut},
+
+    {NULL, NULL}
+};
+
+static JanetMethod it_u64_methods[] = {
+    {"+", cfun_it_u64_add},
+    {"-", cfun_it_u64_sub},
+    {"*", cfun_it_u64_mul},
+    {"/", cfun_it_u64_div},
+    {"%", cfun_it_u64_mod},
+    {"<", cfun_it_u64_lt},
+    {">", cfun_it_u64_gt},
+    {"<=", cfun_it_u64_le},
+    {">=", cfun_it_u64_ge},
+    {"==", cfun_it_u64_eq},
+    {"!=", cfun_it_u64_ne},
+    {"&", cfun_it_u64_and},
+    {"|", cfun_it_u64_or},
+    {"^", cfun_it_u64_xor},
+    {"<<", cfun_it_u64_lshift},
+    {">>", cfun_it_u64_rshift},
+
+    {"+!", cfun_it_u64_add_mut},
+    {"-!", cfun_it_u64_sub_mut},
+    {"*!", cfun_it_u64_mul_mut},
+    {"/!", cfun_it_u64_div_mut},
+    {"%!", cfun_it_u64_mod_mut},
+    {"&!", cfun_it_u64_and_mut},
+    {"|!", cfun_it_u64_or_mut},
+    {"^!", cfun_it_u64_xor_mut},
+    {"<<!", cfun_it_u64_lshift_mut},
+    {">>!", cfun_it_u64_rshift_mut},
+
+    {NULL, NULL}
+};
+
+static int it_s64_get(void *p, Janet key, Janet *out) {
+    (void) p;
+    if (!janet_checktype(key, JANET_KEYWORD))
+        return 0;
+    return janet_getmethod(janet_unwrap_keyword(key), it_s64_methods, out);
+}
+
+static int it_u64_get(void *p, Janet key, Janet *out) {
+    (void) p;
+    if (!janet_checktype(key, JANET_KEYWORD))
+        return 0;
+    return janet_getmethod(janet_unwrap_keyword(key), it_u64_methods, out);
+}
+
+static const JanetReg it_cfuns[] = {
+    {
+        "int/s64", cfun_it_s64_new,
+        JDOC("(int/s64 value)\n\n"
+             "Create a boxed signed 64 bit integer from a string value.")
+    },
+    {
+        "int/u64", cfun_it_u64_new,
+        JDOC("(int/u64 value)\n\n"
+             "Create a boxed unsigned 64 bit integer from a string value.")
+    },
+    {NULL, NULL, NULL}
+};
+
+/* Module entry point */
+void janet_lib_inttypes(JanetTable *env) {
+    janet_core_cfuns(env, NULL, it_cfuns);
+    janet_register_abstract_type(&it_s64_type);
+    janet_register_abstract_type(&it_u64_type);
+}
+
+#endif

src/core/io.c

@@ -28,10 +28,14 @@
 #include <errno.h>
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "util.h"
 #endif
 
+#ifndef JANET_WINDOWS
+#include <sys/wait.h>
+#endif
+
 #define IO_WRITE 1
 #define IO_READ 2
 #define IO_APPEND 4
@@ -49,10 +53,16 @@ struct IOFile {
 };
 
 static int cfun_io_gc(void *p, size_t len);
+static int io_file_get(void *p, Janet key, Janet *out);
 
 JanetAbstractType cfun_io_filetype = {
     "core/file",
     cfun_io_gc,
+    NULL,
+    io_file_get,
+    NULL,
+    NULL,
+    NULL,
     NULL
 };
 
@@ -119,7 +129,7 @@ static Janet cfun_io_popen(int32_t argc, Janet *argv) {
     if (argc == 2) {
         fmode = janet_getkeyword(argv, 1);
         if (janet_string_length(fmode) != 1 ||
-            !(fmode[0] == 'r' || fmode[0] == 'w')) {
+                !(fmode[0] == 'r' || fmode[0] == 'w')) {
             janet_panicf("invalid file mode :%S, expected :r or :w", fmode);
         }
         flags = IO_PIPED | (fmode[0] == 'r' ? IO_READ : IO_WRITE);
@@ -154,7 +164,37 @@ static Janet cfun_io_fopen(int32_t argc, Janet *argv) {
     return f ? makef(f, flags) : janet_wrap_nil();
 }
 
-/* Read up to n bytes into buffer. Return error string if error. */
+static Janet cfun_io_fdopen(int32_t argc, Janet *argv) {
+    janet_arity(argc, 1, 2);
+    const int fd = janet_getinteger(argv, 0);
+    const uint8_t *fmode;
+    int flags;
+    if (argc == 2) {
+        fmode = janet_getkeyword(argv, 1);
+        flags = checkflags(fmode);
+    } else {
+        fmode = (const uint8_t *)"r";
+        flags = IO_READ;
+    }
+#ifdef JANET_WINDOWS
+#define fdopen _fdopen
+#endif
+    FILE *f = fdopen(fd, (const char *)fmode);
+    return f ? makef(f, flags) : janet_wrap_nil();
+}
+
+static Janet cfun_io_fileno(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    IOFile *iof = janet_getabstract(argv, 0, &cfun_io_filetype);
+    if (iof->flags & IO_CLOSED)
+        janet_panic("file is closed");
+#ifdef JANET_WINDOWS
+#define fileno _fileno
+#endif
+    return janet_wrap_integer(fileno(iof->file));
+}
+
+/* Read up to n bytes into buffer. */
 static void read_chunk(IOFile *iof, JanetBuffer *buffer, int32_t nBytesMax) {
     if (!(iof->flags & (IO_READ | IO_UPDATE)))
         janet_panic("file is not readable");
@@ -177,24 +217,17 @@ static Janet cfun_io_fread(int32_t argc, Janet *argv) {
     } else {
         buffer = janet_getbuffer(argv, 2);
     }
+    int32_t bufstart = buffer->count;
     if (janet_checktype(argv[1], JANET_KEYWORD)) {
         const uint8_t *sym = janet_unwrap_keyword(argv[1]);
         if (!janet_cstrcmp(sym, "all")) {
-            /* Read whole file */
-            int status = fseek(iof->file, 0, SEEK_SET);
-            if (status) {
-                /* backwards fseek did not work (stream like popen) */
-                int32_t sizeBefore;
-                do {
-                    sizeBefore = buffer->count;
-                    read_chunk(iof, buffer, 1024);
-                } while (sizeBefore < buffer->count);
-            } else {
-                fseek(iof->file, 0, SEEK_END);
-                long fsize = ftell(iof->file);
-                fseek(iof->file, 0, SEEK_SET);
-                read_chunk(iof, buffer, (int32_t) fsize);
-            }
+            int32_t sizeBefore;
+            do {
+                sizeBefore = buffer->count;
+                read_chunk(iof, buffer, 4096);
+            } while (sizeBefore < buffer->count);
+            /* Never return nil for :all */
+            return janet_wrap_buffer(buffer);
         } else if (!janet_cstrcmp(sym, "line")) {
             for (;;) {
                 int x = fgetc(iof->file);
@@ -209,6 +242,7 @@ static Janet cfun_io_fread(int32_t argc, Janet *argv) {
         if (len < 0) janet_panic("expected positive integer");
         read_chunk(iof, buffer, len);
     }
+    if (bufstart == buffer->count) return janet_wrap_nil();
     return janet_wrap_buffer(buffer);
 }
 
@@ -269,13 +303,17 @@ static Janet cfun_io_fclose(int32_t argc, Janet *argv) {
     if (iof->flags & IO_PIPED) {
 #ifdef JANET_WINDOWS
 #define pclose _pclose
+#define WEXITSTATUS(x) x
 #endif
-        if (pclose(iof->file)) janet_panic("could not close file");
+        int status = pclose(iof->file);
+        iof->flags |= IO_CLOSED;
+        if (status == -1) janet_panic("could not close file");
+        return janet_wrap_integer(WEXITSTATUS(status));
     } else {
         if (fclose(iof->file)) janet_panic("could not close file");
+        iof->flags |= IO_CLOSED;
+        return janet_wrap_nil();
     }
-    iof->flags |= IO_CLOSED;
-    return argv[0];
 }
 
 /* Seek a file */
@@ -305,88 +343,352 @@ static Janet cfun_io_fseek(int32_t argc, Janet *argv) {
     return argv[0];
 }
 
+static JanetMethod io_file_methods[] = {
+    {"close", cfun_io_fclose},
+    {"fileno", cfun_io_fileno},
+    {"flush", cfun_io_fflush},
+    {"read", cfun_io_fread},
+    {"seek", cfun_io_fseek},
+    {"write", cfun_io_fwrite},
+    {NULL, NULL}
+};
+
+static int io_file_get(void *p, Janet key, Janet *out) {
+    (void) p;
+    if (!janet_checktype(key, JANET_KEYWORD))
+        return 0;
+    return janet_getmethod(janet_unwrap_keyword(key), io_file_methods, out);
+}
+
+FILE *janet_dynfile(const char *name, FILE *def) {
+    Janet x = janet_dyn(name);
+    if (!janet_checktype(x, JANET_ABSTRACT)) return def;
+    void *abstract = janet_unwrap_abstract(x);
+    if (janet_abstract_type(abstract) != &cfun_io_filetype) return def;
+    IOFile *iofile = abstract;
+    return iofile->file;
+}
+
+static Janet cfun_io_print_impl(int32_t argc, Janet *argv,
+                                int newline, const char *name, FILE *dflt_file) {
+    FILE *f;
+    Janet x = janet_dyn(name);
+    switch (janet_type(x)) {
+        default:
+            /* Other values simply do nothing */
+            return janet_wrap_nil();
+        case JANET_BUFFER: {
+            /* Special case buffer */
+            JanetBuffer *buf = janet_unwrap_buffer(x);
+            for (int32_t i = 0; i < argc; ++i) {
+                janet_to_string_b(buf, argv[i]);
+            }
+            if (newline)
+                janet_buffer_push_u8(buf, '\n');
+            return janet_wrap_nil();
+        }
+        case JANET_NIL:
+            f = dflt_file;
+            break;
+        case JANET_ABSTRACT: {
+            void *abstract = janet_unwrap_abstract(x);
+            if (janet_abstract_type(abstract) != &cfun_io_filetype)
+                return janet_wrap_nil();
+            IOFile *iofile = abstract;
+            f = iofile->file;
+            break;
+        }
+    }
+    for (int32_t i = 0; i < argc; ++i) {
+        int32_t len;
+        const uint8_t *vstr;
+        if (janet_checktype(argv[i], JANET_BUFFER)) {
+            JanetBuffer *b = janet_unwrap_buffer(argv[i]);
+            vstr = b->data;
+            len = b->count;
+        } else {
+            vstr = janet_to_string(argv[i]);
+            len = janet_string_length(vstr);
+        }
+        if (len) {
+            if (1 != fwrite(vstr, len, 1, f)) {
+                janet_panicf("could not print %d bytes to (dyn :%s)", len, name);
+            }
+        }
+    }
+    if (newline)
+        putc('\n', f);
+    return janet_wrap_nil();
+}
+
+static Janet cfun_io_print(int32_t argc, Janet *argv) {
+    return cfun_io_print_impl(argc, argv, 1, "out", stdout);
+}
+
+static Janet cfun_io_prin(int32_t argc, Janet *argv) {
+    return cfun_io_print_impl(argc, argv, 0, "out", stdout);
+}
+
+static Janet cfun_io_eprint(int32_t argc, Janet *argv) {
+    return cfun_io_print_impl(argc, argv, 1, "err", stderr);
+}
+
+static Janet cfun_io_eprin(int32_t argc, Janet *argv) {
+    return cfun_io_print_impl(argc, argv, 0, "err", stderr);
+}
+
+static Janet cfun_io_printf_impl(int32_t argc, Janet *argv, int newline,
+                                 const char *name, FILE *dflt_file) {
+    FILE *f;
+    janet_arity(argc, 1, -1);
+    const char *fmt = janet_getcstring(argv, 0);
+    Janet x = janet_dyn(name);
+    switch (janet_type(x)) {
+        default:
+            /* Other values simply do nothing */
+            return janet_wrap_nil();
+        case JANET_BUFFER: {
+            /* Special case buffer */
+            JanetBuffer *buf = janet_unwrap_buffer(x);
+            janet_buffer_format(buf, fmt, 0, argc, argv);
+            if (newline) janet_buffer_push_u8(buf, '\n');
+            return janet_wrap_nil();
+        }
+        case JANET_NIL:
+            f = dflt_file;
+            break;
+        case JANET_ABSTRACT: {
+            void *abstract = janet_unwrap_abstract(x);
+            if (janet_abstract_type(abstract) != &cfun_io_filetype)
+                return janet_wrap_nil();
+            IOFile *iofile = abstract;
+            f = iofile->file;
+            break;
+        }
+    }
+    JanetBuffer *buf = janet_buffer(10);
+    janet_buffer_format(buf, fmt, 0, argc, argv);
+    if (newline) janet_buffer_push_u8(buf, '\n');
+    if (buf->count) {
+        if (1 != fwrite(buf->data, buf->count, 1, f)) {
+            janet_panicf("could not print %d bytes to file", buf->count, name);
+        }
+    }
+    /* Clear buffer to make things easier for GC */
+    buf->count = 0;
+    buf->capacity = 0;
+    free(buf->data);
+    buf->data = NULL;
+    return janet_wrap_nil();
+}
+
+static Janet cfun_io_printf(int32_t argc, Janet *argv) {
+    return cfun_io_printf_impl(argc, argv, 1, "out", stdout);
+}
+
+static Janet cfun_io_prinf(int32_t argc, Janet *argv) {
+    return cfun_io_printf_impl(argc, argv, 0, "out", stdout);
+}
+
+static Janet cfun_io_eprintf(int32_t argc, Janet *argv) {
+    return cfun_io_printf_impl(argc, argv, 1, "err", stderr);
+}
+
+static Janet cfun_io_eprinf(int32_t argc, Janet *argv) {
+    return cfun_io_printf_impl(argc, argv, 0, "err", stderr);
+}
+
+void janet_dynprintf(const char *name, FILE *dflt_file, const char *format, ...) {
+    va_list args;
+    va_start(args, format);
+    Janet x = janet_dyn(name);
+    JanetType xtype = janet_type(x);
+    switch (xtype) {
+        default:
+            /* Other values simply do nothing */
+            break;
+        case JANET_NIL:
+        case JANET_ABSTRACT: {
+            FILE *f = dflt_file;
+            JanetBuffer buffer;
+            int32_t len = 0;
+            while (format[len]) len++;
+            janet_buffer_init(&buffer, len);
+            janet_formatb(&buffer, format, args);
+            if (xtype == JANET_ABSTRACT) {
+                void *abstract = janet_unwrap_abstract(x);
+                if (janet_abstract_type(abstract) != &cfun_io_filetype)
+                    break;
+                IOFile *iofile = abstract;
+                f = iofile->file;
+            }
+            fwrite(buffer.data, buffer.count, 1, f);
+            janet_buffer_deinit(&buffer);
+            break;
+        }
+        case JANET_BUFFER:
+            janet_formatb(janet_unwrap_buffer(x), format, args);
+            break;
+    }
+    va_end(args);
+    return;
+}
+
 static const JanetReg io_cfuns[] = {
+    {
+        "print", cfun_io_print,
+        JDOC("(print & xs)\n\n"
+             "Print values to the console (standard out). Value are converted "
+             "to strings if they are not already. After printing all values, a "
+             "newline character is printed. Use the value of (dyn :out stdout) to determine "
+             "what to push characters to. Expects (dyn :out stdout) to be either a core/file or "
+             "a buffer. Returns nil.")
+    },
+    {
+        "prin", cfun_io_prin,
+        JDOC("(prin & xs)\n\n"
+             "Same as print, but does not add trailing newline.")
+    },
+    {
+        "printf", cfun_io_printf,
+        JDOC("(printf fmt & xs)\n\n"
+             "Prints output formatted as if with (string/format fmt ;xs) to (dyn :out stdout) with a trailing newline.")
+    },
+    {
+        "prinf", cfun_io_prinf,
+        JDOC("(prinf fmt & xs)\n\n"
+             "Like printf but with no trailing newline.")
+    },
+    {
+        "eprin", cfun_io_eprin,
+        JDOC("(eprin & xs)\n\n"
+             "Same as prin, but uses (dyn :err stderr) instead of (dyn :out stdout).")
+    },
+    {
+        "eprint", cfun_io_eprint,
+        JDOC("(eprint & xs)\n\n"
+             "Same as print, but uses (dyn :err stderr) instead of (dyn :out stdout).")
+    },
+    {
+        "eprintf", cfun_io_eprintf,
+        JDOC("(eprintf fmt & xs)\n\n"
+             "Prints output formatted as if with (string/format fmt ;xs) to (dyn :err stderr) with a trailing newline.")
+    },
+    {
+        "eprinf", cfun_io_eprinf,
+        JDOC("(eprinf fmt & xs)\n\n"
+             "Like eprintf but with no trailing newline.")
+    },
     {
         "file/open", cfun_io_fopen,
-        JDOC("(file/open path [,mode])\n\n"
-                "Open a file. path is an absolute or relative path, and "
-                "mode is a set of flags indicating the mode to open the file in. "
-                "mode is a keyword where each character represents a flag. If the file "
-                "cannot be opened, returns nil, otherwise returns the new file handle. "
-                "Mode flags:\n\n"
-                "\tr - allow reading from the file\n"
-                "\tw - allow writing to the file\n"
-                "\ta - append to the file\n"
-                "\tb - open the file in binary mode (rather than text mode)\n"
-                "\t+ - append to the file instead of overwriting it")
+        JDOC("(file/open path &opt mode)\n\n"
+             "Open a file. path is an absolute or relative path, and "
+             "mode is a set of flags indicating the mode to open the file in. "
+             "mode is a keyword where each character represents a flag. If the file "
+             "cannot be opened, returns nil, otherwise returns the new file handle. "
+             "Mode flags:\n\n"
+             "\tr - allow reading from the file\n"
+             "\tw - allow writing to the file\n"
+             "\ta - append to the file\n"
+             "\tb - open the file in binary mode (rather than text mode)\n"
+             "\t+ - append to the file instead of overwriting it")
+    },
+    {
+        "file/fdopen", cfun_io_fdopen,
+        JDOC("(file/fdopen fd &opt mode)\n\n"
+             "Create a file from an fd. fd is a platform specific file descriptor, and "
+             "mode is a set of flags indicating the mode to open the file in. "
+             "mode is a keyword where each character represents a flag. If the file "
+             "cannot be opened, returns nil, otherwise returns the new file handle. "
+             "Mode flags:\n\n"
+             "\tr - allow reading from the file\n"
+             "\tw - allow writing to the file\n"
+             "\ta - append to the file\n"
+             "\tb - open the file in binary mode (rather than text mode)\n"
+             "\t+ - append to the file instead of overwriting it")
+    },
+    {
+        "file/fileno", cfun_io_fileno,
+        JDOC("(file/fileno f)\n\n"
+             "Return the underlying file descriptor for the file as a number."
+             "The meaning of this number is platform specific.")
     },
     {
         "file/close", cfun_io_fclose,
         JDOC("(file/close f)\n\n"
-                "Close a file and release all related resources. When you are "
-                "done reading a file, close it to prevent a resource leak and let "
-                "other processes read the file.")
+             "Close a file and release all related resources. When you are "
+             "done reading a file, close it to prevent a resource leak and let "
+             "other processes read the file.")
     },
     {
         "file/read", cfun_io_fread,
-        JDOC("(file/read f what [,buf])\n\n"
-                "Read a number of bytes from a file into a buffer. A buffer can "
-                "be provided as an optional fourth argument, otherwise a new buffer "
-                "is created. 'what' can either be an integer or a keyword. Returns the "
-                "buffer with file contents. "
-                "Values for 'what':\n\n"
-                "\t:all - read the whole file\n"
-                "\t:line - read up to and including the next newline character\n"
-                "\tn (integer) - read up to n bytes from the file")
+        JDOC("(file/read f what &opt buf)\n\n"
+             "Read a number of bytes from a file into a buffer. A buffer can "
+             "be provided as an optional fourth argument, otherwise a new buffer "
+             "is created. 'what' can either be an integer or a keyword. Returns the "
+             "buffer with file contents. "
+             "Values for 'what':\n\n"
+             "\t:all - read the whole file\n"
+             "\t:line - read up to and including the next newline character\n"
+             "\tn (integer) - read up to n bytes from the file")
     },
     {
         "file/write", cfun_io_fwrite,
         JDOC("(file/write f bytes)\n\n"
-                "Writes to a file. 'bytes' must be string, buffer, or symbol. Returns the "
-                "file.")
+             "Writes to a file. 'bytes' must be string, buffer, or symbol. Returns the "
+             "file.")
     },
     {
         "file/flush", cfun_io_fflush,
         JDOC("(file/flush f)\n\n"
-                "Flush any buffered bytes to the file system. In most files, writes are "
-                "buffered for efficiency reasons. Returns the file handle.")
+             "Flush any buffered bytes to the file system. In most files, writes are "
+             "buffered for efficiency reasons. Returns the file handle.")
     },
     {
         "file/seek", cfun_io_fseek,
-        JDOC("(file/seek f [,whence [,n]])\n\n"
-                "Jump to a relative location in the file. 'whence' must be one of\n\n"
-                "\t:cur - jump relative to the current file location\n"
-                "\t:set - jump relative to the beginning of the file\n"
-                "\t:end - jump relative to the end of the file\n\n"
-                "By default, 'whence' is :cur. Optionally a value n may be passed "
-                "for the relative number of bytes to seek in the file. n may be a real "
-                "number to handle large files of more the 4GB. Returns the file handle.")
+        JDOC("(file/seek f &opt whence n)\n\n"
+             "Jump to a relative location in the file. 'whence' must be one of\n\n"
+             "\t:cur - jump relative to the current file location\n"
+             "\t:set - jump relative to the beginning of the file\n"
+             "\t:end - jump relative to the end of the file\n\n"
+             "By default, 'whence' is :cur. Optionally a value n may be passed "
+             "for the relative number of bytes to seek in the file. n may be a real "
+             "number to handle large files of more the 4GB. Returns the file handle.")
     },
     {
         "file/popen", cfun_io_popen,
-        JDOC("(file/popen path [,mode])\n\n"
-                "Open a file that is backed by a process. The file must be opened in either "
-                "the :r (read) or the :w (write) mode. In :r mode, the stdout of the "
-                "process can be read from the file. In :w mode, the stdin of the process "
-                "can be written to. Returns the new file.")
+        JDOC("(file/popen path &opt mode)\n\n"
+             "Open a file that is backed by a process. The file must be opened in either "
+             "the :r (read) or the :w (write) mode. In :r mode, the stdout of the "
+             "process can be read from the file. In :w mode, the stdin of the process "
+             "can be written to. Returns the new file.")
     },
     {NULL, NULL, NULL}
 };
 
+/* C API */
+
+FILE *janet_getfile(const Janet *argv, int32_t n, int *flags) {
+    IOFile *iof = janet_getabstract(argv, n, &cfun_io_filetype);
+    if (NULL != flags) *flags = iof->flags;
+    return iof->file;
+}
+
 /* Module entry point */
 void janet_lib_io(JanetTable *env) {
-    janet_cfuns(env, NULL, io_cfuns);
+    janet_core_cfuns(env, NULL, io_cfuns);
 
     /* stdout */
-    janet_def(env, "stdout",
-            makef(stdout, IO_APPEND | IO_NOT_CLOSEABLE | IO_SERIALIZABLE),
-            JDOC("The standard output file."));
+    janet_core_def(env, "stdout",
+                   makef(stdout, IO_APPEND | IO_NOT_CLOSEABLE | IO_SERIALIZABLE),
+                   JDOC("The standard output file."));
     /* stderr */
-    janet_def(env, "stderr",
-            makef(stderr, IO_APPEND | IO_NOT_CLOSEABLE | IO_SERIALIZABLE),
-            JDOC("The standard error file."));
+    janet_core_def(env, "stderr",
+                   makef(stderr, IO_APPEND | IO_NOT_CLOSEABLE | IO_SERIALIZABLE),
+                   JDOC("The standard error file."));
     /* stdin */
-    janet_def(env, "stdin",
-            makef(stdin, IO_READ | IO_NOT_CLOSEABLE | IO_SERIALIZABLE),
-            JDOC("The standard input file."));
+    janet_core_def(env, "stdin",
+                   makef(stdin, IO_READ | IO_NOT_CLOSEABLE | IO_SERIALIZABLE),
+                   JDOC("The standard input file."));
+
 }

src/core/math.c

@@ -23,27 +23,206 @@
 #include <math.h>
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "util.h"
 #endif
 
+static JANET_THREAD_LOCAL JanetRNG janet_vm_rng = {0, 0, 0, 0, 0};
+
+static int janet_rng_get(void *p, Janet key, Janet *out);
+
+static void janet_rng_marshal(void *p, JanetMarshalContext *ctx) {
+    JanetRNG *rng = (JanetRNG *)p;
+    janet_marshal_abstract(ctx, p);
+    janet_marshal_int(ctx, (int32_t) rng->a);
+    janet_marshal_int(ctx, (int32_t) rng->b);
+    janet_marshal_int(ctx, (int32_t) rng->c);
+    janet_marshal_int(ctx, (int32_t) rng->d);
+    janet_marshal_int(ctx, (int32_t) rng->counter);
+}
+
+static void *janet_rng_unmarshal(JanetMarshalContext *ctx) {
+    JanetRNG *rng = janet_unmarshal_abstract(ctx, sizeof(JanetRNG));
+    rng->a = (uint32_t) janet_unmarshal_int(ctx);
+    rng->b = (uint32_t) janet_unmarshal_int(ctx);
+    rng->c = (uint32_t) janet_unmarshal_int(ctx);
+    rng->d = (uint32_t) janet_unmarshal_int(ctx);
+    rng->counter = (uint32_t) janet_unmarshal_int(ctx);
+    return rng;
+}
+
+static JanetAbstractType JanetRNG_type = {
+    "core/rng",
+    NULL,
+    NULL,
+    janet_rng_get,
+    NULL,
+    janet_rng_marshal,
+    janet_rng_unmarshal,
+    NULL
+};
+
+JanetRNG *janet_default_rng(void) {
+    return &janet_vm_rng;
+}
+
+void janet_rng_seed(JanetRNG *rng, uint32_t seed) {
+    rng->a = seed;
+    rng->b = 0x97654321u;
+    rng->c = 123871873u;
+    rng->d = 0xf23f56c8u;
+    rng->counter = 0u;
+    /* First several numbers aren't that random. */
+    for (int i = 0; i < 16; i++) janet_rng_u32(rng);
+}
+
+void janet_rng_longseed(JanetRNG *rng, const uint8_t *bytes, int32_t len) {
+    uint8_t state[16] = {0};
+    for (int32_t i = 0; i < len; i++)
+        state[i & 0xF] ^= bytes[i];
+    rng->a = state[0] + (state[1] << 8) + (state[2] << 16) + (state[3] << 24);
+    rng->b = state[4] + (state[5] << 8) + (state[6] << 16) + (state[7] << 24);
+    rng->c = state[8] + (state[9] << 8) + (state[10] << 16) + (state[11] << 24);
+    rng->d = state[12] + (state[13] << 8) + (state[14] << 16) + (state[15] << 24);
+    rng->counter = 0u;
+    /* a, b, c, d can't all be 0 */
+    if (rng->a == 0) rng->a = 1u;
+    for (int i = 0; i < 16; i++) janet_rng_u32(rng);
+}
+
+uint32_t janet_rng_u32(JanetRNG *rng) {
+    /* Algorithm "xorwow" from p. 5 of Marsaglia, "Xorshift RNGs" */
+    uint32_t t = rng->d;
+    uint32_t const s = rng->a;
+    rng->d = rng->c;
+    rng->c = rng->b;
+    rng->b = s;
+    t ^= t >> 2;
+    t ^= t << 1;
+    t ^= s ^ (s << 4);
+    rng->a = t;
+    rng->counter += 362437;
+    return t + rng->counter;
+}
+
+double janet_rng_double(JanetRNG *rng) {
+    uint32_t hi = janet_rng_u32(rng);
+    uint32_t lo = janet_rng_u32(rng);
+    uint64_t big = (uint64_t)(lo) | (((uint64_t) hi) << 32);
+    return ldexp((double)(big >> (64 - 52)), -52);
+}
+
+static Janet cfun_rng_make(int32_t argc, Janet *argv) {
+    janet_arity(argc, 0, 1);
+    JanetRNG *rng = janet_abstract(&JanetRNG_type, sizeof(JanetRNG));
+    if (argc == 1) {
+        if (janet_checkint(argv[0])) {
+            uint32_t seed = (uint32_t)(janet_getinteger(argv, 0));
+            janet_rng_seed(rng, seed);
+        } else {
+            JanetByteView bytes = janet_getbytes(argv, 0);
+            janet_rng_longseed(rng, bytes.bytes, bytes.len);
+        }
+    } else {
+        janet_rng_seed(rng, 0);
+    }
+    return janet_wrap_abstract(rng);
+}
+
+static Janet cfun_rng_uniform(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    JanetRNG *rng = janet_getabstract(argv, 0, &JanetRNG_type);
+    return janet_wrap_number(janet_rng_double(rng));
+}
+
+static Janet cfun_rng_int(int32_t argc, Janet *argv) {
+    janet_arity(argc, 1, 2);
+    JanetRNG *rng = janet_getabstract(argv, 0, &JanetRNG_type);
+    if (argc == 1) {
+        uint32_t word = janet_rng_u32(rng) >> 1;
+        return janet_wrap_integer(word);
+    } else {
+        int32_t max = janet_optnat(argv, argc, 1, INT32_MAX);
+        if (max == 0) return janet_wrap_number(0.0);
+        uint32_t modulo = (uint32_t) max;
+        uint32_t maxgen = INT32_MAX;
+        uint32_t maxword = maxgen - (maxgen % modulo);
+        uint32_t word;
+        do {
+            word = janet_rng_u32(rng) >> 1;
+        } while (word > maxword);
+        return janet_wrap_integer(word % modulo);
+    }
+}
+
+static void rng_get_4bytes(JanetRNG *rng, uint8_t *buf) {
+    uint32_t word = janet_rng_u32(rng);
+    buf[0] = word & 0xFF;
+    buf[1] = (word >> 8) & 0xFF;
+    buf[2] = (word >> 16) & 0xFF;
+    buf[3] = (word >> 24) & 0xFF;
+}
+
+static Janet cfun_rng_buffer(int32_t argc, Janet *argv) {
+    janet_arity(argc, 2, 3);
+    JanetRNG *rng = janet_getabstract(argv, 0, &JanetRNG_type);
+    int32_t n = janet_getnat(argv, 1);
+    JanetBuffer *buffer = janet_optbuffer(argv, argc, 2, n);
+
+    /* Split into first part (that is divisible by 4), and rest */
+    int32_t first_part = n & ~3;
+    int32_t second_part = n - first_part;
+
+    /* Get first part in chunks of 4 bytes */
+    janet_buffer_extra(buffer, n);
+    uint8_t *buf = buffer->data + buffer->count;
+    for (int32_t i = 0; i < first_part; i += 4) rng_get_4bytes(rng, buf + i);
+    buffer->count += first_part;
+
+    /* Get remaining 0 - 3 bytes */
+    if (second_part) {
+        uint8_t wordbuf[4] = {0};
+        rng_get_4bytes(rng, wordbuf);
+        janet_buffer_push_bytes(buffer, wordbuf, second_part);
+    }
+
+    return janet_wrap_buffer(buffer);
+}
+
+static const JanetMethod rng_methods[] = {
+    {"uniform", cfun_rng_uniform},
+    {"int", cfun_rng_int},
+    {"buffer", cfun_rng_buffer},
+    {NULL, NULL}
+};
+
+static int janet_rng_get(void *p, Janet key, Janet *out) {
+    (void) p;
+    if (!janet_checktype(key, JANET_KEYWORD)) return 0;
+    return janet_getmethod(janet_unwrap_keyword(key), rng_methods, out);
+}
+
 /* Get a random number */
-Janet janet_rand(int32_t argc, Janet *argv) {
+static Janet janet_rand(int32_t argc, Janet *argv) {
     (void) argv;
     janet_fixarity(argc, 0);
-    double r = (rand() % RAND_MAX) / ((double) RAND_MAX);
-    return janet_wrap_number(r);
+    return janet_wrap_number(janet_rng_double(&janet_vm_rng));
 }
 
 /* Seed the random number generator */
-Janet janet_srand(int32_t argc, Janet *argv) {
+static Janet janet_srand(int32_t argc, Janet *argv) {
     janet_fixarity(argc, 1);
-    int32_t x = janet_getinteger(argv, 0);
-    srand((unsigned) x);
+    if (janet_checkint(argv[0])) {
+        uint32_t seed = (uint32_t)(janet_getinteger(argv, 0));
+        janet_rng_seed(&janet_vm_rng, seed);
+    } else {
+        JanetByteView bytes = janet_getbytes(argv, 0);
+        janet_rng_longseed(&janet_vm_rng, bytes.bytes, bytes.len);
+    }
     return janet_wrap_nil();
 }
 
-Janet janet_remainder(int32_t argc, Janet *argv) {
+static Janet janet_remainder(int32_t argc, Janet *argv) {
     janet_fixarity(argc, 2);
     double x = janet_getnumber(argv, 0);
     double y = janet_getnumber(argv, 1);
@@ -51,7 +230,7 @@ Janet janet_remainder(int32_t argc, Janet *argv) {
 }
 
 #define JANET_DEFINE_MATHOP(name, fop)\
-Janet janet_##name(int32_t argc, Janet *argv) {\
+static Janet janet_##name(int32_t argc, Janet *argv) {\
     janet_fixarity(argc, 1); \
     double x = janet_getnumber(argv, 0); \
     return janet_wrap_number(fop(x)); \
@@ -62,20 +241,29 @@ JANET_DEFINE_MATHOP(asin, asin)
 JANET_DEFINE_MATHOP(atan, atan)
 JANET_DEFINE_MATHOP(cos, cos)
 JANET_DEFINE_MATHOP(cosh, cosh)
+JANET_DEFINE_MATHOP(acosh, acosh)
 JANET_DEFINE_MATHOP(sin, sin)
 JANET_DEFINE_MATHOP(sinh, sinh)
+JANET_DEFINE_MATHOP(asinh, asinh)
 JANET_DEFINE_MATHOP(tan, tan)
 JANET_DEFINE_MATHOP(tanh, tanh)
+JANET_DEFINE_MATHOP(atanh, atanh)
 JANET_DEFINE_MATHOP(exp, exp)
+JANET_DEFINE_MATHOP(exp2, exp2)
+JANET_DEFINE_MATHOP(expm1, expm1)
 JANET_DEFINE_MATHOP(log, log)
 JANET_DEFINE_MATHOP(log10, log10)
+JANET_DEFINE_MATHOP(log2, log2)
 JANET_DEFINE_MATHOP(sqrt, sqrt)
+JANET_DEFINE_MATHOP(cbrt, cbrt)
 JANET_DEFINE_MATHOP(ceil, ceil)
 JANET_DEFINE_MATHOP(fabs, fabs)
 JANET_DEFINE_MATHOP(floor, floor)
+JANET_DEFINE_MATHOP(trunc, trunc)
+JANET_DEFINE_MATHOP(round, round)
 
 #define JANET_DEFINE_MATH2OP(name, fop)\
-Janet janet_##name(int32_t argc, Janet *argv) {\
+static Janet janet_##name(int32_t argc, Janet *argv) {\
     janet_fixarity(argc, 2); \
     double lhs = janet_getnumber(argv, 0); \
     double rhs = janet_getnumber(argv, 1); \
@@ -84,6 +272,7 @@ Janet janet_##name(int32_t argc, Janet *argv) {\
 
 JANET_DEFINE_MATH2OP(atan2, atan2)
 JANET_DEFINE_MATH2OP(pow, pow)
+JANET_DEFINE_MATH2OP(hypot, hypot)
 
 static Janet janet_not(int32_t argc, Janet *argv) {
     janet_fixarity(argc, 1);
@@ -94,7 +283,7 @@ static const JanetReg math_cfuns[] = {
     {
         "%", janet_remainder,
         JDOC("(% dividend divisor)\n\n"
-                "Returns the remainder of dividend / divisor.")
+             "Returns the remainder of dividend / divisor.")
     },
     {
         "not", janet_not,
@@ -103,91 +292,193 @@ static const JanetReg math_cfuns[] = {
     {
         "math/random", janet_rand,
         JDOC("(math/random)\n\n"
-                "Returns a uniformly distributed random number between 0 and 1.")
+             "Returns a uniformly distributed random number between 0 and 1.")
     },
     {
         "math/seedrandom", janet_srand,
         JDOC("(math/seedrandom seed)\n\n"
-                "Set the seed for the random number generator. 'seed' should be an "
-                "an integer.")
+             "Set the seed for the random number generator. seed should be "
+             "an integer or a buffer.")
     },
     {
         "math/cos", janet_cos,
         JDOC("(math/cos x)\n\n"
-                "Returns the cosine of x.")
+             "Returns the cosine of x.")
     },
     {
         "math/sin", janet_sin,
         JDOC("(math/sin x)\n\n"
-                "Returns the sine of x.")
+             "Returns the sine of x.")
     },
     {
         "math/tan", janet_tan,
         JDOC("(math/tan x)\n\n"
-                "Returns the tangent of x.")
+             "Returns the tangent of x.")
     },
     {
         "math/acos", janet_acos,
         JDOC("(math/acos x)\n\n"
-                "Returns the arccosine of x.")
+             "Returns the arccosine of x.")
     },
     {
         "math/asin", janet_asin,
         JDOC("(math/asin x)\n\n"
-                "Returns the arcsine of x.")
+             "Returns the arcsine of x.")
     },
     {
         "math/atan", janet_atan,
         JDOC("(math/atan x)\n\n"
-                "Returns the arctangent of x.")
+             "Returns the arctangent of x.")
     },
     {
         "math/exp", janet_exp,
         JDOC("(math/exp x)\n\n"
-                "Returns e to the power of x.")
+             "Returns e to the power of x.")
     },
     {
         "math/log", janet_log,
         JDOC("(math/log x)\n\n"
-                "Returns log base 2 of x.")
+             "Returns log base natural number of x.")
     },
     {
         "math/log10", janet_log10,
         JDOC("(math/log10 x)\n\n"
-                "Returns log base 10 of x.")
+             "Returns log base 10 of x.")
+    },
+    {
+        "math/log2", janet_log2,
+        JDOC("(math/log2 x)\n\n"
+             "Returns log base 2 of x.")
     },
     {
         "math/sqrt", janet_sqrt,
         JDOC("(math/sqrt x)\n\n"
-                "Returns the square root of x.")
+             "Returns the square root of x.")
+    },
+    {
+        "math/cbrt", janet_cbrt,
+        JDOC("(math/cbrt x)\n\n"
+             "Returns the cube root of x.")
     },
     {
         "math/floor", janet_floor,
         JDOC("(math/floor x)\n\n"
-                "Returns the largest integer value number that is not greater than x.")
+             "Returns the largest integer value number that is not greater than x.")
     },
     {
         "math/ceil", janet_ceil,
         JDOC("(math/ceil x)\n\n"
-                "Returns the smallest integer value number that is not less than x.")
+             "Returns the smallest integer value number that is not less than x.")
     },
     {
         "math/pow", janet_pow,
         JDOC("(math/pow a x)\n\n"
-                "Return a to the power of x.")
+             "Return a to the power of x.")
+    },
+    {
+        "math/abs", janet_fabs,
+        JDOC("(math/abs x)\n\n"
+             "Return the absolute value of x.")
+    },
+    {
+        "math/sinh", janet_sinh,
+        JDOC("(math/sinh x)\n\n"
+             "Return the hyperbolic sine of x.")
+    },
+    {
+        "math/cosh", janet_cosh,
+        JDOC("(math/cosh x)\n\n"
+             "Return the hyperbolic cosine of x.")
+    },
+    {
+        "math/tanh", janet_tanh,
+        JDOC("(math/tanh x)\n\n"
+             "Return the hyperbolic tangent of x.")
+    },
+    {
+        "math/atanh", janet_atanh,
+        JDOC("(math/atanh x)\n\n"
+             "Return the hyperbolic arctangent of x.")
+    },
+    {
+        "math/asinh", janet_asinh,
+        JDOC("(math/asinh x)\n\n"
+             "Return the hyperbolic arcsine of x.")
+    },
+    {
+        "math/acosh", janet_acosh,
+        JDOC("(math/acosh x)\n\n"
+             "Return the hyperbolic arccosine of x.")
+    },
+    {
+        "math/atan2", janet_atan2,
+        JDOC("(math/atan2 y x)\n\n"
+             "Return the arctangent of y/x. Works even when x is 0.")
+    },
+    {
+        "math/rng", cfun_rng_make,
+        JDOC("(math/rng &opt seed)\n\n"
+             "Creates a Psuedo-Random number generator, with an optional seed. "
+             "The seed should be an unsigned 32 bit integer. "
+             "Do not use this for cryptography. Returns a core/rng abstract type.")
+    },
+    {
+        "math/rng-uniform", cfun_rng_uniform,
+        JDOC("(math/rng-seed rng seed)\n\n"
+             "Extract a random number in the range [0, 1) from the RNG.")
+    },
+    {
+        "math/rng-int", cfun_rng_int,
+        JDOC("(math/rng-int rng &opt max)\n\n"
+             "Extract a random random integer in the range [0, max] from the RNG. If "
+             "no max is given, the default is 2^31 - 1.")
+    },
+    {
+        "math/rng-buffer", cfun_rng_buffer,
+        JDOC("(math/rng-buffer rng n &opt buf)\n\n"
+             "Get n random bytes and put them in a buffer. Creates a new buffer if no buffer is "
+             "provided, otherwise appends to the given buffer. Returns the buffer.")
+    },
+    {
+        "math/hypot", janet_hypot,
+        JDOC("(math/hypot a b)\n\n"
+             "Returns the c from the equation c^2 = a^2 + b^2")
+    },
+    {
+        "math/exp2", janet_exp2,
+        JDOC("(math/exp2 x)\n\n"
+             "Returns 2 to the power of x.")
+    },
+    {
+        "math/expm1", janet_expm1,
+        JDOC("(math/expm1 x)\n\n"
+             "Returns e to the power of x minus 1.")
+    },
+    {
+        "math/trunc", janet_trunc,
+        JDOC("(math/trunc x)\n\n"
+             "Returns the integer between x and 0 nearest to x.")
+    },
+    {
+        "math/round", janet_round,
+        JDOC("(math/round x)\n\n"
+             "Returns the integer nearest to x.")
     },
     {NULL, NULL, NULL}
 };
 
 /* Module entry point */
 void janet_lib_math(JanetTable *env) {
-    janet_cfuns(env, NULL, math_cfuns);
-#ifndef JANET_NO_BOOTSTRAP
+    janet_core_cfuns(env, NULL, math_cfuns);
+    janet_register_abstract_type(&JanetRNG_type);
+#ifdef JANET_BOOTSTRAP
     janet_def(env, "math/pi", janet_wrap_number(3.1415926535897931),
-            JDOC("The value pi."));
+              JDOC("The value pi."));
     janet_def(env, "math/e", janet_wrap_number(2.7182818284590451),
-            JDOC("The base of the natural log."));
+              JDOC("The base of the natural log."));
     janet_def(env, "math/inf", janet_wrap_number(INFINITY),
-            JDOC("The number representing positive infinity"));
+              JDOC("The number representing positive infinity"));
+    janet_def(env, "math/-inf", janet_wrap_number(-INFINITY),
+              JDOC("The number representing negative infinity"));
 #endif
 }

src/core/parse.c

@@ -21,34 +21,35 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "util.h"
 #endif
 
 /* Check if a character is whitespace */
 static int is_whitespace(uint8_t c) {
     return c == ' '
-        || c == '\t'
-        || c == '\n'
-        || c == '\r'
-        || c == '\0'
-        || c == '\f';
+           || c == '\t'
+           || c == '\n'
+           || c == '\r'
+           || c == '\0'
+           || c == '\v'
+           || c == '\f';
 }
 
 /* Code generated by tools/symcharsgen.c.
  * The table contains 256 bits, where each bit is 1
- * if the corresponding ascci code is a symbol char, and 0
+ * if the corresponding ascii code is a symbol char, and 0
  * if not. The upper characters are also considered symbol
  * chars and are then checked for utf-8 compliance. */
 static const uint32_t symchars[8] = {
-    0x00000000, 0xf7ffec72, 0xc7ffffff, 0x17fffffe,
+    0x00000000, 0xf7ffec72, 0xc7ffffff, 0x07fffffe,
     0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff
 };
 
 /* Check if a character is a valid symbol character
  * symbol chars are A-Z, a-z, 0-9, or one of !$&*+-./:<=>@\^_~| */
 static int is_symbol_char(uint8_t c) {
-    return symchars[c >> 5] & (1 << (c & 0x1F));
+    return symchars[c >> 5] & ((uint32_t)1 << (c & 0x1F));
 }
 
 /* Validate some utf8. Useful for identifiers. Only validates
@@ -105,7 +106,8 @@ struct JanetParseState {
     int32_t counter;
     int32_t argn;
     int flags;
-    size_t start;
+    size_t line;
+    size_t column;
     Consumer consumer;
 };
 
@@ -143,6 +145,8 @@ DEF_PARSER_STACK(_pushstate, JanetParseState, states, statecount, statecap)
 #define PFLAG_LONGSTRING 0x4000
 #define PFLAG_READERMAC 0x8000
 #define PFLAG_ATSYM 0x10000
+#define PFLAG_COMMENT 0x20000
+#define PFLAG_TOKEN 0x40000
 
 static void pushstate(JanetParser *p, Consumer consumer, int flags) {
     JanetParseState s;
@@ -150,7 +154,8 @@ static void pushstate(JanetParser *p, Consumer consumer, int flags) {
     s.argn = 0;
     s.flags = flags;
     s.consumer = consumer;
-    s.start = p->offset;
+    s.line = p->line;
+    s.column = p->column;
     _pushstate(p, s);
 }
 
@@ -161,8 +166,8 @@ static void popstate(JanetParser *p, Janet val) {
         if (newtop->flags & PFLAG_CONTAINER) {
             /* Source mapping info */
             if (janet_checktype(val, JANET_TUPLE)) {
-                janet_tuple_sm_start(janet_unwrap_tuple(val)) = (int32_t) top.start;
-                janet_tuple_sm_end(janet_unwrap_tuple(val)) = (int32_t) p->offset;
+                janet_tuple_sm_line(janet_unwrap_tuple(val)) = (int32_t) top.line;
+                janet_tuple_sm_column(janet_unwrap_tuple(val)) = (int32_t) top.column;
             }
             newtop->argn++;
             /* Keep track of number of values in the root state */
@@ -176,12 +181,13 @@ static void popstate(JanetParser *p, Janet val) {
                 (c == '\'') ? "quote" :
                 (c == ',') ? "unquote" :
                 (c == ';') ? "splice" :
+                (c == '|') ? "short-fn" :
                 (c == '~') ? "quasiquote" : "<unknown>";
             t[0] = janet_csymbolv(which);
             t[1] = val;
             /* Quote source mapping info */
-            janet_tuple_sm_start(t) = (int32_t) newtop->start;
-            janet_tuple_sm_end(t) = (int32_t) p->offset;
+            janet_tuple_sm_line(t) = (int32_t) newtop->line;
+            janet_tuple_sm_column(t) = (int32_t) newtop->column;
             val = janet_wrap_tuple(janet_tuple_end(t));
         } else {
             return;
@@ -191,17 +197,30 @@ static void popstate(JanetParser *p, Janet val) {
 
 static int checkescape(uint8_t c) {
     switch (c) {
-        default: return -1;
-        case 'x': return 1;
-        case 'n': return '\n';
-        case 't': return '\t';
-        case 'r': return '\r';
-        case '0': return '\0';
-        case 'z': return '\0';
-        case 'f': return '\f';
-        case 'e': return 27;
-        case '"': return '"';
-        case '\\': return '\\';
+        default:
+            return -1;
+        case 'x':
+            return 1;
+        case 'n':
+            return '\n';
+        case 't':
+            return '\t';
+        case 'r':
+            return '\r';
+        case '0':
+            return '\0';
+        case 'z':
+            return '\0';
+        case 'f':
+            return '\f';
+        case 'v':
+            return '\v';
+        case 'e':
+            return 27;
+        case '"':
+            return '"';
+        case '\\':
+            return '\\';
     }
 }
 
@@ -214,7 +233,7 @@ static int escapeh(JanetParser *p, JanetParseState *state, uint8_t c) {
         p->error = "invalid hex digit in hex escape";
         return 1;
     }
-    state->argn = (state->argn << 4) + digit;;
+    state->argn = (state->argn << 4) + digit;
     state->counter--;
     if (!state->counter) {
         push_buf(p, (state->argn & 0xFF));
@@ -243,12 +262,24 @@ static int escape1(JanetParser *p, JanetParseState *state, uint8_t c) {
 
 static int stringend(JanetParser *p, JanetParseState *state) {
     Janet ret;
+    uint8_t *bufstart = p->buf;
+    int32_t buflen = (int32_t) p->bufcount;
+    if (state->flags & PFLAG_LONGSTRING) {
+        /* Check for leading newline character so we can remove it */
+        if (bufstart[0] == '\n') {
+            bufstart++;
+            buflen--;
+        }
+        if (buflen > 0 && bufstart[buflen - 1] == '\n') {
+            buflen--;
+        }
+    }
     if (state->flags & PFLAG_BUFFER) {
-        JanetBuffer *b = janet_buffer((int32_t)p->bufcount);
-        janet_buffer_push_bytes(b, p->buf, (int32_t)p->bufcount);
+        JanetBuffer *b = janet_buffer(buflen);
+        janet_buffer_push_bytes(b, bufstart, buflen);
         ret = janet_wrap_buffer(b);
     } else {
-        ret = janet_wrap_string(janet_string(p->buf, (int32_t)p->bufcount));
+        ret = janet_wrap_string(janet_string(bufstart, buflen));
     }
     p->bufcount = 0;
     popstate(p, ret);
@@ -266,7 +297,7 @@ static int stringchar(JanetParser *p, JanetParseState *state, uint8_t c) {
         return stringend(p, state);
     }
     /* normal char */
-    if (c != '\n')
+    if (c != '\n' && c != '\r')
         push_buf(p, c);
     return 1;
 }
@@ -298,6 +329,12 @@ static int tokenchar(JanetParser *p, JanetParseState *state, uint8_t c) {
     int start_dig = p->buf[0] >= '0' && p->buf[0] <= '9';
     int start_num = start_dig || p->buf[0] == '-' || p->buf[0] == '+' || p->buf[0] == '.';
     if (p->buf[0] == ':') {
+        /* Don't do full utf-8 check unless we have seen non ascii characters. */
+        int valid = (!state->argn) || valid_utf8(p->buf + 1, blen - 1);
+        if (!valid) {
+            p->error = "invalid utf-8 in keyword";
+            return 0;
+        }
         ret = janet_keywordv(p->buf + 1, blen - 1);
     } else if (start_num && !janet_scan_number(p->buf, blen, &numval)) {
         ret = janet_wrap_number(numval);
@@ -307,7 +344,7 @@ static int tokenchar(JanetParser *p, JanetParseState *state, uint8_t c) {
         ret = janet_wrap_false();
     } else if (!check_str_const("true", p->buf, blen)) {
         ret = janet_wrap_true();
-    } else if (p->buf) {
+    } else {
         if (start_dig) {
             p->error = "symbol literal cannot start with a digit";
             return 0;
@@ -320,9 +357,6 @@ static int tokenchar(JanetParser *p, JanetParseState *state, uint8_t c) {
             }
             ret = janet_symbolv(p->buf, blen);
         }
-    } else {
-        p->error = "empty symbol invalid";
-        return 0;
     }
     p->bufcount = 0;
     popstate(p, ret);
@@ -331,12 +365,18 @@ static int tokenchar(JanetParser *p, JanetParseState *state, uint8_t c) {
 
 static int comment(JanetParser *p, JanetParseState *state, uint8_t c) {
     (void) state;
-    if (c == '\n') p->statecount--;
+    if (c == '\n') {
+        p->statecount--;
+        p->bufcount = 0;
+    } else {
+        push_buf(p, c);
+    }
     return 1;
 }
 
-static Janet close_tuple(JanetParser *p, JanetParseState *state) {
+static Janet close_tuple(JanetParser *p, JanetParseState *state, int32_t flag) {
     Janet *ret = janet_tuple_begin(state->argn);
+    janet_tuple_flag(ret) |= flag;
     for (int32_t i = state->argn - 1; i >= 0; i--)
         ret[i] = p->args[--p->argcount];
     return janet_wrap_tuple(janet_tuple_end(ret));
@@ -416,30 +456,30 @@ static int longstring(JanetParser *p, JanetParseState *state, uint8_t c) {
 
 static int root(JanetParser *p, JanetParseState *state, uint8_t c);
 
-static int ampersand(JanetParser *p, JanetParseState *state, uint8_t c) {
+static int atsign(JanetParser *p, JanetParseState *state, uint8_t c) {
     (void) state;
     p->statecount--;
     switch (c) {
-    case '{':
-        pushstate(p, root, PFLAG_CONTAINER | PFLAG_CURLYBRACKETS | PFLAG_ATSYM);
-        return 1;
-    case '"':
-        pushstate(p, stringchar, PFLAG_BUFFER | PFLAG_STRING);
-        return 1;
-    case '`':
-        pushstate(p, longstring, PFLAG_BUFFER | PFLAG_LONGSTRING);
-        return 1;
-    case '[':
-        pushstate(p, root, PFLAG_CONTAINER | PFLAG_SQRBRACKETS | PFLAG_ATSYM);
-        return 1;
-    case '(':
-        pushstate(p, root, PFLAG_CONTAINER | PFLAG_PARENS | PFLAG_ATSYM);
-        return 1;
-    default:
-        break;
+        case '{':
+            pushstate(p, root, PFLAG_CONTAINER | PFLAG_CURLYBRACKETS | PFLAG_ATSYM);
+            return 1;
+        case '"':
+            pushstate(p, stringchar, PFLAG_BUFFER | PFLAG_STRING);
+            return 1;
+        case '`':
+            pushstate(p, longstring, PFLAG_BUFFER | PFLAG_LONGSTRING);
+            return 1;
+        case '[':
+            pushstate(p, root, PFLAG_CONTAINER | PFLAG_SQRBRACKETS | PFLAG_ATSYM);
+            return 1;
+        case '(':
+            pushstate(p, root, PFLAG_CONTAINER | PFLAG_PARENS | PFLAG_ATSYM);
+            return 1;
+        default:
+            break;
     }
-    pushstate(p, tokenchar, 0);
-    push_buf(p, '@'); /* Push the leading ampersand that was dropped */
+    pushstate(p, tokenchar, PFLAG_TOKEN);
+    push_buf(p, '@'); /* Push the leading at-sign that was dropped */
     return 0;
 }
 
@@ -452,59 +492,59 @@ static int root(JanetParser *p, JanetParseState *state, uint8_t c) {
                 p->error = "unexpected character";
                 return 1;
             }
-            pushstate(p, tokenchar, 0);
+            pushstate(p, tokenchar, PFLAG_TOKEN);
             return 0;
         case '\'':
         case ',':
         case ';':
         case '~':
+        case '|':
             pushstate(p, root, PFLAG_READERMAC | c);
             return 1;
         case '"':
             pushstate(p, stringchar, PFLAG_STRING);
             return 1;
         case '#':
-            pushstate(p, comment, 0);
+            pushstate(p, comment, PFLAG_COMMENT);
             return 1;
         case '@':
-            pushstate(p, ampersand, 0);
+            pushstate(p, atsign, PFLAG_ATSYM);
             return 1;
         case '`':
             pushstate(p, longstring, PFLAG_LONGSTRING);
             return 1;
         case ')':
         case ']':
-        case '}':
-            {
-                Janet ds;
-                if (p->statecount == 1) {
-                    p->error = "unexpected delimiter";
-                    return 1;
-                }
-                if ((c == ')' && (state->flags & PFLAG_PARENS)) ||
-                        (c == ']' && (state->flags & PFLAG_SQRBRACKETS))) {
-                    if (state->flags & PFLAG_ATSYM) {
-                        ds = close_array(p, state);
-                    } else {
-                        ds = close_tuple(p, state);
-                    }
-                } else if (c == '}' && (state->flags & PFLAG_CURLYBRACKETS)) {
-                    if (state->argn & 1) {
-                        p->error = "struct and table literals expect even number of arguments";
-                        return 1;
-                    }
-                    if (state->flags & PFLAG_ATSYM) {
-                        ds = close_table(p, state);
-                    } else {
-                        ds = close_struct(p, state);
-                    }
-                } else {
-                    p->error = "mismatched delimiter";
-                    return 1;
-                }
-                popstate(p, ds);
+        case '}': {
+            Janet ds;
+            if (p->statecount == 1) {
+                p->error = "unexpected delimiter";
+                return 1;
             }
-            return 1;
+            if ((c == ')' && (state->flags & PFLAG_PARENS)) ||
+                    (c == ']' && (state->flags & PFLAG_SQRBRACKETS))) {
+                if (state->flags & PFLAG_ATSYM) {
+                    ds = close_array(p, state);
+                } else {
+                    ds = close_tuple(p, state, c == ']' ? JANET_TUPLE_FLAG_BRACKETCTOR : 0);
+                }
+            } else if (c == '}' && (state->flags & PFLAG_CURLYBRACKETS)) {
+                if (state->argn & 1) {
+                    p->error = "struct and table literals expect even number of arguments";
+                    return 1;
+                }
+                if (state->flags & PFLAG_ATSYM) {
+                    ds = close_table(p, state);
+                } else {
+                    ds = close_struct(p, state);
+                }
+            } else {
+                p->error = "mismatched delimiter";
+                return 1;
+            }
+            popstate(p, ds);
+        }
+        return 1;
         case '(':
             pushstate(p, root, PFLAG_CONTAINER | PFLAG_PARENS);
             return 1;
@@ -517,20 +557,49 @@ static int root(JanetParser *p, JanetParseState *state, uint8_t c) {
     }
 }
 
-int janet_parser_consume(JanetParser *parser, uint8_t c) {
+static void janet_parser_checkdead(JanetParser *parser) {
+    if (parser->flag) janet_panic("parser is dead, cannot consume");
+    if (parser->error) janet_panic("parser has unchecked error, cannot consume");
+}
+
+/* Public API */
+
+void janet_parser_consume(JanetParser *parser, uint8_t c) {
     int consumed = 0;
-    if (parser->error) return 0;
-    parser->offset++;
+    janet_parser_checkdead(parser);
+    if (c == '\r') {
+        parser->line++;
+        parser->column = 0;
+    } else if (c == '\n') {
+        parser->column = 0;
+        if (parser->lookback != '\r')
+            parser->line++;
+    } else {
+        parser->column++;
+    }
     while (!consumed && !parser->error) {
         JanetParseState *state = parser->states + parser->statecount - 1;
         consumed = state->consumer(parser, state, c);
     }
     parser->lookback = c;
-    return 1;
+}
+
+void janet_parser_eof(JanetParser *parser) {
+    janet_parser_checkdead(parser);
+    size_t oldcolumn = parser->column;
+    size_t oldline = parser->line;
+    janet_parser_consume(parser, '\n');
+    if (parser->statecount > 1) {
+        parser->error = "unexpected end of source";
+    }
+    parser->line = oldline;
+    parser->column = oldcolumn;
+    parser->flag = 1;
 }
 
 enum JanetParserStatus janet_parser_status(JanetParser *parser) {
     if (parser->error) return JANET_PARSE_ERROR;
+    if (parser->flag) return JANET_PARSE_DEAD;
     if (parser->statecount > 1) return JANET_PARSE_PENDING;
     return JANET_PARSE_ROOT;
 }
@@ -578,8 +647,10 @@ void janet_parser_init(JanetParser *parser) {
     parser->statecap = 0;
     parser->error = NULL;
     parser->lookback = -1;
-    parser->offset = 0;
+    parser->line = 1;
+    parser->column = 0;
     parser->pending = 0;
+    parser->flag = 0;
 
     pushstate(parser, root, PFLAG_CONTAINER);
 }
@@ -590,6 +661,56 @@ void janet_parser_deinit(JanetParser *parser) {
     free(parser->states);
 }
 
+void janet_parser_clone(const JanetParser *src, JanetParser *dest) {
+    /* Misc fields */
+    dest->flag = src->flag;
+    dest->pending = src->pending;
+    dest->lookback = src->lookback;
+    dest->line = src->line;
+    dest->column = src->column;
+    dest->error = src->error;
+
+    /* Keep counts */
+    dest->argcount = src->argcount;
+    dest->bufcount = src->bufcount;
+    dest->statecount = src->statecount;
+
+    /* Capacities are equal to counts */
+    dest->bufcap = dest->bufcount;
+    dest->statecap = dest->statecount;
+    dest->argcap = dest->argcount;
+
+    /* Deep cloned fields */
+    dest->args = NULL;
+    dest->states = NULL;
+    dest->buf = NULL;
+    if (dest->bufcap) {
+        dest->buf = malloc(dest->bufcap);
+        if (!dest->buf) goto nomem;
+    }
+    if (dest->argcap) {
+        dest->args = malloc(sizeof(Janet) * dest->argcap);
+        if (!dest->args) goto nomem;
+    }
+    if (dest->statecap) {
+        dest->states = malloc(sizeof(JanetParseState) * dest->statecap);
+        if (!dest->states) goto nomem;
+    }
+
+    memcpy(dest->buf, src->buf, dest->bufcap);
+    memcpy(dest->args, src->args, dest->argcap * sizeof(Janet));
+    memcpy(dest->states, src->states, dest->statecap * sizeof(JanetParseState));
+
+    return;
+
+nomem:
+    JANET_OUT_OF_MEMORY;
+}
+
+int janet_parser_has_more(JanetParser *parser) {
+    return !!parser->pending;
+}
+
 /* C functions */
 
 static int parsermark(void *p, size_t size) {
@@ -609,10 +730,17 @@ static int parsergc(void *p, size_t size) {
     return 0;
 }
 
+static int parserget(void *p, Janet key, Janet *out);
+
 static JanetAbstractType janet_parse_parsertype = {
     "core/parser",
     parsergc,
-    parsermark
+    parsermark,
+    parserget,
+    NULL,
+    NULL,
+    NULL,
+    NULL
 };
 
 /* C Function parser */
@@ -649,6 +777,46 @@ static Janet cfun_parse_consume(int32_t argc, Janet *argv) {
     return janet_wrap_integer(i);
 }
 
+static Janet cfun_parse_eof(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    JanetParser *p = janet_getabstract(argv, 0, &janet_parse_parsertype);
+    janet_parser_eof(p);
+    return argv[0];
+}
+
+static Janet cfun_parse_insert(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 2);
+    JanetParser *p = janet_getabstract(argv, 0, &janet_parse_parsertype);
+    JanetParseState *s = p->states + p->statecount - 1;
+    if (s->consumer == tokenchar) {
+        janet_parser_consume(p, ' ');
+        p->column--;
+        s = p->states + p->statecount - 1;
+    }
+    if (s->flags & PFLAG_CONTAINER) {
+        s->argn++;
+        if (p->statecount == 1) p->pending++;
+        push_arg(p, argv[1]);
+    } else if (s->flags & (PFLAG_STRING | PFLAG_LONGSTRING)) {
+        const uint8_t *str = janet_to_string(argv[1]);
+        int32_t slen = janet_string_length(str);
+        size_t newcount = p->bufcount + slen;
+        if (p->bufcap < newcount) {
+            size_t newcap = 2 * newcount;
+            p->buf = realloc(p->buf, newcap);
+            if (p->buf == NULL) {
+                JANET_OUT_OF_MEMORY;
+            }
+            p->bufcap = newcap;
+        }
+        memcpy(p->buf + p->bufcount, str, slen);
+        p->bufcount = newcount;
+    } else {
+        janet_panic("cannot insert value into parser");
+    }
+    return argv[0];
+}
+
 static Janet cfun_parse_has_more(int32_t argc, Janet *argv) {
     janet_fixarity(argc, 1);
     JanetParser *p = janet_getabstract(argv, 0, &janet_parse_parsertype);
@@ -677,6 +845,9 @@ static Janet cfun_parse_status(int32_t argc, Janet *argv) {
         case JANET_PARSE_ROOT:
             stat = "root";
             break;
+        case JANET_PARSE_DEAD:
+            stat = "dead";
+            break;
     }
     return janet_ckeywordv(stat);
 }
@@ -705,112 +876,286 @@ static Janet cfun_parse_flush(int32_t argc, Janet *argv) {
 static Janet cfun_parse_where(int32_t argc, Janet *argv) {
     janet_fixarity(argc, 1);
     JanetParser *p = janet_getabstract(argv, 0, &janet_parse_parsertype);
-    return janet_wrap_integer(p->offset);
+    Janet *tup = janet_tuple_begin(2);
+    tup[0] = janet_wrap_integer(p->line);
+    tup[1] = janet_wrap_integer(p->column);
+    return janet_wrap_tuple(janet_tuple_end(tup));
 }
 
-static Janet cfun_parse_state(int32_t argc, Janet *argv) {
+static Janet janet_wrap_parse_state(JanetParseState *s, Janet *args,
+                                    uint8_t *buff, uint32_t bufcount) {
+    JanetTable *state = janet_table(0);
+    const uint8_t *buffer;
+    int add_buffer = 0;
+    const char *type = NULL;
+
+    if (s->flags & PFLAG_CONTAINER) {
+        JanetArray *container_args = janet_array(s->argn);
+        container_args->count = s->argn;
+        memcpy(container_args->data, args, sizeof(args[0])*s->argn);
+        janet_table_put(state, janet_ckeywordv("args"),
+                        janet_wrap_array(container_args));
+    }
+
+    if (s->flags & PFLAG_PARENS || s->flags & PFLAG_SQRBRACKETS) {
+        if (s->flags & PFLAG_ATSYM) {
+            type = "array";
+        } else {
+            type = "tuple";
+        }
+    } else if (s->flags & PFLAG_CURLYBRACKETS) {
+        if (s->flags & PFLAG_ATSYM) {
+            type = "table";
+        } else {
+            type = "struct";
+        }
+    } else if (s->flags & PFLAG_STRING || s->flags & PFLAG_LONGSTRING) {
+        if (s->flags & PFLAG_BUFFER) {
+            type = "buffer";
+        } else {
+            type = "string";
+        }
+        add_buffer = 1;
+    } else if (s->flags & PFLAG_COMMENT) {
+        type = "comment";
+        add_buffer = 1;
+    } else if (s->flags & PFLAG_TOKEN) {
+        type = "token";
+        add_buffer = 1;
+    } else if (s->flags & PFLAG_ATSYM) {
+        type = "at";
+    } else if (s->flags & PFLAG_READERMAC) {
+        int c = s->flags & 0xFF;
+        type = (c == '\'') ? "quote" :
+               (c == ',') ? "unquote" :
+               (c == ';') ? "splice" :
+               (c == '~') ? "quasiquote" : "<reader>";
+    } else {
+        type = "root";
+    }
+
+    if (type) {
+        janet_table_put(state, janet_ckeywordv("type"),
+                        janet_ckeywordv(type));
+    }
+
+    if (add_buffer) {
+        buffer = janet_string(buff, bufcount);
+        janet_table_put(state, janet_ckeywordv("buffer"), janet_wrap_string(buffer));
+    }
+
+    janet_table_put(state, janet_ckeywordv("line"), janet_wrap_integer(s->line));
+    janet_table_put(state, janet_ckeywordv("column"), janet_wrap_integer(s->column));
+    return janet_wrap_table(state);
+}
+
+struct ParserStateGetter {
+    const char *name;
+    Janet(*fn)(const JanetParser *p);
+};
+
+static Janet parser_state_delimiters(const JanetParser *_p) {
+    JanetParser *clone = janet_abstract(&janet_parse_parsertype, sizeof(JanetParser));
+    janet_parser_clone(_p, clone);
     size_t i;
     const uint8_t *str;
     size_t oldcount;
-    janet_fixarity(argc, 1);
-    JanetParser *p = janet_getabstract(argv, 0, &janet_parse_parsertype);
-    oldcount = p->bufcount;
-    for (i = 0; i < p->statecount; i++) {
-        JanetParseState *s = p->states + i;
+    oldcount = clone->bufcount;
+    for (i = 0; i < clone->statecount; i++) {
+        JanetParseState *s = clone->states + i;
         if (s->flags & PFLAG_PARENS) {
-            push_buf(p, '(');
+            push_buf(clone, '(');
         } else if (s->flags & PFLAG_SQRBRACKETS) {
-            push_buf(p, '[');
+            push_buf(clone, '[');
         } else if (s->flags & PFLAG_CURLYBRACKETS) {
-            push_buf(p, '{');
+            push_buf(clone, '{');
         } else if (s->flags & PFLAG_STRING) {
-            push_buf(p, '"');
+            push_buf(clone, '"');
         } else if (s->flags & PFLAG_LONGSTRING) {
             int32_t i;
             for (i = 0; i < s->argn; i++) {
-                push_buf(p, '`');
+                push_buf(clone, '`');
             }
         }
     }
-    str = janet_string(p->buf + oldcount, (int32_t)(p->bufcount - oldcount));
-    p->bufcount = oldcount;
+    str = janet_string(clone->buf + oldcount, (int32_t)(clone->bufcount - oldcount));
+    clone->bufcount = oldcount;
     return janet_wrap_string(str);
 }
 
+static Janet parser_state_frames(const JanetParser *p) {
+    int32_t count = (int32_t) p->statecount;
+    JanetArray *states = janet_array(count);
+    states->count = count;
+    uint8_t *buf = p->buf;
+    Janet *args = p->args;
+    for (int32_t i = count - 1; i >= 0; --i) {
+        JanetParseState *s = p->states + i;
+        states->data[i] = janet_wrap_parse_state(s, args, buf, (uint32_t) p->bufcount);
+        args -= s->argn;
+    }
+    return janet_wrap_array(states);
+}
+
+static const struct ParserStateGetter parser_state_getters[] = {
+    {"frames", parser_state_frames},
+    {"delimiters", parser_state_delimiters},
+    {NULL, NULL}
+};
+
+static Janet cfun_parse_state(int32_t argc, Janet *argv) {
+    janet_arity(argc, 1, 2);
+    const uint8_t *key = NULL;
+    JanetParser *p = janet_getabstract(argv, 0, &janet_parse_parsertype);
+    if (argc == 2) {
+        key = janet_getkeyword(argv, 1);
+    }
+
+    if (key) {
+        /* Get one result */
+        for (const struct ParserStateGetter *sg = parser_state_getters;
+                sg->name != NULL; sg++) {
+            if (janet_cstrcmp(key, sg->name)) continue;
+            return sg->fn(p);
+        }
+        janet_panicf("unexpected keyword %v", janet_wrap_keyword(key));
+        return janet_wrap_nil();
+    } else {
+        /* Put results in table */
+        JanetTable *tab = janet_table(0);
+        for (const struct ParserStateGetter *sg = parser_state_getters;
+                sg->name != NULL; sg++) {
+            janet_table_put(tab, janet_ckeywordv(sg->name), sg->fn(p));
+        }
+        return janet_wrap_table(tab);
+    }
+}
+
+static Janet cfun_parse_clone(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    JanetParser *src = janet_getabstract(argv, 0, &janet_parse_parsertype);
+    JanetParser *dest = janet_abstract(&janet_parse_parsertype, sizeof(JanetParser));
+    janet_parser_clone(src, dest);
+    return janet_wrap_abstract(dest);
+}
+
+static const JanetMethod parser_methods[] = {
+    {"byte", cfun_parse_byte},
+    {"clone", cfun_parse_clone},
+    {"consume", cfun_parse_consume},
+    {"eof", cfun_parse_eof},
+    {"error", cfun_parse_error},
+    {"flush", cfun_parse_flush},
+    {"has-more", cfun_parse_has_more},
+    {"insert", cfun_parse_insert},
+    {"produce", cfun_parse_produce},
+    {"state", cfun_parse_state},
+    {"status", cfun_parse_status},
+    {"where", cfun_parse_where},
+    {NULL, NULL}
+};
+
+static int parserget(void *p, Janet key, Janet *out) {
+    (void) p;
+    if (!janet_checktype(key, JANET_KEYWORD)) return 0;
+    return janet_getmethod(janet_unwrap_keyword(key), parser_methods, out);
+}
+
 static const JanetReg parse_cfuns[] = {
     {
         "parser/new", cfun_parse_parser,
         JDOC("(parser/new)\n\n"
-                "Creates and returns a new parser object. Parsers are state machines "
-                "that can receive bytes, and generate a stream of janet values. ")
+             "Creates and returns a new parser object. Parsers are state machines "
+             "that can receive bytes, and generate a stream of janet values.")
+    },
+    {
+        "parser/clone", cfun_parse_clone,
+        JDOC("(parser/clone p)\n\n"
+             "Creates a deep clone of a parser that is identical to the input parser. "
+             "This cloned parser can be used to continue parsing from a good checkpoint "
+             "if parsing later fails. Returns a new parser.")
     },
     {
         "parser/has-more", cfun_parse_has_more,
         JDOC("(parser/has-more parser)\n\n"
-                "Check if the parser has more values in the value queue.")
+             "Check if the parser has more values in the value queue.")
     },
     {
         "parser/produce", cfun_parse_produce,
         JDOC("(parser/produce parser)\n\n"
-                "Dequeue the next value in the parse queue. Will return nil if "
-                "no parsed values are in the queue, otherwise will dequeue the "
-                "next value.")
+             "Dequeue the next value in the parse queue. Will return nil if "
+             "no parsed values are in the queue, otherwise will dequeue the "
+             "next value.")
     },
     {
         "parser/consume", cfun_parse_consume,
-        JDOC("(parser/consume parser bytes [, index])\n\n"
-                "Input bytes into the parser and parse them. Will not throw errors "
-                "if there is a parse error. Starts at the byte index given by index. Returns "
-                "the number of bytes read.")
+        JDOC("(parser/consume parser bytes &opt index)\n\n"
+             "Input bytes into the parser and parse them. Will not throw errors "
+             "if there is a parse error. Starts at the byte index given by index. Returns "
+             "the number of bytes read.")
     },
     {
         "parser/byte", cfun_parse_byte,
         JDOC("(parser/byte parser b)\n\n"
-                "Input a single byte into the parser byte stream. Returns the parser.")
+             "Input a single byte into the parser byte stream. Returns the parser.")
     },
     {
         "parser/error", cfun_parse_error,
         JDOC("(parser/error parser)\n\n"
-                "If the parser is in the error state, returns the message associated with "
-                "that error. Otherwise, returns nil. Also flushes the parser state and parser "
-                "queue, so be sure to handle everything in the queue before calling "
-                "parser/error.")
+             "If the parser is in the error state, returns the message associated with "
+             "that error. Otherwise, returns nil. Also flushes the parser state and parser "
+             "queue, so be sure to handle everything in the queue before calling "
+             "parser/error.")
     },
     {
         "parser/status", cfun_parse_status,
         JDOC("(parser/status parser)\n\n"
-                "Gets the current status of the parser state machine. The status will "
-                "be one of:\n\n"
-                "\t:pending - a value is being parsed.\n"
-                "\t:error - a parsing error was encountered.\n"
-                "\t:root - the parser can either read more values or safely terminate.")
+             "Gets the current status of the parser state machine. The status will "
+             "be one of:\n\n"
+             "\t:pending - a value is being parsed.\n"
+             "\t:error - a parsing error was encountered.\n"
+             "\t:root - the parser can either read more values or safely terminate.")
     },
     {
         "parser/flush", cfun_parse_flush,
         JDOC("(parser/flush parser)\n\n"
-                "Clears the parser state and parse queue. Can be used to reset the parser "
-                "if an error was encountered. Does not reset the line and column counter, so "
-                "to begin parsing in a new context, create a new parser.")
+             "Clears the parser state and parse queue. Can be used to reset the parser "
+             "if an error was encountered. Does not reset the line and column counter, so "
+             "to begin parsing in a new context, create a new parser.")
     },
     {
         "parser/state", cfun_parse_state,
-        JDOC("(parser/state parser)\n\n"
-                "Returns a string representation of the internal state of the parser. "
-                "Each byte in the string represents a nested data structure. For example, "
-                "if the parser state is '([\"', then the parser is in the middle of parsing a "
-                "string inside of square brackets inside parentheses. Can be used to augment a REPL prompt.")
+        JDOC("(parser/state parser &opt key)\n\n"
+             "Returns a representation of the internal state of the parser. If a key is passed, "
+             "only that information about the state is returned. Allowed keys are:\n\n"
+             "\t:delimiters - Each byte in the string represents a nested data structure. For example, "
+             "if the parser state is '([\"', then the parser is in the middle of parsing a "
+             "string inside of square brackets inside parentheses. Can be used to augment a REPL prompt."
+             "\t:frames - Each table in the array represents a 'frame' in the parser state. Frames "
+             "contain information about the start of the expression being parsed as well as the "
+             "type of that expression and some type-specific information.")
     },
     {
         "parser/where", cfun_parse_where,
         JDOC("(parser/where parser)\n\n"
-                "Returns the current line number and column number of the parser's location "
-                "in the byte stream as a tuple (line, column). Lines and columns are counted from "
-                "1, (the first byte is line 1, column 1) and a newline is considered ASCII 0x0A.")
+             "Returns the current line number and column of the parser's internal state.")
+    },
+    {
+        "parser/eof", cfun_parse_eof,
+        JDOC("(parser/eof parser)\n\n"
+             "Indicate that the end of file was reached to the parser. This puts the parser in the :dead state.")
+    },
+    {
+        "parser/insert", cfun_parse_insert,
+        JDOC("(parser/insert parser value)\n\n"
+             "Insert a value into the parser. This means that the parser state can be manipulated "
+             "in between chunks of bytes. This would allow a user to add extra elements to arrays "
+             "and tuples, for example. Returns the parser.")
     },
     {NULL, NULL, NULL}
 };
 
 /* Load the library */
 void janet_lib_parse(JanetTable *env) {
-    janet_cfuns(env, NULL, parse_cfuns);
+    janet_core_cfuns(env, NULL, parse_cfuns);
 }

src/core/pp.c

@@ -20,21 +20,29 @@
 * IN THE SOFTWARE.
 */
 
+#include <string.h>
+#include <ctype.h>
+
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "util.h"
 #include "state.h"
 #endif
 
 /* Implements a pretty printer for Janet. The pretty printer
- * is farily simple and not that flexible, but fast. */
+ * is simple and not that flexible, but fast. */
 
 /* Temporary buffer size */
 #define BUFSIZE 64
 
 static void number_to_string_b(JanetBuffer *buffer, double x) {
     janet_buffer_ensure(buffer, buffer->count + BUFSIZE, 2);
-    int count = snprintf((char *) buffer->data + buffer->count, BUFSIZE, "%g", x);
+    /* Use int32_t range for valid integers because that is the
+     * range most integer-expecting functions in the C api use. */
+    const char *fmt = (x == floor(x) &&
+                       x <= ((double) INT32_MAX) &&
+                       x >= ((double) INT32_MIN)) ? "%.0f" : "%g";
+    int count = snprintf((char *) buffer->data + buffer->count, BUFSIZE, fmt, x);
     buffer->count += count;
 }
 
@@ -70,7 +78,7 @@ static void integer_to_string_b(JanetBuffer *buffer, int32_t x) {
     len = count_dig10(x);
     buf += len;
     while (x) {
-        uint8_t digit = (uint8_t) -(x % 10);
+        uint8_t digit = (uint8_t) - (x % 10);
         *(--buf) = '0' + digit;
         x /= 10;
     }
@@ -133,6 +141,15 @@ static void janet_escape_string_impl(JanetBuffer *buffer, const uint8_t *str, in
             case '\0':
                 janet_buffer_push_bytes(buffer, (const uint8_t *)"\\0", 2);
                 break;
+            case '\f':
+                janet_buffer_push_bytes(buffer, (const uint8_t *)"\\f", 2);
+                break;
+            case '\v':
+                janet_buffer_push_bytes(buffer, (const uint8_t *)"\\v", 2);
+                break;
+            case 27:
+                janet_buffer_push_bytes(buffer, (const uint8_t *)"\\e", 2);
+                break;
             case '\\':
                 janet_buffer_push_bytes(buffer, (const uint8_t *)"\\\\", 2);
                 break;
@@ -164,53 +181,60 @@ static void janet_escape_buffer_b(JanetBuffer *buffer, JanetBuffer *bx) {
 
 void janet_description_b(JanetBuffer *buffer, Janet x) {
     switch (janet_type(x)) {
-    case JANET_NIL:
-        janet_buffer_push_cstring(buffer, "nil");
-        return;
-    case JANET_TRUE:
-        janet_buffer_push_cstring(buffer, "true");
-        return;
-    case JANET_FALSE:
-        janet_buffer_push_cstring(buffer, "false");
-        return;
-    case JANET_NUMBER:
-        number_to_string_b(buffer, janet_unwrap_number(x));
-        return;
-    case JANET_KEYWORD:
-        janet_buffer_push_u8(buffer, ':');
+        case JANET_NIL:
+            janet_buffer_push_cstring(buffer, "nil");
+            return;
+        case JANET_BOOLEAN:
+            janet_buffer_push_cstring(buffer,
+                                      janet_unwrap_boolean(x) ? "true" : "false");
+            return;
+        case JANET_NUMBER:
+            number_to_string_b(buffer, janet_unwrap_number(x));
+            return;
+        case JANET_KEYWORD:
+            janet_buffer_push_u8(buffer, ':');
         /* fallthrough */
-    case JANET_SYMBOL:
-        janet_buffer_push_bytes(buffer,
-                janet_unwrap_string(x),
-                janet_string_length(janet_unwrap_string(x)));
-        return;
-    case JANET_STRING:
-        janet_escape_string_b(buffer, janet_unwrap_string(x));
-        return;
-    case JANET_BUFFER:
-        janet_escape_buffer_b(buffer, janet_unwrap_buffer(x));
-        return;
-    case JANET_ABSTRACT:
-        {
-            const char *n = janet_abstract_type(janet_unwrap_abstract(x))->name;
-            string_description_b(buffer, n, janet_unwrap_abstract(x));
-			return;
+        case JANET_SYMBOL:
+            janet_buffer_push_bytes(buffer,
+                                    janet_unwrap_string(x),
+                                    janet_string_length(janet_unwrap_string(x)));
+            return;
+        case JANET_STRING:
+            janet_escape_string_b(buffer, janet_unwrap_string(x));
+            return;
+        case JANET_BUFFER: {
+            JanetBuffer *b = janet_unwrap_buffer(x);
+            if (b == buffer) {
+                /* Ensures buffer won't resize while escaping */
+                janet_buffer_ensure(b, 5 * b->count + 3, 1);
+            }
+            janet_escape_buffer_b(buffer, b);
+            return;
         }
-    case JANET_CFUNCTION:
-        {
+        case JANET_ABSTRACT: {
+            void *p = janet_unwrap_abstract(x);
+            const JanetAbstractType *at = janet_abstract_type(p);
+            if (at->tostring) {
+                at->tostring(p, buffer);
+            } else {
+                const char *n = at->name;
+                string_description_b(buffer, n, janet_unwrap_abstract(x));
+            }
+            return;
+        }
+        case JANET_CFUNCTION: {
             Janet check = janet_table_get(janet_vm_registry, x);
             if (janet_checktype(check, JANET_SYMBOL)) {
                 janet_buffer_push_cstring(buffer, "<cfunction ");
                 janet_buffer_push_bytes(buffer,
-                        janet_unwrap_symbol(check),
-                        janet_string_length(janet_unwrap_symbol(check)));
+                                        janet_unwrap_symbol(check),
+                                        janet_string_length(janet_unwrap_symbol(check)));
                 janet_buffer_push_u8(buffer, '>');
                 break;
             }
             goto fallthrough;
         }
-    case JANET_FUNCTION:
-        {
+        case JANET_FUNCTION: {
             JanetFunction *fun = janet_unwrap_function(x);
             JanetFuncDef *def = fun->def;
             if (def->name) {
@@ -223,9 +247,9 @@ void janet_description_b(JanetBuffer *buffer, Janet x) {
             goto fallthrough;
         }
     fallthrough:
-    default:
-        string_description_b(buffer, janet_type_names[janet_type(x)], janet_unwrap_pointer(x));
-        break;
+        default:
+            string_description_b(buffer, janet_type_names[janet_type(x)], janet_unwrap_pointer(x));
+            break;
     }
 }
 
@@ -234,17 +258,19 @@ void janet_to_string_b(JanetBuffer *buffer, Janet x) {
         default:
             janet_description_b(buffer, x);
             break;
-        case JANET_BUFFER:
-            janet_buffer_push_bytes(buffer,
-                    janet_unwrap_buffer(x)->data,
-                    janet_unwrap_buffer(x)->count);
+        case JANET_BUFFER: {
+            JanetBuffer *to = janet_unwrap_buffer(x);
+            /* Prevent resizing buffer while appending */
+            if (buffer == to) janet_buffer_extra(buffer, to->count);
+            janet_buffer_push_bytes(buffer, to->data, to->count);
             break;
+        }
         case JANET_STRING:
         case JANET_SYMBOL:
         case JANET_KEYWORD:
             janet_buffer_push_bytes(buffer,
-                    janet_unwrap_string(x),
-                    janet_string_length(janet_unwrap_string(x)));
+                                    janet_unwrap_string(x),
+                                    janet_string_length(janet_unwrap_string(x)));
             break;
     }
 }
@@ -262,15 +288,14 @@ const uint8_t *janet_description(Janet x) {
  * strings, symbols, and buffers will return their content. */
 const uint8_t *janet_to_string(Janet x) {
     switch (janet_type(x)) {
-        default:
-            {
-                JanetBuffer b;
-                janet_buffer_init(&b, 10);
-                janet_to_string_b(&b, x);
-                const uint8_t *ret = janet_string(b.data, b.count);
-                janet_buffer_deinit(&b);
-                return ret;
-            }
+        default: {
+            JanetBuffer b;
+            janet_buffer_init(&b, 10);
+            janet_to_string_b(&b, x);
+            const uint8_t *ret = janet_string(b.data, b.count);
+            janet_buffer_deinit(&b);
+            return ret;
+        }
         case JANET_BUFFER:
             return janet_string(janet_unwrap_buffer(x)->data, janet_unwrap_buffer(x)->count);
         case JANET_STRING:
@@ -285,12 +310,14 @@ struct pretty {
     JanetBuffer *buffer;
     int depth;
     int indent;
+    int flags;
+    int32_t bufstartlen;
     JanetTable seen;
 };
 
 static void print_newline(struct pretty *S, int just_a_space) {
     int i;
-    if (just_a_space) {
+    if (just_a_space || (S->flags & JANET_PRETTY_ONELINE)) {
         janet_buffer_push_u8(S->buffer, ' ');
         return;
     }
@@ -300,6 +327,30 @@ static void print_newline(struct pretty *S, int just_a_space) {
     }
 }
 
+/* Color coding for types */
+static const char janet_cycle_color[] = "\x1B[36m";
+static const char *janet_pretty_colors[] = {
+    "\x1B[32m",
+    "\x1B[36m",
+    "\x1B[36m",
+    "\x1B[36m",
+    "\x1B[35m",
+    "\x1B[34m",
+    "\x1B[33m",
+    "\x1B[36m",
+    "\x1B[36m",
+    "\x1B[36m",
+    "\x1B[36m"
+    "\x1B[35m",
+    "\x1B[36m",
+    "\x1B[36m",
+    "\x1B[36m",
+    "\x1B[36m"
+};
+
+#define JANET_PRETTY_DICT_ONELINE 4
+#define JANET_PRETTY_IND_ONELINE 10
+
 /* Helper for pretty printing */
 static void janet_pretty_one(struct pretty *S, Janet x, int is_dict_value) {
     /* Add to seen */
@@ -307,113 +358,130 @@ static void janet_pretty_one(struct pretty *S, Janet x, int is_dict_value) {
         case JANET_NIL:
         case JANET_NUMBER:
         case JANET_SYMBOL:
-        case JANET_TRUE:
-        case JANET_FALSE:
+        case JANET_BOOLEAN:
             break;
-        default:
-            {
-                Janet seenid = janet_table_get(&S->seen, x);
-                if (janet_checktype(seenid, JANET_NUMBER)) {
-                    janet_buffer_push_cstring(S->buffer, "<cycle ");
-                    integer_to_string_b(S->buffer, janet_unwrap_integer(seenid));
-                    janet_buffer_push_u8(S->buffer, '>');
-                    return;
-                } else {
-                    janet_table_put(&S->seen, x, janet_wrap_integer(S->seen.count));
-                    break;
+        default: {
+            Janet seenid = janet_table_get(&S->seen, x);
+            if (janet_checktype(seenid, JANET_NUMBER)) {
+                if (S->flags & JANET_PRETTY_COLOR) {
+                    janet_buffer_push_cstring(S->buffer, janet_cycle_color);
                 }
+                janet_buffer_push_cstring(S->buffer, "<cycle ");
+                integer_to_string_b(S->buffer, janet_unwrap_integer(seenid));
+                janet_buffer_push_u8(S->buffer, '>');
+                if (S->flags & JANET_PRETTY_COLOR) {
+                    janet_buffer_push_cstring(S->buffer, "\x1B[0m");
+                }
+                return;
+            } else {
+                janet_table_put(&S->seen, x, janet_wrap_integer(S->seen.count));
+                break;
             }
+        }
     }
 
     switch (janet_type(x)) {
-        default:
-            janet_description_b(S->buffer, x);
+        default: {
+            const char *color = janet_pretty_colors[janet_type(x)];
+            if (color && (S->flags & JANET_PRETTY_COLOR)) {
+                janet_buffer_push_cstring(S->buffer, color);
+            }
+            if (janet_checktype(x, JANET_BUFFER) && janet_unwrap_buffer(x) == S->buffer) {
+                janet_buffer_ensure(S->buffer, S->buffer->count + S->bufstartlen * 4 + 3, 1);
+                janet_buffer_push_u8(S->buffer, '@');
+                janet_escape_string_impl(S->buffer, S->buffer->data, S->bufstartlen);
+            } else {
+                janet_description_b(S->buffer, x);
+            }
+            if (color && (S->flags & JANET_PRETTY_COLOR)) {
+                janet_buffer_push_cstring(S->buffer, "\x1B[0m");
+            }
             break;
+        }
         case JANET_ARRAY:
-        case JANET_TUPLE:
-            {
-                int isarray = janet_checktype(x, JANET_ARRAY);
-                janet_buffer_push_cstring(S->buffer, isarray ? "@[" : "(");
-                S->depth--;
-                S->indent += 2;
-                if (S->depth == 0) {
-                    janet_buffer_push_cstring(S->buffer, "...");
-                } else {
-                    int32_t i, len;
-                    const Janet *arr;
-                    janet_indexed_view(x, &arr, &len);
-                    if (!isarray && len >= 5)
-                        janet_buffer_push_u8(S->buffer, ' ');
-                    if (is_dict_value && len >= 5) print_newline(S, 0);
-                    for (i = 0; i < len; i++) {
-                        if (i) print_newline(S, len < 5);
-                        janet_pretty_one(S, arr[i], 0);
-                    }
+        case JANET_TUPLE: {
+            int32_t i = 0, len = 0;
+            const Janet *arr = NULL;
+            int isarray = janet_checktype(x, JANET_ARRAY);
+            janet_indexed_view(x, &arr, &len);
+            int hasbrackets = !isarray && (janet_tuple_flag(arr) & JANET_TUPLE_FLAG_BRACKETCTOR);
+            const char *startstr = isarray ? "@[" : hasbrackets ? "[" : "(";
+            const char endchar = isarray ? ']' : hasbrackets ? ']' : ')';
+            janet_buffer_push_cstring(S->buffer, startstr);
+            S->depth--;
+            S->indent += 2;
+            if (S->depth == 0) {
+                janet_buffer_push_cstring(S->buffer, "...");
+            } else {
+                if (!isarray && !(S->flags & JANET_PRETTY_ONELINE) && len >= JANET_PRETTY_IND_ONELINE)
+                    janet_buffer_push_u8(S->buffer, ' ');
+                if (is_dict_value && len >= JANET_PRETTY_IND_ONELINE) print_newline(S, 0);
+                for (i = 0; i < len; i++) {
+                    if (i) print_newline(S, len < JANET_PRETTY_IND_ONELINE);
+                    janet_pretty_one(S, arr[i], 0);
                 }
-                S->indent -= 2;
-                S->depth++;
-                janet_buffer_push_u8(S->buffer, isarray ? ']' : ')');
-                break;
             }
+            S->indent -= 2;
+            S->depth++;
+            janet_buffer_push_u8(S->buffer, endchar);
+            break;
+        }
         case JANET_STRUCT:
-        case JANET_TABLE:
-            {
-                int istable = janet_checktype(x, JANET_TABLE);
-                janet_buffer_push_cstring(S->buffer, istable ? "@" : "{");
+        case JANET_TABLE: {
+            int istable = janet_checktype(x, JANET_TABLE);
+            janet_buffer_push_cstring(S->buffer, istable ? "@" : "{");
 
-                /* For object-like tables, print class name */
-                if (istable) {
-                    JanetTable *t = janet_unwrap_table(x);
-                    JanetTable *proto = t->proto;
-                    if (NULL != proto) {
-                        Janet name = janet_table_get(proto, janet_csymbolv(":name"));
-                        if (janet_checktype(name, JANET_SYMBOL)) {
-                            const uint8_t *sym = janet_unwrap_symbol(name);
-                            janet_buffer_push_bytes(S->buffer, sym, janet_string_length(sym));
-                        }
-                    }
-                    janet_buffer_push_cstring(S->buffer, "{");
-                }
-
-                S->depth--;
-                S->indent += 2;
-                if (S->depth == 0) {
-                    janet_buffer_push_cstring(S->buffer, "...");
-                } else {
-                    int32_t i, len, cap;
-                    int first_kv_pair = 1;
-                    const JanetKV *kvs;
-                    janet_dictionary_view(x, &kvs, &len, &cap);
-                    if (!istable && len >= 4)
-                        janet_buffer_push_u8(S->buffer, ' ');
-                    if (is_dict_value && len >= 5) print_newline(S, 0);
-                    for (i = 0; i < cap; i++) {
-                        if (!janet_checktype(kvs[i].key, JANET_NIL)) {
-                            if (first_kv_pair) {
-                                first_kv_pair = 0;
-                            } else {
-                                print_newline(S, len < 4);
-                            }
-                            janet_pretty_one(S, kvs[i].key, 0);
-                            janet_buffer_push_u8(S->buffer, ' ');
-                            janet_pretty_one(S, kvs[i].value, 1);
-                        }
+            /* For object-like tables, print class name */
+            if (istable) {
+                JanetTable *t = janet_unwrap_table(x);
+                JanetTable *proto = t->proto;
+                if (NULL != proto) {
+                    Janet name = janet_table_get(proto, janet_csymbolv(":name"));
+                    if (janet_checktype(name, JANET_SYMBOL)) {
+                        const uint8_t *sym = janet_unwrap_symbol(name);
+                        janet_buffer_push_bytes(S->buffer, sym, janet_string_length(sym));
                     }
                 }
-                S->indent -= 2;
-                S->depth++;
-                janet_buffer_push_u8(S->buffer, '}');
-                break;
+                janet_buffer_push_cstring(S->buffer, "{");
             }
+
+            S->depth--;
+            S->indent += 2;
+            if (S->depth == 0) {
+                janet_buffer_push_cstring(S->buffer, "...");
+            } else {
+                int32_t i = 0, len = 0, cap = 0;
+                int first_kv_pair = 1;
+                const JanetKV *kvs = NULL;
+                janet_dictionary_view(x, &kvs, &len, &cap);
+                if (!istable && len >= JANET_PRETTY_DICT_ONELINE)
+                    janet_buffer_push_u8(S->buffer, ' ');
+                if (is_dict_value && len >= JANET_PRETTY_DICT_ONELINE) print_newline(S, 0);
+                for (i = 0; i < cap; i++) {
+                    if (!janet_checktype(kvs[i].key, JANET_NIL)) {
+                        if (first_kv_pair) {
+                            first_kv_pair = 0;
+                        } else {
+                            print_newline(S, len < JANET_PRETTY_DICT_ONELINE);
+                        }
+                        janet_pretty_one(S, kvs[i].key, 0);
+                        janet_buffer_push_u8(S->buffer, ' ');
+                        janet_pretty_one(S, kvs[i].value, 1);
+                    }
+                }
+            }
+            S->indent -= 2;
+            S->depth++;
+            janet_buffer_push_u8(S->buffer, '}');
+            break;
+        }
     }
     /* Remove from seen */
     janet_table_remove(&S->seen, x);
     return;
 }
 
-/* Helper for printing a janet value in a pretty form. Not meant to be used
- * for serialization or anything like that. */
-JanetBuffer *janet_pretty(JanetBuffer *buffer, int depth, Janet x) {
+static JanetBuffer *janet_pretty_(JanetBuffer *buffer, int depth, int flags, Janet x, int32_t startlen) {
     struct pretty S;
     if (NULL == buffer) {
         buffer = janet_buffer(0);
@@ -421,17 +489,25 @@ JanetBuffer *janet_pretty(JanetBuffer *buffer, int depth, Janet x) {
     S.buffer = buffer;
     S.depth = depth;
     S.indent = 0;
+    S.flags = flags;
+    S.bufstartlen = startlen;
     janet_table_init(&S.seen, 10);
     janet_pretty_one(&S, x, 0);
     janet_table_deinit(&S.seen);
     return S.buffer;
 }
 
+/* Helper for printing a janet value in a pretty form. Not meant to be used
+ * for serialization or anything like that. */
+JanetBuffer *janet_pretty(JanetBuffer *buffer, int depth, int flags, Janet x) {
+    return janet_pretty_(buffer, depth, flags, x, buffer ? buffer->count : 0);
+}
+
 static const char *typestr(Janet x) {
     JanetType t = janet_type(x);
     return (t == JANET_ABSTRACT)
-        ? janet_abstract_type(janet_unwrap_abstract(x))->name
-        : janet_type_names[t];
+           ? janet_abstract_type(janet_unwrap_abstract(x))->name
+           : janet_type_names[t];
 }
 
 static void pushtypes(JanetBuffer *buffer, int types) {
@@ -451,39 +527,18 @@ static void pushtypes(JanetBuffer *buffer, int types) {
     }
 }
 
-/* Helper function for formatting strings. Useful for generating error messages and the like.
- * Similar to printf, but specialized for operating with janet. */
-const uint8_t *janet_formatc(const char *format, ...) {
-    va_list args;
-    int32_t len = 0;
-    int32_t i;
-    const uint8_t *ret;
-    JanetBuffer buffer;
-    JanetBuffer *bufp = &buffer;
-
-    /* Calculate length */
-    while (format[len]) len++;
-
-    /* Initialize buffer */
-    janet_buffer_init(bufp, len);
-
-    /* Start args */
-    va_start(args, format);
-
-    /* Iterate length */
-    for (i = 0; i < len; i++) {
-        uint8_t c = format[i];
-        switch (c) {
+void janet_formatb(JanetBuffer *bufp, const char *format, va_list args) {
+    for (const char *c = format; *c; c++) {
+        switch (*c) {
             default:
-                janet_buffer_push_u8(bufp, c);
+                janet_buffer_push_u8(bufp, *c);
                 break;
-            case '%':
-            {
-                if (i + 1 >= len)
+            case '%': {
+                if (c[1] == '\0')
                     break;
-                switch (format[++i]) {
+                switch (*++c) {
                     default:
-                        janet_buffer_push_u8(bufp, format[i]);
+                        janet_buffer_push_u8(bufp, *c);
                         break;
                     case 'f':
                         number_to_string_b(bufp, va_arg(args, double));
@@ -491,8 +546,7 @@ const uint8_t *janet_formatc(const char *format, ...) {
                     case 'd':
                         integer_to_string_b(bufp, va_arg(args, long));
                         break;
-                    case 'S':
-                    {
+                    case 'S': {
                         const uint8_t *str = va_arg(args, const uint8_t *);
                         janet_buffer_push_bytes(bufp, str, janet_string_length(str));
                         break;
@@ -503,42 +557,59 @@ const uint8_t *janet_formatc(const char *format, ...) {
                     case 'c':
                         janet_buffer_push_u8(bufp, (uint8_t) va_arg(args, long));
                         break;
-                    case 'q':
-                    {
+                    case 'q': {
                         const uint8_t *str = va_arg(args, const uint8_t *);
                         janet_escape_string_b(bufp, str);
                         break;
                     }
-                    case 't':
-                    {
+                    case 't': {
                         janet_buffer_push_cstring(bufp, typestr(va_arg(args, Janet)));
                         break;
                     }
-                    case 'T':
-                    {
+                    case 'T': {
                         int types = va_arg(args, long);
                         pushtypes(bufp, types);
                         break;
                     }
-                    case 'V':
-                    {
+                    case 'V': {
                         janet_to_string_b(bufp, va_arg(args, Janet));
                         break;
                     }
-                    case 'v':
-                    {
+                    case 'v': {
                         janet_description_b(bufp, va_arg(args, Janet));
                         break;
                     }
-                    case 'p':
-                    {
-                        janet_pretty(bufp, 4, va_arg(args, Janet));
+                    case 'p': {
+                        janet_pretty(bufp, 4, 0, va_arg(args, Janet));
+                        break;
+                    }
+                    case 'P': {
+                        janet_pretty(bufp, 4, JANET_PRETTY_COLOR, va_arg(args, Janet));
+                        break;
                     }
                 }
             }
         }
     }
+}
 
+/* Helper function for formatting strings. Useful for generating error messages and the like.
+ * Similar to printf, but specialized for operating with janet. */
+const uint8_t *janet_formatc(const char *format, ...) {
+    va_list args;
+    const uint8_t *ret;
+    JanetBuffer buffer;
+    int32_t len = 0;
+
+    /* Calculate length, init buffer and args */
+    while (format[len]) len++;
+    janet_buffer_init(&buffer, len);
+    va_start(args, format);
+
+    /* Run format */
+    janet_formatb(&buffer, format, args);
+
+    /* Iterate length */
     va_end(args);
 
     ret = janet_string(buffer.data, buffer.count);
@@ -546,3 +617,147 @@ const uint8_t *janet_formatc(const char *format, ...) {
     return ret;
 }
 
+/*
+ * code adapted from lua/lstrlib.c http://lua.org
+ */
+
+#define MAX_ITEM  256
+#define FMT_FLAGS "-+ #0"
+#define MAX_FORMAT 32
+
+static const char *scanformat(
+    const char *strfrmt,
+    char *form,
+    char width[3],
+    char precision[3]) {
+    const char *p = strfrmt;
+    memset(width, '\0', 3);
+    memset(precision, '\0', 3);
+    while (*p != '\0' && strchr(FMT_FLAGS, *p) != NULL)
+        p++; /* skip flags */
+    if ((size_t)(p - strfrmt) >= sizeof(FMT_FLAGS) / sizeof(char))
+        janet_panic("invalid format (repeated flags)");
+    if (isdigit((int)(*p)))
+        width[0] = *p++; /* skip width */
+    if (isdigit((int)(*p)))
+        width[1] = *p++; /* (2 digits at most) */
+    if (*p == '.') {
+        p++;
+        if (isdigit((int)(*p)))
+            precision[0] = *p++; /* skip precision */
+        if (isdigit((int)(*p)))
+            precision[1] = *p++; /* (2 digits at most) */
+    }
+    if (isdigit((int)(*p)))
+        janet_panic("invalid format (width or precision too long)");
+    *(form++) = '%';
+    memcpy(form, strfrmt, ((p - strfrmt) + 1) * sizeof(char));
+    form += (p - strfrmt) + 1;
+    *form = '\0';
+    return p;
+}
+
+/* Shared implementation between string/format and
+ * buffer/format */
+void janet_buffer_format(
+    JanetBuffer *b,
+    const char *strfrmt,
+    int32_t argstart,
+    int32_t argc,
+    Janet *argv) {
+    size_t sfl = strlen(strfrmt);
+    const char *strfrmt_end = strfrmt + sfl;
+    int32_t arg = argstart;
+    int32_t startlen = b->count;
+    while (strfrmt < strfrmt_end) {
+        if (*strfrmt != '%')
+            janet_buffer_push_u8(b, (uint8_t) * strfrmt++);
+        else if (*++strfrmt == '%')
+            janet_buffer_push_u8(b, (uint8_t) * strfrmt++); /* %% */
+        else { /* format item */
+            char form[MAX_FORMAT], item[MAX_ITEM];
+            char width[3], precision[3];
+            int nb = 0; /* number of bytes in added item */
+            if (++arg >= argc)
+                janet_panic("not enough values for format");
+            strfrmt = scanformat(strfrmt, form, width, precision);
+            switch (*strfrmt++) {
+                case 'c': {
+                    nb = snprintf(item, MAX_ITEM, form, (int)
+                                  janet_getinteger(argv, arg));
+                    break;
+                }
+                case 'd':
+                case 'i':
+                case 'o':
+                case 'u':
+                case 'x':
+                case 'X': {
+                    int32_t n = janet_getinteger(argv, arg);
+                    nb = snprintf(item, MAX_ITEM, form, n);
+                    break;
+                }
+                case 'a':
+                case 'A':
+                case 'e':
+                case 'E':
+                case 'f':
+                case 'g':
+                case 'G': {
+                    double d = janet_getnumber(argv, arg);
+                    nb = snprintf(item, MAX_ITEM, form, d);
+                    break;
+                }
+                case 's': {
+                    const uint8_t *s = janet_getstring(argv, arg);
+                    int32_t l = janet_string_length(s);
+                    if (form[2] == '\0')
+                        janet_buffer_push_bytes(b, s, l);
+                    else {
+                        if (l != (int32_t) strlen((const char *) s))
+                            janet_panic("string contains zeros");
+                        if (!strchr(form, '.') && l >= 100) {
+                            janet_panic("no precision and string is too long to be formatted");
+                        } else {
+                            nb = snprintf(item, MAX_ITEM, form, s);
+                        }
+                    }
+                    break;
+                }
+                case 'V': {
+                    janet_to_string_b(b, argv[arg]);
+                    break;
+                }
+                case 'v': {
+                    janet_description_b(b, argv[arg]);
+                    break;
+                }
+                case 'Q':
+                case 'q':
+                case 'P':
+                case 'p': { /* janet pretty , precision = depth */
+                    int depth = atoi(precision);
+                    if (depth < 1)
+                        depth = 4;
+                    char c = strfrmt[-1];
+                    int has_color = (c == 'P') || (c == 'Q');
+                    int has_oneline = (c == 'Q') || (c == 'q');
+                    int flags = 0;
+                    flags |= has_color ? JANET_PRETTY_COLOR : 0;
+                    flags |= has_oneline ? JANET_PRETTY_ONELINE : 0;
+                    janet_pretty_(b, depth, flags, argv[arg], startlen);
+                    break;
+                }
+                default: {
+                    /* also treat cases 'nLlh' */
+                    janet_panicf("invalid conversion '%s' to 'format'",
+                                 form);
+                }
+            }
+            if (nb >= MAX_ITEM)
+                janet_panicf("format buffer overflow", form);
+            if (nb > 0)
+                janet_buffer_push_bytes(b, (uint8_t *) item, nb);
+        }
+    }
+}

src/core/regalloc.c

@@ -21,8 +21,9 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "regalloc.h"
+#include "util.h"
 #endif
 
 void janetc_regalloc_init(JanetcRegisterAllocator *ra) {
@@ -66,12 +67,16 @@ void janetc_regalloc_clone(JanetcRegisterAllocator *dest, JanetcRegisterAllocato
     dest->capacity = src->capacity;
     dest->max = src->max;
     size = sizeof(uint32_t) * dest->capacity;
-    dest->chunks = malloc(size);
     dest->regtemps = 0;
-    if (!dest->chunks) {
-        JANET_OUT_OF_MEMORY;
+    if (size) {
+        dest->chunks = malloc(size);
+        if (!dest->chunks) {
+            JANET_OUT_OF_MEMORY;
+        }
+        memcpy(dest->chunks, src->chunks, size);
+    } else {
+        dest->chunks = NULL;
     }
-    memcpy(dest->chunks, src->chunks, size);
 }
 
 /* Allocate one more chunk in chunks */

src/core/run.c

@@ -21,78 +21,24 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "state.h"
-#include "vector.h"
 #endif
 
-/* Error reporting */
-void janet_stacktrace(JanetFiber *fiber, const char *errtype, Janet err) {
-    int32_t fi;
-    const char *errstr = (const char *)janet_to_string(err);
-    JanetFiber **fibers = NULL;
-    fprintf(stderr, "%s error: %s\n", errtype, errstr);
-
-    while (fiber) {
-        janet_v_push(fibers, fiber);
-        fiber = fiber->child;
-    }
-
-    for (fi = janet_v_count(fibers) - 1; fi >= 0; fi--) {
-        fiber = fibers[fi];
-        int32_t i = fiber->frame;
-        while (i > 0) {
-            JanetStackFrame *frame = (JanetStackFrame *)(fiber->data + i - JANET_FRAME_SIZE);
-            JanetFuncDef *def = NULL;
-            i = frame->prevframe;
-            fprintf(stderr, "  in");
-            if (frame->func) {
-                def = frame->func->def;
-                fprintf(stderr, " %s", def->name ? (const char *)def->name : "<anonymous>");
-                if (def->source) {
-                    fprintf(stderr, " [%s]", (const char *)def->source);
-                }
-            } else {
-                JanetCFunction cfun = (JanetCFunction)(frame->pc);
-                if (cfun) {
-                    Janet name = janet_table_get(janet_vm_registry, janet_wrap_cfunction(cfun));
-                    if (!janet_checktype(name, JANET_NIL))
-                        fprintf(stderr, " %s", (const char *)janet_to_string(name));
-                    else
-                        fprintf(stderr, " <cfunction>");
-                }
-            }
-            if (frame->flags & JANET_STACKFRAME_TAILCALL)
-                fprintf(stderr, " (tailcall)");
-            if (frame->func && frame->pc) {
-                int32_t off = (int32_t) (frame->pc - def->bytecode);
-                if (def->sourcemap) {
-                    JanetSourceMapping mapping = def->sourcemap[off];
-                    fprintf(stderr, " at (%d:%d)", mapping.start, mapping.end);
-                } else {
-                    fprintf(stderr, " pc=%d", off);
-                }
-            }
-            fprintf(stderr, "\n");
-        }
-    }
-
-    janet_v_free(fibers);
-}
-
 /* Run a string */
 int janet_dobytes(JanetTable *env, const uint8_t *bytes, int32_t len, const char *sourcePath, Janet *out) {
     JanetParser parser;
-    int errflags = 0;
+    int errflags = 0, done = 0;
     int32_t index = 0;
-    int dudeol = 0;
-    int done = 0;
     Janet ret = janet_wrap_nil();
     const uint8_t *where = sourcePath ? janet_cstring(sourcePath) : NULL;
+
     if (where) janet_gcroot(janet_wrap_string(where));
+    if (NULL == sourcePath) sourcePath = "<unknown>";
     janet_parser_init(&parser);
 
-    while (!errflags && !done) {
+    /* While we haven't seen an error */
+    while (!done) {
 
         /* Evaluate parsed values */
         while (janet_parser_has_more(&parser)) {
@@ -101,41 +47,47 @@ int janet_dobytes(JanetTable *env, const uint8_t *bytes, int32_t len, const char
             if (cres.status == JANET_COMPILE_OK) {
                 JanetFunction *f = janet_thunk(cres.funcdef);
                 JanetFiber *fiber = janet_fiber(f, 64, 0, NULL);
+                fiber->env = env;
                 JanetSignal status = janet_continue(fiber, janet_wrap_nil(), &ret);
                 if (status != JANET_SIGNAL_OK) {
-                    janet_stacktrace(fiber, "runtime", ret);
+                    janet_stacktrace(fiber, ret);
                     errflags |= 0x01;
+                    done = 1;
                 }
             } else {
-                fprintf(stderr, "source path: %s\n", sourcePath);
-                janet_stacktrace(cres.macrofiber, "compile",
-                        janet_wrap_string(cres.error));
+                if (cres.macrofiber) {
+                    janet_eprintf("compile error in %s: ", sourcePath);
+                    janet_stacktrace(cres.macrofiber, janet_wrap_string(cres.error));
+                } else {
+                    janet_eprintf("compile error in %s: %s\n", sourcePath,
+                                  (const char *)cres.error);
+                }
                 errflags |= 0x02;
+                done = 1;
             }
         }
 
         /* Dispatch based on parse state */
         switch (janet_parser_status(&parser)) {
+            case JANET_PARSE_DEAD:
+                done = 1;
+                break;
             case JANET_PARSE_ERROR:
                 errflags |= 0x04;
-                fprintf(stderr, "parse error: %s\n", janet_parser_error(&parser));
+                janet_eprintf("parse error in %s: %s\n",
+                              sourcePath, janet_parser_error(&parser));
+                done = 1;
                 break;
             case JANET_PARSE_PENDING:
-                if (index >= len) {
-                    if (dudeol) {
-                        errflags |= 0x04;
-                        fprintf(stderr, "internal parse error: unexpected end of source\n");
-                    } else {
-                        dudeol = 1;
-                        janet_parser_consume(&parser, '\n');
-                    }
+                if (index == len) {
+                    janet_parser_eof(&parser);
                 } else {
                     janet_parser_consume(&parser, bytes[index++]);
                 }
                 break;
             case JANET_PARSE_ROOT:
                 if (index >= len) {
-                    done = 1;
+                    janet_parser_eof(&parser);
                 } else {
                     janet_parser_consume(&parser, bytes[index++]);
                 }
@@ -143,6 +95,8 @@ int janet_dobytes(JanetTable *env, const uint8_t *bytes, int32_t len, const char
         }
 
     }
+
+    /* Clean up and return errors */
     janet_parser_deinit(&parser);
     if (where) janet_gcunroot(janet_wrap_string(where));
     if (out) *out = ret;

src/core/specials.c

@@ -21,7 +21,7 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "compile.h"
 #include "util.h"
 #include "vector.h"
@@ -55,50 +55,60 @@ static JanetSlot qq_slots(JanetFopts opts, JanetSlot *slots, int makeop) {
     return target;
 }
 
-static JanetSlot quasiquote(JanetFopts opts, Janet x) {
+static JanetSlot quasiquote(JanetFopts opts, Janet x, int depth, int level) {
+    if (depth == 0) {
+        janetc_cerror(opts.compiler, "quasiquote too deeply nested");
+        return janetc_cslot(janet_wrap_nil());
+    }
     JanetSlot *slots = NULL;
     switch (janet_type(x)) {
         default:
             return janetc_cslot(x);
-        case JANET_TUPLE:
-            {
-                int32_t i, len;
-                const Janet *tup = janet_unwrap_tuple(x);
-                len = janet_tuple_length(tup);
-                if (len > 1 && janet_checktype(tup[0], JANET_SYMBOL)) {
-                    const uint8_t *head = janet_unwrap_symbol(tup[0]);
-                    if (!janet_cstrcmp(head, "unquote"))
+        case JANET_TUPLE: {
+            int32_t i, len;
+            const Janet *tup = janet_unwrap_tuple(x);
+            len = janet_tuple_length(tup);
+            if (len > 1 && janet_checktype(tup[0], JANET_SYMBOL)) {
+                const uint8_t *head = janet_unwrap_symbol(tup[0]);
+                if (!janet_cstrcmp(head, "unquote")) {
+                    if (level == 0) {
                         return janetc_value(janetc_fopts_default(opts.compiler), tup[1]);
+                    } else {
+                        level--;
+                    }
+                } else if (!janet_cstrcmp(head, "quasiquote")) {
+                    level++;
                 }
-                for (i = 0; i < len; i++)
-                    janet_v_push(slots, quasiquote(opts, tup[i]));
-                return qq_slots(opts, slots, JOP_MAKE_TUPLE);
-            }
-        case JANET_ARRAY:
-            {
-                int32_t i;
-                JanetArray *array = janet_unwrap_array(x);
-                for (i = 0; i < array->count; i++)
-                    janet_v_push(slots, quasiquote(opts, array->data[i]));
-                return qq_slots(opts, slots, JOP_MAKE_ARRAY);
             }
+            for (i = 0; i < len; i++)
+                janet_v_push(slots, quasiquote(opts, tup[i], depth - 1, level));
+            return qq_slots(opts, slots, (janet_tuple_flag(tup) & JANET_TUPLE_FLAG_BRACKETCTOR)
+                            ? JOP_MAKE_BRACKET_TUPLE
+                            : JOP_MAKE_TUPLE);
+        }
+        case JANET_ARRAY: {
+            int32_t i;
+            JanetArray *array = janet_unwrap_array(x);
+            for (i = 0; i < array->count; i++)
+                janet_v_push(slots, quasiquote(opts, array->data[i], depth - 1, level));
+            return qq_slots(opts, slots, JOP_MAKE_ARRAY);
+        }
         case JANET_TABLE:
-        case JANET_STRUCT:
-            {
-                const JanetKV *kv = NULL, *kvs = NULL;
-                int32_t len, cap;
-                janet_dictionary_view(x, &kvs, &len, &cap);
-                while ((kv = janet_dictionary_next(kvs, cap, kv))) {
-                    JanetSlot key = quasiquote(opts, kv->key);
-                    JanetSlot value =  quasiquote(opts, kv->value);
-                    key.flags &= ~JANET_SLOT_SPLICED;
-                    value.flags &= ~JANET_SLOT_SPLICED;
-                    janet_v_push(slots, key);
-                    janet_v_push(slots, value);
-                }
-                return qq_slots(opts, slots,
-                        janet_checktype(x, JANET_TABLE) ? JOP_MAKE_TABLE : JOP_MAKE_STRUCT);
+        case JANET_STRUCT: {
+            const JanetKV *kv = NULL, *kvs = NULL;
+            int32_t len, cap = 0;
+            janet_dictionary_view(x, &kvs, &len, &cap);
+            while ((kv = janet_dictionary_next(kvs, cap, kv))) {
+                JanetSlot key = quasiquote(opts, kv->key, depth - 1, level);
+                JanetSlot value =  quasiquote(opts, kv->value, depth - 1, level);
+                key.flags &= ~JANET_SLOT_SPLICED;
+                value.flags &= ~JANET_SLOT_SPLICED;
+                janet_v_push(slots, key);
+                janet_v_push(slots, value);
             }
+            return qq_slots(opts, slots,
+                            janet_checktype(x, JANET_TABLE) ? JOP_MAKE_TABLE : JOP_MAKE_STRUCT);
+        }
     }
 }
 
@@ -107,7 +117,7 @@ static JanetSlot janetc_quasiquote(JanetFopts opts, int32_t argn, const Janet *a
         janetc_cerror(opts.compiler, "expected 1 argument");
         return janetc_cslot(janet_wrap_nil());
     }
-    return quasiquote(opts, argv[0]);
+    return quasiquote(opts, argv[0], JANET_RECURSION_GUARD, 0);
 }
 
 static JanetSlot janetc_unquote(JanetFopts opts, int32_t argn, const Janet *argv) {
@@ -117,17 +127,17 @@ static JanetSlot janetc_unquote(JanetFopts opts, int32_t argn, const Janet *argv
     return janetc_cslot(janet_wrap_nil());
 }
 
-/* Preform destructuring. Be careful to
+/* Perform destructuring. Be careful to
  * keep the order registers are freed.
  * Returns if the slot 'right' can be freed. */
 static int destructure(JanetCompiler *c,
-        Janet left,
-        JanetSlot right,
-        int (*leaf)(JanetCompiler *c,
-            const uint8_t *sym,
-            JanetSlot s,
-            JanetTable *attr),
-        JanetTable *attr) {
+                       Janet left,
+                       JanetSlot right,
+                       int (*leaf)(JanetCompiler *c,
+                                   const uint8_t *sym,
+                                   JanetSlot s,
+                                   JanetTable *attr),
+                       JanetTable *attr) {
     switch (janet_type(left)) {
         default:
             janetc_cerror(c, "unexpected type in destructuring");
@@ -136,50 +146,48 @@ static int destructure(JanetCompiler *c,
             /* Leaf, assign right to left */
             return leaf(c, janet_unwrap_symbol(left), right, attr);
         case JANET_TUPLE:
-        case JANET_ARRAY:
-            {
-                int32_t i, len;
-                const Janet *values;
-                janet_indexed_view(left, &values, &len);
-                for (i = 0; i < len; i++) {
-                    JanetSlot nextright = janetc_farslot(c);
-                    Janet subval = values[i];
-                    if (i < 0x100) {
-                        janetc_emit_ssu(c, JOP_GET_INDEX, nextright, right, (uint8_t) i, 1);
-                    } else {
-                        JanetSlot k = janetc_cslot(janet_wrap_integer(i));
-                        janetc_emit_sss(c, JOP_GET, nextright, right, k, 1);
-                    }
-                    if (destructure(c, subval, nextright, leaf, attr))
-                        janetc_freeslot(c, nextright);
+        case JANET_ARRAY: {
+            int32_t len = 0;
+            const Janet *values = NULL;
+            janet_indexed_view(left, &values, &len);
+            for (int32_t i = 0; i < len; i++) {
+                JanetSlot nextright = janetc_farslot(c);
+                Janet subval = values[i];
+                if (i < 0x100) {
+                    janetc_emit_ssu(c, JOP_GET_INDEX, nextright, right, (uint8_t) i, 1);
+                } else {
+                    JanetSlot k = janetc_cslot(janet_wrap_integer(i));
+                    janetc_emit_sss(c, JOP_IN, nextright, right, k, 1);
                 }
+                if (destructure(c, subval, nextright, leaf, attr))
+                    janetc_freeslot(c, nextright);
             }
-            return 1;
+        }
+        return 1;
         case JANET_TABLE:
-        case JANET_STRUCT:
-            {
-                const JanetKV *kvs = NULL;
-                int32_t i, cap, len;
-                janet_dictionary_view(left, &kvs, &len, &cap);
-                for (i = 0; i < cap; i++) {
-                    if (janet_checktype(kvs[i].key, JANET_NIL)) continue;
-                    JanetSlot nextright = janetc_farslot(c);
-                    JanetSlot k = janetc_value(janetc_fopts_default(c), kvs[i].key);
-                    janetc_emit_sss(c, JOP_GET, nextright, right, k, 1);
-                    if (destructure(c, kvs[i].value, nextright, leaf, attr))
-                        janetc_freeslot(c, nextright);
-                }
+        case JANET_STRUCT: {
+            const JanetKV *kvs = NULL;
+            int32_t cap = 0, len = 0;
+            janet_dictionary_view(left, &kvs, &len, &cap);
+            for (int32_t i = 0; i < cap; i++) {
+                if (janet_checktype(kvs[i].key, JANET_NIL)) continue;
+                JanetSlot nextright = janetc_farslot(c);
+                JanetSlot k = janetc_value(janetc_fopts_default(c), kvs[i].key);
+                janetc_emit_sss(c, JOP_IN, nextright, right, k, 1);
+                if (destructure(c, kvs[i].value, nextright, leaf, attr))
+                    janetc_freeslot(c, nextright);
             }
-            return 1;
+        }
+        return 1;
     }
 }
 
 /* Create a source map for definitions. */
 static const Janet *janetc_make_sourcemap(JanetCompiler *c) {
     Janet *tup = janet_tuple_begin(3);
-    tup[0] = janet_wrap_string(c->source);
-    tup[1] = janet_wrap_integer(c->current_mapping.start);
-    tup[2] = janet_wrap_integer(c->current_mapping.end);
+    tup[0] = c->source ? janet_wrap_string(c->source) : janet_wrap_nil();
+    tup[1] = janet_wrap_integer(c->current_mapping.line);
+    tup[2] = janet_wrap_integer(c->current_mapping.column);
     return janet_tuple_end(tup);
 }
 
@@ -221,7 +229,6 @@ static JanetSlot janetc_varset(JanetFopts opts, int32_t argn, const Janet *argv)
         return rvalue;
     } else {
         /* Error */
-        janet_inspect(argv[0]);
         janetc_cerror(opts.compiler, "expected symbol or tuple for l-value to set");
         return janetc_cslot(janet_wrap_nil());
     }
@@ -265,8 +272,8 @@ static JanetSlot dohead(JanetCompiler *c, JanetFopts opts, Janet *head, int32_t
 /* Def or var a symbol in a local scope */
 static int namelocal(JanetCompiler *c, const uint8_t *head, int32_t flags, JanetSlot ret) {
     int isUnnamedRegister = !(ret.flags & JANET_SLOT_NAMED) &&
-        ret.index > 0 &&
-        ret.envindex >= 0;
+                            ret.index > 0 &&
+                            ret.envindex >= 0;
     if (!isUnnamedRegister) {
         /* Slot is not able to be named */
         JanetSlot localslot = janetc_farslot(c);
@@ -279,21 +286,20 @@ static int namelocal(JanetCompiler *c, const uint8_t *head, int32_t flags, Janet
 }
 
 static int varleaf(
-        JanetCompiler *c,
-        const uint8_t *sym,
-        JanetSlot s,
-        JanetTable *attr) {
+    JanetCompiler *c,
+    const uint8_t *sym,
+    JanetSlot s,
+    JanetTable *reftab) {
     if (c->scope->flags & JANET_SCOPE_TOP) {
         /* Global var, generate var */
         JanetSlot refslot;
-        JanetTable *reftab = janet_table(1);
-        reftab->proto = attr;
+        JanetTable *entry = janet_table_clone(reftab);
         JanetArray *ref = janet_array(1);
         janet_array_push(ref, janet_wrap_nil());
-        janet_table_put(reftab, janet_ckeywordv("ref"), janet_wrap_array(ref));
-        janet_table_put(reftab, janet_ckeywordv("source-map"),
-                janet_wrap_tuple(janetc_make_sourcemap(c)));
-        janet_table_put(c->env, janet_wrap_symbol(sym), janet_wrap_table(reftab));
+        janet_table_put(entry, janet_ckeywordv("ref"), janet_wrap_array(ref));
+        janet_table_put(entry, janet_ckeywordv("source-map"),
+                        janet_wrap_tuple(janetc_make_sourcemap(c)));
+        janet_table_put(c->env, janet_wrap_symbol(sym), janet_wrap_table(entry));
         refslot = janetc_cslot(janet_wrap_array(ref));
         janetc_emit_ssu(c, JOP_PUT_INDEX, refslot, s, 0, 0);
         return 1;
@@ -313,20 +319,19 @@ static JanetSlot janetc_var(JanetFopts opts, int32_t argn, const Janet *argv) {
 }
 
 static int defleaf(
-        JanetCompiler *c,
-        const uint8_t *sym,
-        JanetSlot s,
-        JanetTable *attr) {
+    JanetCompiler *c,
+    const uint8_t *sym,
+    JanetSlot s,
+    JanetTable *tab) {
     if (c->scope->flags & JANET_SCOPE_TOP) {
-        JanetTable *tab = janet_table(2);
-        janet_table_put(tab, janet_ckeywordv("source-map"),
-                janet_wrap_tuple(janetc_make_sourcemap(c)));
-        tab->proto = attr;
+        JanetTable *entry = janet_table_clone(tab);
+        janet_table_put(entry, janet_ckeywordv("source-map"),
+                        janet_wrap_tuple(janetc_make_sourcemap(c)));
         JanetSlot valsym = janetc_cslot(janet_ckeywordv("value"));
-        JanetSlot tabslot = janetc_cslot(janet_wrap_table(tab));
+        JanetSlot tabslot = janetc_cslot(janet_wrap_table(entry));
 
         /* Add env entry to env */
-        janet_table_put(c->env, janet_wrap_symbol(sym), janet_wrap_table(tab));
+        janet_table_put(c->env, janet_wrap_symbol(sym), janet_wrap_table(entry));
 
         /* Put value in table when evaulated */
         janetc_emit_sss(c, JOP_PUT, tabslot, valsym, s, 0);
@@ -383,8 +388,8 @@ static JanetSlot janetc_if(JanetFopts opts, int32_t argn, const Janet *argv) {
 
     /* Set target for compilation */
     target = (drop || tail)
-        ? janetc_cslot(janet_wrap_nil())
-        : janetc_gettarget(opts);
+             ? janetc_cslot(janet_wrap_nil())
+             : janetc_gettarget(opts);
 
     /* Compile condition */
     janetc_scope(&condscope, c, 0, "if");
@@ -477,6 +482,61 @@ static int32_t janetc_addfuncdef(JanetCompiler *c, JanetFuncDef *def) {
     return janet_v_count(scope->defs) - 1;
 }
 
+/*
+ * break
+ *
+ * jump :end or retn if in function
+ */
+static JanetSlot janetc_break(JanetFopts opts, int32_t argn, const Janet *argv) {
+    JanetCompiler *c = opts.compiler;
+    JanetScope *scope = c->scope;
+    if (argn > 1) {
+        janetc_cerror(c, "expected at most 1 argument");
+        return janetc_cslot(janet_wrap_nil());
+    }
+
+    /* Find scope to break from */
+    while (scope) {
+        if (scope->flags & (JANET_SCOPE_FUNCTION | JANET_SCOPE_WHILE))
+            break;
+        scope = scope->parent;
+    }
+    if (NULL == scope) {
+        janetc_cerror(c, "break must occur in while loop or closure");
+        return janetc_cslot(janet_wrap_nil());
+    }
+
+    /* Emit code to break from that scope */
+    JanetFopts subopts = janetc_fopts_default(c);
+    if (scope->flags & JANET_SCOPE_FUNCTION) {
+        if (!(scope->flags & JANET_SCOPE_WHILE) && argn) {
+            /* Closure body with return argument */
+            subopts.flags |= JANET_FOPTS_TAIL;
+            JanetSlot ret = janetc_value(subopts, argv[0]);
+            ret.flags |= JANET_SLOT_RETURNED;
+            return ret;
+        } else {
+            /* while loop IIFE or no argument */
+            if (argn) {
+                subopts.flags |= JANET_FOPTS_DROP;
+                janetc_value(subopts, argv[0]);
+            }
+            janetc_emit(c, JOP_RETURN_NIL);
+            JanetSlot s = janetc_cslot(janet_wrap_nil());
+            s.flags |= JANET_SLOT_RETURNED;
+            return s;
+        }
+    } else {
+        if (argn) {
+            subopts.flags |= JANET_FOPTS_DROP;
+            janetc_value(subopts, argv[0]);
+        }
+        /* Tag the instruction so the while special can turn it into a proper jump */
+        janetc_emit(c, 0x80 | JOP_JUMP);
+        return janetc_cslot(janet_wrap_nil());
+    }
+}
+
 /*
  * :whiletop
  * ...
@@ -501,7 +561,7 @@ static JanetSlot janetc_while(JanetFopts opts, int32_t argn, const Janet *argv)
 
     labelwt = janet_v_count(c->buffer);
 
-    janetc_scope(&tempscope, c, 0, "while");
+    janetc_scope(&tempscope, c, JANET_SCOPE_WHILE, "while");
 
     /* Compile condition */
     cond = janetc_value(subopts, argv[0]);
@@ -519,8 +579,8 @@ static JanetSlot janetc_while(JanetFopts opts, int32_t argn, const Janet *argv)
 
     /* Infinite loop does not need to check condition */
     labelc = infinite
-        ? 0
-        : janetc_emit_si(c, JOP_JUMP_IF_NOT, cond, 0, 0);
+             ? 0
+             : janetc_emit_si(c, JOP_JUMP_IF_NOT, cond, 0, 0);
 
     /* Compile body */
     for (i = 1; i < argn; i++) {
@@ -553,6 +613,7 @@ static JanetSlot janetc_while(JanetFopts opts, int32_t argn, const Janet *argv)
         int32_t tempself = janetc_regalloc_temp(&tempscope.ra, JANETC_REGTEMP_0);
         janetc_emit(c, JOP_LOAD_SELF | (tempself << 8));
         janetc_emit(c, JOP_TAILCALL | (tempself << 8));
+        janetc_regalloc_freetemp(&c->scope->ra, tempself, JANETC_REGTEMP_0);
         /* Compile function */
         JanetFuncDef *def = janetc_pop_funcdef(c);
         def->name = janet_cstring("_while");
@@ -561,7 +622,7 @@ static JanetSlot janetc_while(JanetFopts opts, int32_t argn, const Janet *argv)
         int32_t cloreg = janetc_regalloc_temp(&c->scope->ra, JANETC_REGTEMP_0);
         janetc_emit(c, JOP_CLOSURE | (cloreg << 8) | (defindex << 16));
         janetc_emit(c, JOP_CALL | (cloreg << 8) | (cloreg << 16));
-        janetc_regalloc_free(&c->scope->ra, cloreg);
+        janetc_regalloc_freetemp(&c->scope->ra, cloreg, JANETC_REGTEMP_0);
         c->scope->flags |= JANET_SCOPE_CLOSURE;
         return janetc_cslot(janet_wrap_nil());
     }
@@ -572,8 +633,15 @@ static JanetSlot janetc_while(JanetFopts opts, int32_t argn, const Janet *argv)
 
     /* Calculate jumps */
     labeld = janet_v_count(c->buffer);
-    if (!infinite) c->buffer[labelc] |= (labeld - labelc) << 16;
-    c->buffer[labeljt] |= (labelwt - labeljt) << 8;
+    if (!infinite) c->buffer[labelc] |= (uint32_t)(labeld - labelc) << 16;
+    c->buffer[labeljt] |= (uint32_t)(labelwt - labeljt) << 8;
+
+    /* Calculate breaks */
+    for (int32_t i = labelwt; i < labeld; i++) {
+        if (c->buffer[i] == (0x80 | JOP_JUMP)) {
+            c->buffer[i] = JOP_JUMP | ((labeld - i) << 8);
+        }
+    }
 
     /* Pop scope and return nil slot */
     janetc_popscope(c);
@@ -587,23 +655,25 @@ static JanetSlot janetc_fn(JanetFopts opts, int32_t argn, const Janet *argv) {
     JanetSlot ret;
     Janet head;
     JanetScope fnscope;
-    int32_t paramcount, argi, parami, arity, defindex, i;
+    int32_t paramcount, argi, parami, arity, min_arity, max_arity, defindex, i;
     JanetFopts subopts = janetc_fopts_default(c);
     const Janet *params;
     const char *errmsg = NULL;
 
     /* Function flags */
     int vararg = 0;
-    int fixarity = 1;
+    int structarg = 0;
+    int allow_extra = 0;
     int selfref = 0;
     int seenamp = 0;
+    int seenopt = 0;
 
     /* Begin function */
     c->scope->flags |= JANET_SCOPE_CLOSURE;
     janetc_scope(&fnscope, c, JANET_SCOPE_FUNCTION, "function");
 
-    if (argn < 2) {
-        errmsg = "expected at least 2 arguments to function literal";
+    if (argn == 0) {
+        errmsg = "expected at least 1 argument to function literal";
         goto error;
     }
 
@@ -619,6 +689,9 @@ static JanetSlot janetc_fn(JanetFopts opts, int32_t argn, const Janet *argv) {
         goto error;
     }
 
+    /* Keep track of destructured parameters */
+    JanetSlot *destructed_params = NULL;
+
     /* Compile function parameters */
     params = janet_unwrap_tuple(argv[parami]);
     paramcount = janet_tuple_length(params);
@@ -627,27 +700,68 @@ static JanetSlot janetc_fn(JanetFopts opts, int32_t argn, const Janet *argv) {
         Janet param = params[i];
         if (janet_checktype(param, JANET_SYMBOL)) {
             /* Check for varargs and unfixed arity */
-            if ((!seenamp) &&
-                    (0 == janet_cstrcmp(janet_unwrap_symbol(param), "&"))) {
-                seenamp = 1;
-                fixarity = 0;
-                if (i == paramcount - 1) {
+            if (!janet_cstrcmp(janet_unwrap_symbol(param), "&")) {
+                if (seenamp) {
+                    errmsg = "& in unexpected location";
+                    goto error;
+                } else if (i == paramcount - 1) {
+                    allow_extra = 1;
                     arity--;
                 } else if (i == paramcount - 2) {
                     vararg = 1;
                     arity -= 2;
                 } else {
-                    errmsg = "variable argument symbol in unexpected location";
+                    errmsg = "& in unexpected location";
                     goto error;
                 }
+                seenamp = 1;
+            } else if (!janet_cstrcmp(janet_unwrap_symbol(param), "&opt")) {
+                if (seenopt) {
+                    errmsg = "only one &opt allowed";
+                    goto error;
+                } else if (i == paramcount - 1) {
+                    errmsg = "&opt cannot be last item in parameter list";
+                    goto error;
+                }
+                min_arity = i;
+                arity--;
+                seenopt = 1;
+            } else if (!janet_cstrcmp(janet_unwrap_symbol(param), "&keys")) {
+                if (seenamp) {
+                    errmsg = "&keys in unexpected location";
+                    goto error;
+                } else if (i == paramcount - 2) {
+                    vararg = 1;
+                    structarg = 1;
+                    arity -= 2;
+                } else {
+                    errmsg = "&keys in unexpected location";
+                    goto error;
+                }
+                seenamp = 1;
             } else {
                 janetc_nameslot(c, janet_unwrap_symbol(param), janetc_farslot(c));
             }
         } else {
-            destructure(c, param, janetc_farslot(c), defleaf, NULL);
+            janet_v_push(destructed_params, janetc_farslot(c));
         }
     }
 
+    /* Compile destructed params */
+    int32_t j = 0;
+    for (i = 0; i < paramcount; i++) {
+        Janet param = params[i];
+        if (!janet_checktype(param, JANET_SYMBOL)) {
+            JanetSlot reg = destructed_params[j++];
+            destructure(c, param, reg, defleaf, NULL);
+            janetc_freeslot(c, reg);
+        }
+    }
+    janet_v_free(destructed_params);
+
+    max_arity = (vararg || allow_extra) ? INT32_MAX : arity;
+    if (!seenopt) min_arity = arity;
+
     /* Check for self ref */
     if (selfref) {
         JanetSlot slot = janetc_farslot(c);
@@ -659,18 +773,22 @@ static JanetSlot janetc_fn(JanetFopts opts, int32_t argn, const Janet *argv) {
     /* Compile function body */
     if (parami + 1 == argn) {
         janetc_emit(c, JOP_RETURN_NIL);
-    } else for (argi = parami + 1; argi < argn; argi++) {
-        subopts.flags = (argi == (argn - 1)) ? JANET_FOPTS_TAIL : JANET_FOPTS_DROP;
-        janetc_value(subopts, argv[argi]);
-        if (c->result.status == JANET_COMPILE_ERROR)
-            goto error2;
+    } else {
+        for (argi = parami + 1; argi < argn; argi++) {
+            subopts.flags = (argi == (argn - 1)) ? JANET_FOPTS_TAIL : JANET_FOPTS_DROP;
+            janetc_value(subopts, argv[argi]);
+            if (c->result.status == JANET_COMPILE_ERROR)
+                goto error2;
+        }
     }
 
     /* Build function */
     def = janetc_pop_funcdef(c);
     def->arity = arity;
-    if (fixarity) def->flags |= JANET_FUNCDEF_FLAG_FIXARITY;
+    def->min_arity = min_arity;
+    def->max_arity = max_arity;
     if (vararg) def->flags |= JANET_FUNCDEF_FLAG_VARARG;
+    if (structarg) def->flags |= JANET_FUNCDEF_FLAG_STRUCTARG;
 
     if (selfref) def->name = janet_unwrap_symbol(head);
     defindex = janetc_addfuncdef(c, def);
@@ -692,6 +810,7 @@ error2:
 
 /* Keep in lexicographic order */
 static const JanetSpecial janetc_specials[] = {
+    {"break", janetc_break},
     {"def", janetc_def},
     {"do", janetc_do},
     {"fn", janetc_fn},
@@ -708,9 +827,9 @@ static const JanetSpecial janetc_specials[] = {
 /* Find a special */
 const JanetSpecial *janetc_special(const uint8_t *name) {
     return janet_strbinsearch(
-            &janetc_specials,
-            sizeof(janetc_specials)/sizeof(JanetSpecial),
-            sizeof(JanetSpecial),
-            name);
+               &janetc_specials,
+               sizeof(janetc_specials) / sizeof(JanetSpecial),
+               sizeof(JanetSpecial),
+               name);
 }
 

src/core/state.h

@@ -32,6 +32,9 @@
  * be in it. However, thread local global variables for interpreter
  * state should allow easy multi-threading. */
 
+/* Cache the core environment */
+extern JANET_THREAD_LOCAL JanetTable *janet_vm_core_env;
+
 /* How many VM stacks have been entered */
 extern JANET_THREAD_LOCAL int janet_vm_stackn;
 
@@ -39,6 +42,11 @@ extern JANET_THREAD_LOCAL int janet_vm_stackn;
  * Set and unset by janet_run. */
 extern JANET_THREAD_LOCAL JanetFiber *janet_vm_fiber;
 
+/* The current pointer to the inner most jmp_buf. The current
+ * return point for panics. */
+extern JANET_THREAD_LOCAL jmp_buf *janet_vm_jmp_buf;
+extern JANET_THREAD_LOCAL Janet *janet_vm_return_reg;
+
 /* The global registry for c functions. Used to store meta-data
  * along with otherwise bare c function pointers. */
 extern JANET_THREAD_LOCAL JanetTable *janet_vm_registry;
@@ -60,4 +68,15 @@ extern JANET_THREAD_LOCAL Janet *janet_vm_roots;
 extern JANET_THREAD_LOCAL uint32_t janet_vm_root_count;
 extern JANET_THREAD_LOCAL uint32_t janet_vm_root_capacity;
 
+/* Scratch memory */
+extern JANET_THREAD_LOCAL void **janet_scratch_mem;
+extern JANET_THREAD_LOCAL size_t janet_scratch_cap;
+extern JANET_THREAD_LOCAL size_t janet_scratch_len;
+
+/* Setup / teardown */
+#ifdef JANET_THREADS
+void janet_threads_init(void);
+void janet_threads_deinit(void);
+#endif
+
 #endif /* JANET_STATE_H_defined */

src/core/string.c

@@ -23,7 +23,7 @@
 #include <string.h>
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "gc.h"
 #include "util.h"
 #include "state.h"
@@ -31,11 +31,11 @@
 
 /* Begin building a string */
 uint8_t *janet_string_begin(int32_t length) {
-    char *data = janet_gcalloc(JANET_MEMORY_STRING, 2 * sizeof(int32_t) + length + 1);
-    uint8_t *str = (uint8_t *) (data + 2 * sizeof(int32_t));
-    janet_string_length(str) = length;
-    str[length] = 0;
-    return str;
+    JanetStringHead *head = janet_gcalloc(JANET_MEMORY_STRING, sizeof(JanetStringHead) + length + 1);
+    head->length = length;
+    uint8_t *data = (uint8_t *)head->data;
+    data[length] = 0;
+    return data;
 }
 
 /* Finish building a string */
@@ -46,14 +46,13 @@ const uint8_t *janet_string_end(uint8_t *str) {
 
 /* Load a buffer as a string */
 const uint8_t *janet_string(const uint8_t *buf, int32_t len) {
-    int32_t hash = janet_string_calchash(buf, len);
-    char *data = janet_gcalloc(JANET_MEMORY_STRING, 2 * sizeof(int32_t) + len + 1);
-    uint8_t *str = (uint8_t *) (data + 2 * sizeof(int32_t));
-    memcpy(str, buf, len);
-    str[len] = 0;
-    janet_string_length(str) = len;
-    janet_string_hash(str) = hash;
-    return str;
+    JanetStringHead *head = janet_gcalloc(JANET_MEMORY_STRING, sizeof(JanetStringHead) + len + 1);
+    head->length = len;
+    head->hash = janet_string_calchash(buf, len);
+    uint8_t *data = (uint8_t *)head->data;
+    memcpy(data, buf, len);
+    data[len] = 0;
+    return data;
 }
 
 /* Compare two strings */
@@ -81,7 +80,7 @@ int janet_string_equalconst(const uint8_t *lhs, const uint8_t *rhs, int32_t rlen
 /* Check if two strings are equal */
 int janet_string_equal(const uint8_t *lhs, const uint8_t *rhs) {
     return janet_string_equalconst(lhs, rhs,
-            janet_string_length(rhs), janet_string_hash(rhs));
+                                   janet_string_length(rhs), janet_string_hash(rhs));
 }
 
 /* Load a c string */
@@ -102,9 +101,12 @@ struct kmp_state {
 };
 
 static void kmp_init(
-        struct kmp_state *s,
-        const uint8_t *text, int32_t textlen,
-        const uint8_t *pat, int32_t patlen) {
+    struct kmp_state *s,
+    const uint8_t *text, int32_t textlen,
+    const uint8_t *pat, int32_t patlen) {
+    if (patlen == 0) {
+        janet_panic("expected non-empty pattern");
+    }
     int32_t *lookup = calloc(patlen, sizeof(int32_t));
     if (!lookup) {
         JANET_OUT_OF_MEMORY;
@@ -168,8 +170,8 @@ static int32_t kmp_next(struct kmp_state *state) {
 /* CFuns */
 
 static Janet cfun_string_slice(int32_t argc, Janet *argv) {
-    JanetRange range = janet_getslice(argc, argv);
     JanetByteView view = janet_getbytes(argv, 0);
+    JanetRange range = janet_getslice(argc, argv);
     return janet_stringv(view.bytes + range.start, range.end - range.start);
 }
 
@@ -183,8 +185,7 @@ static Janet cfun_string_repeat(int32_t argc, Janet *argv) {
     if (mulres > INT32_MAX) janet_panic("result string is too long");
     uint8_t *newbuf = janet_string_begin((int32_t) mulres);
     uint8_t *end = newbuf + mulres;
-    uint8_t *p = newbuf;
-    for (p = newbuf; p < end; p += view.len) {
+    for (uint8_t *p = newbuf; p < end; p += view.len) {
         memcpy(p, view.bytes, view.len);
     }
     return janet_wrap_string(janet_string_end(newbuf));
@@ -272,8 +273,28 @@ static Janet cfun_string_find(int32_t argc, Janet *argv) {
     result = kmp_next(&state);
     kmp_deinit(&state);
     return result < 0
-        ? janet_wrap_nil()
-        : janet_wrap_integer(result);
+           ? janet_wrap_nil()
+           : janet_wrap_integer(result);
+}
+
+static Janet cfun_string_hasprefix(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 2);
+    JanetByteView prefix = janet_getbytes(argv, 0);
+    JanetByteView str = janet_getbytes(argv, 1);
+    return str.len < prefix.len
+           ? janet_wrap_false()
+           : janet_wrap_boolean(memcmp(prefix.bytes, str.bytes, prefix.len) == 0);
+}
+
+static Janet cfun_string_hassuffix(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 2);
+    JanetByteView suffix = janet_getbytes(argv, 0);
+    JanetByteView str = janet_getbytes(argv, 1);
+    return str.len < suffix.len
+           ? janet_wrap_false()
+           : janet_wrap_boolean(memcmp(suffix.bytes,
+                                       str.bytes + str.len - suffix.len,
+                                       suffix.len) == 0);
 }
 
 static Janet cfun_string_findall(int32_t argc, Janet *argv) {
@@ -324,8 +345,8 @@ static Janet cfun_string_replace(int32_t argc, Janet *argv) {
     memcpy(buf, s.kmp.text, result);
     memcpy(buf + result, s.subst, s.substlen);
     memcpy(buf + result + s.substlen,
-            s.kmp.text + result + s.kmp.patlen,
-            s.kmp.textlen - result - s.kmp.patlen);
+           s.kmp.text + result + s.kmp.patlen,
+           s.kmp.textlen - result - s.kmp.patlen);
     kmp_deinit(&s.kmp);
     return janet_wrap_string(janet_string_end(buf));
 }
@@ -360,40 +381,32 @@ static Janet cfun_string_split(int32_t argc, Janet *argv) {
     }
     findsetup(argc, argv, &state, 1);
     array = janet_array(0);
-    while ((result = kmp_next(&state)) >= 0 && limit--) {
+    while ((result = kmp_next(&state)) >= 0 && --limit) {
         const uint8_t *slice = janet_string(state.text + lastindex, result - lastindex);
         janet_array_push(array, janet_wrap_string(slice));
         lastindex = result + state.patlen;
     }
-    {
-        const uint8_t *slice = janet_string(state.text + lastindex, state.textlen - lastindex);
-        janet_array_push(array, janet_wrap_string(slice));
-    }
+    const uint8_t *slice = janet_string(state.text + lastindex, state.textlen - lastindex);
+    janet_array_push(array, janet_wrap_string(slice));
     kmp_deinit(&state);
     return janet_wrap_array(array);
 }
 
 static Janet cfun_string_checkset(int32_t argc, Janet *argv) {
     uint32_t bitset[8] = {0, 0, 0, 0, 0, 0, 0, 0};
-    janet_arity(argc, 2, 3);
+    janet_fixarity(argc, 2);
     JanetByteView set = janet_getbytes(argv, 0);
     JanetByteView str = janet_getbytes(argv, 1);
     /* Populate set */
     for (int32_t i = 0; i < set.len; i++) {
         int index = set.bytes[i] >> 5;
-        uint32_t mask = 1 << (set.bytes[i] & 7);
+        uint32_t mask = 1 << (set.bytes[i] & 0x1F);
         bitset[index] |= mask;
     }
-    if (argc == 3) {
-        if (janet_getboolean(argv, 2)) {
-            for (int i = 0; i < 8; i++)
-                bitset[i] = ~bitset[i];
-        }
-    }
     /* Check set */
     for (int32_t i = 0; i < str.len; i++) {
         int index = str.bytes[i] >> 5;
-        uint32_t mask = 1 << (str.bytes[i] & 7);
+        uint32_t mask = 1 << (str.bytes[i] & 0x1F);
         if (!(bitset[index] & mask)) {
             return janet_wrap_false();
         }
@@ -441,183 +454,203 @@ static Janet cfun_string_join(int32_t argc, Janet *argv) {
     return janet_wrap_string(janet_string_end(buf));
 }
 
-static struct formatter {
-    const char *lead;
-    const char *f1;
-    const char *f2;
-} formatters[] = {
-    {"g", "%g", "%.*g"},
-    {"G", "%G", "%.*G"},
-    {"e", "%e", "%.*e"},
-    {"E", "%E", "%.*E"},
-    {"f", "%f", "%.*f"},
-    {"F", "%F", "%.*F"}
-};
-
-static Janet cfun_string_number(int32_t argc, Janet *argv) {
-    janet_arity(argc, 1, 4);
-    double x = janet_getnumber(argv, 0);
-    struct formatter fmter = formatters[0];
-    char buf[100];
-    int formatNargs = 1;
-    int32_t precision = 0;
-    if (argc >= 2) {
-        const uint8_t *flag = janet_getkeyword(argv, 1);
-        int i;
-        for (i = 0; i < 6; i++) {
-            struct formatter fmttest = formatters[i];
-            if (!janet_cstrcmp(flag, fmttest.lead)) {
-                fmter = fmttest;
-                break;
-            }
-        }
-        if (i == 6)
-            janet_panicf("unsupported formatter %v", argv[1]);
-    }
-
-    if (argc >= 3) {
-        precision = janet_getinteger(argv, 2);
-        formatNargs++;
-    }
-
-    if (formatNargs == 1) {
-        snprintf(buf, sizeof(buf), fmter.f1, x);
-    } else if (formatNargs == 2) {
-        snprintf(buf, sizeof(buf), fmter.f2, precision, x);
-    }
-
-    return janet_cstringv(buf);
+static Janet cfun_string_format(int32_t argc, Janet *argv) {
+    janet_arity(argc, 1, -1);
+    JanetBuffer *buffer = janet_buffer(0);
+    const char *strfrmt = (const char *) janet_getstring(argv, 0);
+    janet_buffer_format(buffer, strfrmt, 0, argc, argv);
+    return janet_stringv(buffer->data, buffer->count);
 }
 
-static Janet cfun_string_pretty(int32_t argc, Janet *argv) {
-    janet_arity(argc, 1, 3);
-    JanetBuffer *buffer = NULL;
-    int32_t depth = 4;
-    if (argc > 1)
-        depth = janet_getinteger(argv, 1);
-    if (argc > 2)
-        buffer = janet_getbuffer(argv, 2);
-    buffer = janet_pretty(buffer, depth, argv[0]);
-    return janet_wrap_buffer(buffer);
+static int trim_help_checkset(JanetByteView set, uint8_t x) {
+    for (int32_t j = 0; j < set.len; j++)
+        if (set.bytes[j] == x)
+            return 1;
+    return 0;
+}
+
+static int32_t trim_help_leftedge(JanetByteView str, JanetByteView set) {
+    for (int32_t i = 0; i < str.len; i++)
+        if (!trim_help_checkset(set, str.bytes[i]))
+            return i;
+    return str.len;
+}
+
+static int32_t trim_help_rightedge(JanetByteView str, JanetByteView set) {
+    for (int32_t i = str.len - 1; i >= 0; i--)
+        if (!trim_help_checkset(set, str.bytes[i]))
+            return i + 1;
+    return 0;
+}
+
+static void trim_help_args(int32_t argc, Janet *argv, JanetByteView *str, JanetByteView *set) {
+    janet_arity(argc, 1, 2);
+    *str = janet_getbytes(argv, 0);
+    if (argc >= 2) {
+        *set = janet_getbytes(argv, 1);
+    } else {
+        set->bytes = (const uint8_t *)(" \t\r\n\v\f");
+        set->len = 6;
+    }
+}
+
+static Janet cfun_string_trim(int32_t argc, Janet *argv) {
+    JanetByteView str, set;
+    trim_help_args(argc, argv, &str, &set);
+    int32_t left_edge = trim_help_leftedge(str, set);
+    int32_t right_edge = trim_help_rightedge(str, set);
+    if (right_edge < left_edge)
+        return janet_stringv(NULL, 0);
+    return janet_stringv(str.bytes + left_edge, right_edge - left_edge);
+}
+
+static Janet cfun_string_triml(int32_t argc, Janet *argv) {
+    JanetByteView str, set;
+    trim_help_args(argc, argv, &str, &set);
+    int32_t left_edge = trim_help_leftedge(str, set);
+    return janet_stringv(str.bytes + left_edge, str.len - left_edge);
+}
+
+static Janet cfun_string_trimr(int32_t argc, Janet *argv) {
+    JanetByteView str, set;
+    trim_help_args(argc, argv, &str, &set);
+    int32_t right_edge = trim_help_rightedge(str, set);
+    return janet_stringv(str.bytes, right_edge);
 }
 
 static const JanetReg string_cfuns[] = {
     {
         "string/slice", cfun_string_slice,
-        JDOC("(string/slice bytes [,start=0 [,end=(length str)]])\n\n"
-                "Returns a substring from a byte sequence. The substring is from "
-                "index start inclusive to index end exclusive. All indexing "
-                "is from 0. 'start' and 'end' can also be negative to indicate indexing "
-                "from the end of the string.")
+        JDOC("(string/slice bytes &opt start end)\n\n"
+             "Returns a substring from a byte sequence. The substring is from "
+             "index start inclusive to index end exclusive. All indexing "
+             "is from 0. 'start' and 'end' can also be negative to indicate indexing "
+             "from the end of the string. Note that index -1 is synonymous with "
+             "index (length bytes) to allow a full negative slice range. ")
     },
     {
         "string/repeat", cfun_string_repeat,
         JDOC("(string/repeat bytes n)\n\n"
-                "Returns a string that is n copies of bytes concatenated.")
+             "Returns a string that is n copies of bytes concatenated.")
     },
     {
         "string/bytes", cfun_string_bytes,
         JDOC("(string/bytes str)\n\n"
-                "Returns an array of integers that are the byte values of the string.")
+             "Returns an array of integers that are the byte values of the string.")
     },
     {
         "string/from-bytes", cfun_string_frombytes,
-        JDOC("(string/from-bytes byte-array)\n\n"
-                "Creates a string from an array of integers with byte values. All integers "
-                "will be coerced to the range of 1 byte 0-255.")
+        JDOC("(string/from-bytes & byte-vals)\n\n"
+             "Creates a string from integer params with byte values. All integers "
+             "will be coerced to the range of 1 byte 0-255.")
     },
     {
         "string/ascii-lower", cfun_string_asciilower,
         JDOC("(string/ascii-lower str)\n\n"
-                "Returns a new string where all bytes are replaced with the "
-                "lowercase version of themselves in ASCII. Does only a very simple "
-                "case check, meaning no unicode support.")
+             "Returns a new string where all bytes are replaced with the "
+             "lowercase version of themselves in ASCII. Does only a very simple "
+             "case check, meaning no unicode support.")
     },
     {
         "string/ascii-upper", cfun_string_asciiupper,
         JDOC("(string/ascii-upper str)\n\n"
-                "Returns a new string where all bytes are replaced with the "
-                "uppercase version of themselves in ASCII. Does only a very simple "
-                "case check, meaning no unicode support.")
+             "Returns a new string where all bytes are replaced with the "
+             "uppercase version of themselves in ASCII. Does only a very simple "
+             "case check, meaning no unicode support.")
     },
     {
         "string/reverse", cfun_string_reverse,
         JDOC("(string/reverse str)\n\n"
-                "Returns a string that is the reversed version of str.")
+             "Returns a string that is the reversed version of str.")
     },
     {
         "string/find", cfun_string_find,
         JDOC("(string/find patt str)\n\n"
-                "Searches for the first instance of pattern patt in string "
-                "str. Returns the index of the first character in patt if found, "
-                "otherwise returns nil.")
+             "Searches for the first instance of pattern patt in string "
+             "str. Returns the index of the first character in patt if found, "
+             "otherwise returns nil.")
     },
     {
         "string/find-all", cfun_string_findall,
         JDOC("(string/find patt str)\n\n"
-                "Searches for all instances of pattern patt in string "
-                "str. Returns an array of all indices of found patterns. Overlapping "
-                "instances of the pattern are not counted, meaning a byte in string "
-                "will only contribute to finding at most on occurrence of pattern. If no "
-                "occurrences are found, will return an empty array.")
+             "Searches for all instances of pattern patt in string "
+             "str. Returns an array of all indices of found patterns. Overlapping "
+             "instances of the pattern are not counted, meaning a byte in string "
+             "will only contribute to finding at most on occurrence of pattern. If no "
+             "occurrences are found, will return an empty array.")
+    },
+    {
+        "string/has-prefix?", cfun_string_hasprefix,
+        JDOC("(string/has-prefix? pfx str)\n\n"
+             "Tests whether str starts with pfx.")
+    },
+    {
+        "string/has-suffix?", cfun_string_hassuffix,
+        JDOC("(string/has-suffix? sfx str)\n\n"
+             "Tests whether str ends with sfx.")
     },
     {
         "string/replace", cfun_string_replace,
         JDOC("(string/replace patt subst str)\n\n"
-                "Replace the first occurrence of patt with subst in the string str. "
-                "Will return the new string if patt is found, otherwise returns str.")
+             "Replace the first occurrence of patt with subst in the string str. "
+             "Will return the new string if patt is found, otherwise returns str.")
     },
     {
         "string/replace-all", cfun_string_replaceall,
         JDOC("(string/replace-all patt subst str)\n\n"
-                "Replace all instances of patt with subst in the string str. "
-                "Will return the new string if patt is found, otherwise returns str.")
+             "Replace all instances of patt with subst in the string str. "
+             "Will return the new string if patt is found, otherwise returns str.")
     },
     {
         "string/split", cfun_string_split,
-        JDOC("(string/split delim str)\n\n"
-                "Splits a string str with delimiter delim and returns an array of "
-                "substrings. The substrings will not contain the delimiter delim. If delim "
-                "is not found, the returned array will have one element.")
+        JDOC("(string/split delim str &opt start limit)\n\n"
+             "Splits a string str with delimiter delim and returns an array of "
+             "substrings. The substrings will not contain the delimiter delim. If delim "
+             "is not found, the returned array will have one element. Will start searching "
+             "for delim at the index start (if provided), and return up to a maximum "
+             "of limit results (if provided).")
     },
     {
         "string/check-set", cfun_string_checkset,
         JDOC("(string/check-set set str)\n\n"
-                "Checks if any of the bytes in the string set appear in the string str. "
-                "Returns true if some bytes in set do appear in str, false if no bytes do.")
+             "Checks that the string str only contains bytes that appear in the string set. "
+             "Returns true if all bytes in str appear in set, false if some bytes in str do "
+             "not appear in set.")
     },
     {
         "string/join", cfun_string_join,
-        JDOC("(string/join parts [,sep])\n\n"
-                "Joins an array of strings into one string, optionally separated by "
-                "a separator string sep.")
+        JDOC("(string/join parts &opt sep)\n\n"
+             "Joins an array of strings into one string, optionally separated by "
+             "a separator string sep.")
     },
     {
-        "string/number", cfun_string_number,
-        JDOC("(string/number x [,format [,maxlen [,precision]]])\n\n"
-                "Formats a number as string. The format parameter indicates how "
-                "to display the number, either as floating point, scientific, or "
-                "whichever representation is shorter. format can be:\n\n"
-                "\t:g - (default) shortest representation with lowercase e.\n"
-                "\t:G - shortest representation with uppercase E.\n"
-                "\t:e - scientific with lowercase e.\n"
-                "\t:E - scientific with uppercase E.\n"
-                "\t:f - floating point representation.\n"
-                "\t:F - same as :f\n\n"
-                "The programmer can also specify the max length of the output string "
-                "and the precision (number of places after decimal) in the output number. "
-                "Returns a string representation of x.")
+        "string/format", cfun_string_format,
+        JDOC("(string/format format & values)\n\n"
+             "Similar to snprintf, but specialized for operating with janet. Returns "
+             "a new string.")
     },
     {
-        "string/pretty", cfun_string_pretty,
-        JDOC("(string/pretty x [,depth=4 [,buffer=@\"\"]])\n\n"
-                "Pretty prints a value to a buffer. Optionally allows setting max "
-                "recursion depth, as well as writing to a buffer. Returns the buffer.")
+        "string/trim", cfun_string_trim,
+        JDOC("(string/trim str &opt set)\n\n"
+             "Trim leading and trailing whitespace from a byte sequence. If the argument "
+             "set is provided, consider only characters in set to be whitespace.")
+    },
+    {
+        "string/triml", cfun_string_triml,
+        JDOC("(string/triml str &opt set)\n\n"
+             "Trim leading whitespace from a byte sequence. If the argument "
+             "set is provided, consider only characters in set to be whitespace.")
+    },
+    {
+        "string/trimr", cfun_string_trimr,
+        JDOC("(string/trimr str &opt set)\n\n"
+             "Trim trailing whitespace from a byte sequence. If the argument "
+             "set is provided, consider only characters in set to be whitespace.")
     },
     {NULL, NULL, NULL}
 };
 
 /* Module entry point */
 void janet_lib_string(JanetTable *env) {
-    janet_cfuns(env, NULL, string_cfuns);
+    janet_core_cfuns(env, NULL, string_cfuns);
 }

src/core/strtod.c

@@ -26,42 +26,39 @@
  * This version has been modified for much greater flexibility in parsing, such
  * as choosing the radix and supporting scientific notation with any radix.
  *
- * Numbers are of the form [-+]R[rR]I.F[eE&][-+]X where R is the radix, I is
- * the integer part, F is the fractional part, and X is the exponent. All
- * signs, radix, decimal point, fractional part, and exponent can be omitted.
- * The number will be considered and integer if the there is no decimal point
- * and no exponent. Any number greater the 2^32-1 or less than -(2^32) will be
- * coerced to a double. If there is an error, the function janet_scan_number will
- * return a janet nil. The radix is assumed to be 10 if omitted, and the E
+ * Numbers are of the form [-+]R[rR]I.F[eE&][-+]X in pseudo-regex form, where R
+ * is the radix, I is the integer part, F is the fractional part, and X is the
+ * exponent. All signs, radix, decimal point, fractional part, and exponent can
+ * be omitted.  The radix is assumed to be 10 if omitted, and the E or e
  * separator for the exponent can only be used when the radix is 10. This is
- * because E is a valid digit in bases 15 or greater. For bases greater than 10,
- * the letters are used as digits. A through Z correspond to the digits 10
+ * because E is a valid digit in bases 15 or greater. For bases greater than
+ * 10, the letters are used as digits. A through Z correspond to the digits 10
  * through 35, and the lowercase letters have the same values. The radix number
  * is always in base 10. For example, a hexidecimal number could be written
  * '16rdeadbeef'. janet_scan_number also supports some c style syntax for
  * hexidecimal literals. The previous number could also be written
- * '0xdeadbeef'. Note that in this case, the number will actually be a double
- * as it will not fit in the range for a signed 32 bit integer. The string
- * '0xbeef' would parse to an integer as it is in the range of an int32_t. */
+ * '0xdeadbeef'.
+ */
 
 #include <math.h>
 #include <string.h>
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
+#include "util.h"
 #endif
 
 /* Lookup table for getting values of characters when parsing numbers. Handles
  * digits 0-9 and a-z (and A-Z). A-Z have values of 10 to 35. */
 static uint8_t digit_lookup[128] = {
-    0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,
-    0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,
-    0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,
-    0,1,2,3,4,5,6,7,8,9,0xff,0xff,0xff,0xff,0xff,0xff,
-    0xff,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,
-    25,26,27,28,29,30,31,32,33,34,35,0xff,0xff,0xff,0xff,0xff,
-    0xff,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,
-    25,26,27,28,29,30,31,32,33,34,35,0xff,0xff,0xff,0xff,0xff
+    0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
+    0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
+    0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
+    0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
+    0xff, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,
+    25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 0xff, 0xff, 0xff, 0xff, 0xff,
+    0xff, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,
+    25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 0xff, 0xff, 0xff, 0xff, 0xff
 };
 
 #define BIGNAT_NBIT 31
@@ -75,6 +72,7 @@ struct BigNat {
     uint32_t *digits; /* Each digit is base (2 ^ 31). Digits are least significant first. */
 };
 
+/* Initialize a bignat to 0 */
 static void bignat_zero(struct BigNat *x) {
     x->first_digit = 0;
     x->n = 0;
@@ -125,7 +123,7 @@ static void bignat_div(struct BigNat *mant, uint32_t divisor) {
     int32_t i;
     uint32_t quotient, remainder;
     uint64_t dividend;
-    remainder = 0;
+    remainder = 0, quotient = 0;
     for (i = mant->n - 1; i >= 0; i--) {
         dividend = ((uint64_t)remainder * BIGNAT_BASE) + mant->digits[i];
         if (i < mant->n - 1) mant->digits[i + 1] = quotient;
@@ -197,29 +195,38 @@ static double bignat_extract(struct BigNat *mant, int32_t exponent2) {
 }
 
 /* Read in a mantissa and exponent of a certain base, and give
- * back the double value. Should properly handle 0s, Infinities, and
- * denormalized numbers. (When the exponent values are too large) */
+ * back the double value. Should properly handle 0s, infinities, and
+ * denormalized numbers. (When the exponent values are too large or small) */
 static double convert(
-        int negative,
-        struct BigNat *mant,
-        int32_t base,
-        int32_t exponent) {
+    int negative,
+    struct BigNat *mant,
+    int32_t base,
+    int32_t exponent) {
 
     int32_t exponent2 = 0;
 
-    /* Short circuit zero and huge numbers */
+    /* Approximate exponent in base 2 of mant and exponent. This should get us a good estimate of the final size of the
+     * number, within * 2^32 or so. */
+    int32_t mant_exp2_approx = mant->n * 32 + 16;
+    int32_t exp_exp2_approx = (int32_t)(floor(log2(base) * exponent));
+    int32_t exp2_approx = mant_exp2_approx + exp_exp2_approx;
+
+    /* Short circuit zero, huge, and small numbers. We use the exponent range of valid IEEE754 doubles (-1022, 1023)
+     * with a healthy buffer to allow for inaccuracies in the approximation and denormailzed numbers. */
     if (mant->n == 0 && mant->first_digit == 0)
         return negative ? -0.0 : 0.0;
-    if (exponent > 1023)
+    if (exp2_approx > 1176)
         return negative ? -INFINITY : INFINITY;
+    if (exp2_approx < -1175)
+        return negative ? -0.0 : 0.0;
 
     /* Final value is X = mant * base ^ exponent * 2 ^ exponent2
      * Get exponent to zero while holding X constant. */
 
     /* Positive exponents are simple */
-    for (;exponent > 3; exponent -= 4) bignat_muladd(mant, base * base * base * base, 0);
-    for (;exponent > 1; exponent -= 2) bignat_muladd(mant, base * base, 0);
-    for (;exponent > 0; exponent -= 1) bignat_muladd(mant, base, 0);
+    for (; exponent > 3; exponent -= 4) bignat_muladd(mant, base * base * base * base, 0);
+    for (; exponent > 1; exponent -= 2) bignat_muladd(mant, base * base, 0);
+    for (; exponent > 0; exponent -= 1) bignat_muladd(mant, base, 0);
 
     /* Negative exponents are tricky - we don't want to loose bits
      * from integer division, so we need to premultiply. */
@@ -227,22 +234,22 @@ static double convert(
         int32_t shamt = 5 - exponent / 4;
         bignat_lshift_n(mant, shamt);
         exponent2 -= shamt * BIGNAT_NBIT;
-        for (;exponent < -3; exponent += 4) bignat_div(mant, base * base * base * base);
-        for (;exponent < -2; exponent += 2) bignat_div(mant, base * base);
-        for (;exponent <  0; exponent += 1) bignat_div(mant, base);
+        for (; exponent < -3; exponent += 4) bignat_div(mant, base * base * base * base);
+        for (; exponent < -1; exponent += 2) bignat_div(mant, base * base);
+        for (; exponent <  0; exponent += 1) bignat_div(mant, base);
     }
 
     return negative
-        ? -bignat_extract(mant, exponent2)
-        : bignat_extract(mant, exponent2);
+           ? -bignat_extract(mant, exponent2)
+           : bignat_extract(mant, exponent2);
 }
 
 /* Scan a real (double) from a string. If the string cannot be converted into
  * and integer, set *err to 1 and return 0. */
 int janet_scan_number(
-        const uint8_t *str,
-        int32_t len,
-        double *out) {
+    const uint8_t *str,
+    int32_t len,
+    double *out) {
     const uint8_t *end = str + len;
     int seenadigit = 0;
     int ex = 0;
@@ -274,14 +281,14 @@ int janet_scan_number(
         base = 16;
         str += 2;
     } else if (str + 1 < end  &&
-            str[0] >= '0' && str[0] <= '9' &&
-            str[1] == 'r') {
+               str[0] >= '0' && str[0] <= '9' &&
+               str[1] == 'r') {
         base = str[0] - '0';
         str += 2;
     } else if (str + 2 < end  &&
-            str[0] >= '0' && str[0] <= '9' &&
-            str[1] >= '0' && str[1] <= '9' &&
-            str[2] == 'r') {
+               str[0] >= '0' && str[0] <= '9' &&
+               str[1] >= '0' && str[1] <= '9' &&
+               str[2] == 'r') {
         base = 10 * (str[0] - '0') + (str[1] - '0');
         if (base < 2 || base > 36) goto error;
         str += 3;
@@ -293,8 +300,9 @@ int janet_scan_number(
         if (*str == '.') {
             if (seenpoint) goto error;
             seenpoint = 1;
+        } else {
+            seenadigit = 1;
         }
-        seenadigit = 1;
         str++;
     }
 
@@ -327,7 +335,7 @@ int janet_scan_number(
     /* Read exponent */
     if (str < end && foundexp) {
         int eneg = 0;
-        int ee = 0;
+        int32_t ee = 0;
         seenadigit = 0;
         str++;
         if (str >= end) goto error;
@@ -342,14 +350,17 @@ int janet_scan_number(
             str++;
             seenadigit = 1;
         }
-        while (str < end && ee < (INT32_MAX / 40)) {
+        while (str < end) {
             int digit = digit_lookup[*str & 0x7F];
             if (*str > 127 || digit >= base) goto error;
-            ee = base * ee + digit;
+            if (ee < (INT32_MAX / 40)) {
+                ee = base * ee + digit;
+            }
             str++;
             seenadigit = 1;
         }
-        if (eneg) ex -= ee; else ex += ee;
+        if (eneg) ex -= ee;
+        else ex += ee;
     }
 
     if (!seenadigit)
@@ -359,7 +370,104 @@ int janet_scan_number(
     free(mant.digits);
     return 0;
 
-    error:
+error:
     free(mant.digits);
     return 1;
 }
+
+#ifdef JANET_INT_TYPES
+
+static int scan_uint64(
+    const uint8_t *str,
+    int32_t len,
+    uint64_t *out,
+    int *neg) {
+    const uint8_t *end = str + len;
+    int seenadigit = 0;
+    int base = 10;
+    *neg = 0;
+    *out = 0;
+    uint64_t accum = 0;
+    /* len max is INT64_MAX in base 2 with _ between each bits */
+    /* '2r' + 64 bits + 63 _  + sign = 130 => 150 for some leading  */
+    /* zeros */
+    if (len > 150) return 0;
+    /* Get sign */
+    if (str >= end) return 0;
+    if (*str == '-') {
+        *neg = 1;
+        str++;
+    } else if (*str == '+') {
+        str++;
+    }
+    /* Check for leading 0x or digit digit r */
+    if (str + 1 < end && str[0] == '0' && str[1] == 'x') {
+        base = 16;
+        str += 2;
+    } else if (str + 1 < end  &&
+               str[0] >= '0' && str[0] <= '9' &&
+               str[1] == 'r') {
+        base = str[0] - '0';
+        str += 2;
+    } else if (str + 2 < end  &&
+               str[0] >= '0' && str[0] <= '9' &&
+               str[1] >= '0' && str[1] <= '9' &&
+               str[2] == 'r') {
+        base = 10 * (str[0] - '0') + (str[1] - '0');
+        if (base < 2 || base > 36) return 0;
+        str += 3;
+    }
+
+    /* Skip leading zeros */
+    while (str < end && *str == '0') {
+        seenadigit = 1;
+        str++;
+    }
+    /* Parse significant digits */
+    while (str < end) {
+        if (*str == '_') {
+            if (!seenadigit) return 0;
+        } else {
+            int digit = digit_lookup[*str & 0x7F];
+            if (*str > 127 || digit >= base) return 0;
+            if (accum > (UINT64_MAX - digit) / base) return 0;
+            accum = accum * base + digit;
+            seenadigit = 1;
+        }
+        str++;
+    }
+
+    if (!seenadigit) return 0;
+    *out = accum;
+    return 1;
+}
+
+int janet_scan_int64(const uint8_t *str, int32_t len, int64_t *out) {
+    int neg;
+    uint64_t bi;
+    if (scan_uint64(str, len, &bi, &neg)) {
+        if (neg && bi <= 0x8000000000000000ULL) {
+            *out = -((int64_t) bi);
+            return 1;
+        }
+        if (!neg && bi <= 0x7FFFFFFFFFFFFFFFULL) {
+            *out = bi;
+            return 1;
+        }
+    }
+    return 0;
+}
+
+int janet_scan_uint64(const uint8_t *str, int32_t len, uint64_t *out) {
+    int neg;
+    uint64_t bi;
+    if (scan_uint64(str, len, &bi, &neg)) {
+        if (!neg) {
+            *out = bi;
+            return 1;
+        }
+    }
+    return 0;
+}
+
+#endif

src/core/struct.c

@@ -21,25 +21,26 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "gc.h"
 #include "util.h"
+#include <math.h>
 #endif
 
 /* Begin creation of a struct */
 JanetKV *janet_struct_begin(int32_t count) {
-
     /* Calculate capacity as power of 2 after 2 * count. */
     int32_t capacity = janet_tablen(2 * count);
     if (capacity < 0) capacity = janet_tablen(count + 1);
 
-    size_t s = sizeof(int32_t) * 4 + (capacity * sizeof(JanetKV));
-    char *data = janet_gcalloc(JANET_MEMORY_STRUCT, s);
-    JanetKV *st = (JanetKV *) (data + 4 * sizeof(int32_t));
+    size_t size = sizeof(JanetStructHead) + capacity * sizeof(JanetKV);
+    JanetStructHead *head = janet_gcalloc(JANET_MEMORY_STRUCT, size);
+    head->length = count;
+    head->capacity = capacity;
+    head->hash = 0;
+
+    JanetKV *st = (JanetKV *)(head->data);
     janet_memempty(st, capacity);
-    janet_struct_length(st) = count;
-    janet_struct_capacity(st) = capacity;
-    janet_struct_hash(st) = 0;
     return st;
 }
 
@@ -73,59 +74,58 @@ void janet_struct_put(JanetKV *st, Janet key, Janet value) {
     int32_t i, j, dist;
     int32_t bounds[4] = {index, cap, 0, index};
     if (janet_checktype(key, JANET_NIL) || janet_checktype(value, JANET_NIL)) return;
+    if (janet_checktype(key, JANET_NUMBER) && isnan(janet_unwrap_number(key))) return;
     /* Avoid extra items */
     if (janet_struct_hash(st) == janet_struct_length(st)) return;
     for (dist = 0, j = 0; j < 4; j += 2)
-    for (i = bounds[j]; i < bounds[j + 1]; i++, dist++) {
-        int status;
-        int32_t otherhash;
-        int32_t otherindex, otherdist;
-        JanetKV *kv = st + i;
-        /* We found an empty slot, so just add key and value */
-        if (janet_checktype(kv->key, JANET_NIL)) {
-            kv->key = key;
-            kv->value = value;
-            /* Update the temporary count */
-            janet_struct_hash(st)++;
-            return;
+        for (i = bounds[j]; i < bounds[j + 1]; i++, dist++) {
+            int status;
+            int32_t otherhash;
+            int32_t otherindex, otherdist;
+            JanetKV *kv = st + i;
+            /* We found an empty slot, so just add key and value */
+            if (janet_checktype(kv->key, JANET_NIL)) {
+                kv->key = key;
+                kv->value = value;
+                /* Update the temporary count */
+                janet_struct_hash(st)++;
+                return;
+            }
+            /* Robinhood hashing - check if colliding kv pair
+             * is closer to their source than current. We use robinhood
+             * hashing to ensure that equivalent structs that are constructed
+             * with different order have the same internal layout, and therefor
+             * will compare properly - i.e., {1 2 3 4} should equal {3 4 1 2}.
+             * Collisions are resolved via an insertion sort insertion. */
+            otherhash = janet_hash(kv->key);
+            otherindex = janet_maphash(cap, otherhash);
+            otherdist = (i + cap - otherindex) & (cap - 1);
+            if (dist < otherdist)
+                status = -1;
+            else if (otherdist < dist)
+                status = 1;
+            else if (hash < otherhash)
+                status = -1;
+            else if (otherhash < hash)
+                status = 1;
+            else
+                status = janet_compare(key, kv->key);
+            /* If other is closer to their ideal slot */
+            if (status == 1) {
+                /* Swap current kv pair with pair in slot */
+                JanetKV temp = *kv;
+                kv->key = key;
+                kv->value = value;
+                key = temp.key;
+                value = temp.value;
+                /* Save dist and hash of new kv pair */
+                dist = otherdist;
+                hash = otherhash;
+            } else if (status == 0) {
+                /* A key was added to the struct more than once */
+                return;
+            }
         }
-        /* Robinhood hashing - check if colliding kv pair
-         * is closer to their source than current. We use robinhood
-         * hashing to ensure that equivalent structs that are constructed
-         * with different order have the same internal layout, and therefor
-         * will compare properly - i.e., {1 2 3 4} should equal {3 4 1 2}.
-         * Collisions are resolved via an insertion sort insertion. */
-        otherhash = janet_hash(kv->key);
-        otherindex = janet_maphash(cap, otherhash);
-        otherdist = (i + cap - otherindex) & (cap - 1);
-        if (dist < otherdist)
-            status = -1;
-        else if (otherdist < dist)
-            status = 1;
-        else if (hash < otherhash)
-            status = -1;
-        else if (otherhash < hash)
-            status = 1;
-        else
-            status = janet_compare(key, kv->key);
-        /* If other is closer to their ideal slot */
-        if (status == 1) {
-            /* Swap current kv pair with pair in slot */
-            JanetKV temp = *kv;
-            kv->key = key;
-            kv->value = value;
-            key = temp.key;
-            value = temp.value;
-            /* Save dist and hash of new kv pair */
-            dist = otherdist;
-            hash = otherhash;
-        } else if (status == 0) {
-            /* This should not happen - it means
-             * than a key was added to the struct more than once */
-            janet_exit("struct double put fail");
-            return;
-        }
-    }
 }
 
 /* Finish building a struct */
@@ -134,15 +134,8 @@ const JanetKV *janet_struct_end(JanetKV *st) {
         /* Error building struct, probably duplicate values. We need to rebuild
          * the struct using only the values that went in. The second creation should always
          * succeed. */
-        int32_t i, realCount;
-        JanetKV *newst;
-        realCount = 0;
-        for (i = 0; i < janet_struct_capacity(st); i++) {
-            JanetKV *kv = st + i;
-            realCount += janet_checktype(kv->key, JANET_NIL) ? 1 : 0;
-        }
-        newst = janet_struct_begin(realCount);
-        for (i = 0; i < janet_struct_capacity(st); i++) {
+        JanetKV *newst = janet_struct_begin(janet_struct_hash(st));
+        for (int32_t i = 0; i < janet_struct_capacity(st); i++) {
             JanetKV *kv = st + i;
             if (!janet_checktype(kv->key, JANET_NIL)) {
                 janet_struct_put(newst, kv->key, kv->value);

src/core/symcache.c

@@ -25,11 +25,14 @@
  * checks, all symbols are interned so that there is a single copy of it in the
  * whole program. Equality is then just a pointer check. */
 
+#include <string.h>
+
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "state.h"
 #include "gc.h"
 #include "util.h"
+#include "symcache.h"
 #endif
 
 /* Cache state */
@@ -41,7 +44,7 @@ JANET_THREAD_LOCAL uint32_t janet_vm_cache_deleted = 0;
 /* Initialize the cache (allocate cache memory) */
 void janet_symcache_init() {
     janet_vm_cache_capacity = 1024;
-    janet_vm_cache = calloc(1, janet_vm_cache_capacity * sizeof(const uint8_t **));
+    janet_vm_cache = calloc(1, janet_vm_cache_capacity * sizeof(const uint8_t *));
     if (NULL == janet_vm_cache) {
         JANET_OUT_OF_MEMORY;
     }
@@ -65,10 +68,10 @@ static const uint8_t JANET_SYMCACHE_DELETED[1] = {0};
  * If the item is not found, return the location
  * where one would put it. */
 static const uint8_t **janet_symcache_findmem(
-        const uint8_t *str,
-        int32_t len,
-        int32_t hash,
-        int *success) {
+    const uint8_t *str,
+    int32_t len,
+    int32_t hash,
+    int *success) {
     uint32_t bounds[4];
     uint32_t i, j, index;
     const uint8_t **firstEmpty = NULL;
@@ -81,7 +84,7 @@ static const uint8_t **janet_symcache_findmem(
     bounds[2] = 0;
     bounds[3] = index;
     for (j = 0; j < 4; j += 2)
-        for (i = bounds[j]; i < bounds[j+1]; ++i) {
+        for (i = bounds[j]; i < bounds[j + 1]; ++i) {
             const uint8_t *test = janet_vm_cache[i];
             /* Check empty spots */
             if (NULL == test) {
@@ -106,7 +109,7 @@ static const uint8_t **janet_symcache_findmem(
                 return janet_vm_cache + i;
             }
         }
-    notfound:
+notfound:
     *success = 0;
     return firstEmpty;
 }
@@ -118,7 +121,7 @@ static const uint8_t **janet_symcache_findmem(
 static void janet_cache_resize(uint32_t newCapacity) {
     uint32_t i, oldCapacity;
     const uint8_t **oldCache = janet_vm_cache;
-    const uint8_t **newCache = calloc(1, newCapacity * sizeof(const uint8_t **));
+    const uint8_t **newCache = calloc(1, newCapacity * sizeof(const uint8_t *));
     if (newCache == NULL) {
         JANET_OUT_OF_MEMORY;
     }
@@ -175,10 +178,10 @@ const uint8_t *janet_symbol(const uint8_t *str, int32_t len) {
     const uint8_t **bucket = janet_symcache_findmem(str, len, hash, &success);
     if (success)
         return *bucket;
-    newstr = (uint8_t *) janet_gcalloc(JANET_MEMORY_SYMBOL, 2 * sizeof(int32_t) + len + 1)
-        + (2 * sizeof(int32_t));
-    janet_string_hash(newstr) = hash;
-    janet_string_length(newstr) = len;
+    JanetStringHead *head = janet_gcalloc(JANET_MEMORY_SYMBOL, sizeof(JanetStringHead) + len + 1);
+    head->hash = hash;
+    head->length = len;
+    newstr = (uint8_t *)(head->data);
     memcpy(newstr, str, len);
     newstr[len] = 0;
     janet_symcache_put((const uint8_t *)newstr, bucket);
@@ -187,9 +190,7 @@ const uint8_t *janet_symbol(const uint8_t *str, int32_t len) {
 
 /* Get a symbol from a cstring */
 const uint8_t *janet_csymbol(const char *cstr) {
-    int32_t len = 0;
-    while (cstr[len]) len++;
-    return janet_symbol((const uint8_t *)cstr, len);
+    return janet_symbol((const uint8_t *)cstr, (int32_t) strlen(cstr));
 }
 
 /* Store counter for genysm to avoid quadratic behavior */
@@ -225,21 +226,19 @@ const uint8_t *janet_symbol_gen(void) {
      * is enough for resolving collisions. */
     do {
         hash = janet_string_calchash(
-                gensym_counter,
-                sizeof(gensym_counter) - 1);
+                   gensym_counter,
+                   sizeof(gensym_counter) - 1);
         bucket = janet_symcache_findmem(
-                gensym_counter,
-                sizeof(gensym_counter) - 1,
-                hash,
-                &status);
+                     gensym_counter,
+                     sizeof(gensym_counter) - 1,
+                     hash,
+                     &status);
     } while (status && (inc_gensym(), 1));
-    sym = (uint8_t *) janet_gcalloc(
-            JANET_MEMORY_SYMBOL,
-            2 * sizeof(int32_t) + sizeof(gensym_counter)) +
-        (2 * sizeof(int32_t));
+    JanetStringHead *head = janet_gcalloc(JANET_MEMORY_SYMBOL, sizeof(JanetStringHead) + sizeof(gensym_counter));
+    head->length = sizeof(gensym_counter) - 1;
+    head->hash = hash;
+    sym = (uint8_t *)(head->data);
     memcpy(sym, gensym_counter, sizeof(gensym_counter));
-    janet_string_length(sym) = sizeof(gensym_counter) - 1;
-    janet_string_hash(sym) = hash;
     janet_symcache_put((const uint8_t *)sym, bucket);
     return (const uint8_t *)sym;
 }

src/core/symcache.h

@@ -24,7 +24,7 @@
 #define JANET_SYMCACHE_H_defined
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #endif
 
 /* Initialize the cache (allocate cache memory) */

src/core/table.c

@@ -21,19 +21,38 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "gc.h"
 #include "util.h"
+#include <math.h>
 #endif
 
-/* Initialize a table */
-JanetTable *janet_table_init(JanetTable *table, int32_t capacity) {
+#define JANET_TABLE_FLAG_STACK 0x10000
+
+static void *janet_memalloc_empty_local(int32_t count) {
+    int32_t i;
+    void *mem = janet_smalloc(count * sizeof(JanetKV));
+    JanetKV *mmem = (JanetKV *)mem;
+    for (i = 0; i < count; i++) {
+        JanetKV *kv = mmem + i;
+        kv->key = janet_wrap_nil();
+        kv->value = janet_wrap_nil();
+    }
+    return mem;
+}
+
+static JanetTable *janet_table_init_impl(JanetTable *table, int32_t capacity, int stackalloc) {
     JanetKV *data;
     capacity = janet_tablen(capacity);
+    if (stackalloc) table->gc.flags = JANET_TABLE_FLAG_STACK;
     if (capacity) {
-        data = (JanetKV *) janet_memalloc_empty(capacity);
-        if (NULL == data) {
-            JANET_OUT_OF_MEMORY;
+        if (stackalloc) {
+            data = janet_memalloc_empty_local(capacity);
+        } else {
+            data = (JanetKV *) janet_memalloc_empty(capacity);
+            if (NULL == data) {
+                JANET_OUT_OF_MEMORY;
+            }
         }
         table->data = data;
         table->capacity = capacity;
@@ -47,15 +66,20 @@ JanetTable *janet_table_init(JanetTable *table, int32_t capacity) {
     return table;
 }
 
+/* Initialize a table */
+JanetTable *janet_table_init(JanetTable *table, int32_t capacity) {
+    return janet_table_init_impl(table, capacity, 1);
+}
+
 /* Deinitialize a table */
 void janet_table_deinit(JanetTable *table) {
-    free(table->data);
+    janet_sfree(table->data);
 }
 
 /* Create a new table */
 JanetTable *janet_table(int32_t capacity) {
     JanetTable *table = janet_gcalloc(JANET_MEMORY_TABLE, sizeof(JanetTable));
-    return janet_table_init(table, capacity);
+    return janet_table_init_impl(table, capacity, 0);
 }
 
 /* Find the bucket that contains the given key. Will also return
@@ -67,9 +91,15 @@ JanetKV *janet_table_find(JanetTable *t, Janet key) {
 /* Resize the dictionary table. */
 static void janet_table_rehash(JanetTable *t, int32_t size) {
     JanetKV *olddata = t->data;
-    JanetKV *newdata = (JanetKV *) janet_memalloc_empty(size);
-    if (NULL == newdata) {
-        JANET_OUT_OF_MEMORY;
+    JanetKV *newdata;
+    int islocal = t->gc.flags & JANET_TABLE_FLAG_STACK;
+    if (islocal) {
+        newdata = (JanetKV *) janet_memalloc_empty_local(size);
+    } else {
+        newdata = (JanetKV *) janet_memalloc_empty(size);
+        if (NULL == newdata) {
+            JANET_OUT_OF_MEMORY;
+        }
     }
     int32_t i, oldcapacity;
     oldcapacity = t->capacity;
@@ -83,7 +113,11 @@ static void janet_table_rehash(JanetTable *t, int32_t size) {
             *newkv = *kv;
         }
     }
-    free(olddata);
+    if (islocal) {
+        janet_sfree(olddata);
+    } else {
+        free(olddata);
+    }
 }
 
 /* Get a value out of the table */
@@ -103,6 +137,27 @@ Janet janet_table_get(JanetTable *t, Janet key) {
     return janet_wrap_nil();
 }
 
+/* Get a value out of the table, and record which prototype it was from. */
+Janet janet_table_get_ex(JanetTable *t, Janet key, JanetTable **which) {
+    JanetKV *bucket = janet_table_find(t, key);
+    if (NULL != bucket && !janet_checktype(bucket->key, JANET_NIL)) {
+        *which = t;
+        return bucket->value;
+    }
+    /* Check prototypes */
+    {
+        int i;
+        for (i = JANET_MAX_PROTO_DEPTH, t = t->proto; t && i; t = t->proto, --i) {
+            bucket = janet_table_find(t, key);
+            if (NULL != bucket && !janet_checktype(bucket->key, JANET_NIL)) {
+                *which = t;
+                return bucket->value;
+            }
+        }
+    }
+    return janet_wrap_nil();
+}
+
 /* Get a value out of the table. Don't check prototype tables. */
 Janet janet_table_rawget(JanetTable *t, Janet key) {
     JanetKV *bucket = janet_table_find(t, key);
@@ -131,6 +186,7 @@ Janet janet_table_remove(JanetTable *t, Janet key) {
 /* Put a value into the object */
 void janet_table_put(JanetTable *t, Janet key, Janet value) {
     if (janet_checktype(key, JANET_NIL)) return;
+    if (janet_checktype(key, JANET_NUMBER) && isnan(janet_unwrap_number(key))) return;
     if (janet_checktype(value, JANET_NIL)) {
         janet_table_remove(t, key);
     } else {
@@ -142,7 +198,7 @@ void janet_table_put(JanetTable *t, Janet key, Janet value) {
                 janet_table_rehash(t, janet_tablen(2 * t->count + 2));
             }
             bucket = janet_table_find(t, key);
-            if (janet_checktype(bucket->value, JANET_FALSE))
+            if (janet_checktype(bucket->value, JANET_BOOLEAN))
                 --t->deleted;
             bucket->key = key;
             bucket->value = value;
@@ -173,6 +229,21 @@ const JanetKV *janet_table_to_struct(JanetTable *t) {
     return janet_struct_end(st);
 }
 
+/* Clone a table. */
+JanetTable *janet_table_clone(JanetTable *table) {
+    JanetTable *newTable = janet_gcalloc(JANET_MEMORY_TABLE, sizeof(JanetTable));
+    newTable->count = table->count;
+    newTable->capacity = table->capacity;
+    newTable->deleted = table->deleted;
+    newTable->proto = table->proto;
+    newTable->data = malloc(newTable->capacity * sizeof(JanetKV));
+    if (NULL == newTable->data) {
+        JANET_OUT_OF_MEMORY;
+    }
+    memcpy(newTable->data, table->data, table->capacity * sizeof(JanetKV));
+    return newTable;
+}
+
 /* Merge a table or struct into a table */
 static void janet_table_mergekv(JanetTable *table, const JanetKV *kvs, int32_t cap) {
     int32_t i;
@@ -206,8 +277,8 @@ static Janet cfun_table_getproto(int32_t argc, Janet *argv) {
     janet_fixarity(argc, 1);
     JanetTable *t = janet_gettable(argv, 0);
     return t->proto
-        ? janet_wrap_table(t->proto)
-        : janet_wrap_nil();
+           ? janet_wrap_table(t->proto)
+           : janet_wrap_nil();
 }
 
 static Janet cfun_table_setproto(int32_t argc, Janet *argv) {
@@ -233,43 +304,55 @@ static Janet cfun_table_rawget(int32_t argc, Janet *argv) {
     return janet_table_rawget(table, argv[1]);
 }
 
+static Janet cfun_table_clone(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    JanetTable *table = janet_gettable(argv, 0);
+    return janet_wrap_table(janet_table_clone(table));
+}
+
 static const JanetReg table_cfuns[] = {
     {
         "table/new", cfun_table_new,
         JDOC("(table/new capacity)\n\n"
-                "Creates a new empty table with pre-allocated memory "
-                "for capacity entries. This means that if one knows the number of "
-                "entries going to go in a table on creation, extra memory allocation "
-                "can be avoided. Returns the new table.")
+             "Creates a new empty table with pre-allocated memory "
+             "for capacity entries. This means that if one knows the number of "
+             "entries going to go in a table on creation, extra memory allocation "
+             "can be avoided. Returns the new table.")
     },
     {
         "table/to-struct", cfun_table_tostruct,
         JDOC("(table/to-struct tab)\n\n"
-                "Convert a table to a struct. Returns a new struct. This function "
-                "does not take into account prototype tables.")
+             "Convert a table to a struct. Returns a new struct. This function "
+             "does not take into account prototype tables.")
     },
     {
         "table/getproto", cfun_table_getproto,
         JDOC("(table/getproto tab)\n\n"
-                "Get the prototype table of a table. Returns nil if a table "
-                "has no prototype, otherwise returns the prototype.")
+             "Get the prototype table of a table. Returns nil if a table "
+             "has no prototype, otherwise returns the prototype.")
     },
     {
         "table/setproto", cfun_table_setproto,
         JDOC("(table/setproto tab proto)\n\n"
-                "Set the prototype of a table. Returns the original table tab.")
+             "Set the prototype of a table. Returns the original table tab.")
     },
     {
         "table/rawget", cfun_table_rawget,
         JDOC("(table/rawget tab key)\n\n"
-                "Gets a value from a table without looking at the prototype table. "
-                "If a table tab does not contain t directly, the function will return "
-                "nil without checking the prototype. Returns the value in the table.")
+             "Gets a value from a table without looking at the prototype table. "
+             "If a table tab does not contain t directly, the function will return "
+             "nil without checking the prototype. Returns the value in the table.")
+    },
+    {
+        "table/clone", cfun_table_clone,
+        JDOC("(table/clone tab)\n\n"
+             "Create a copy of a table. Updates to the new table will not change the old table, "
+             "and vice versa.")
     },
     {NULL, NULL, NULL}
 };
 
 /* Load the table module */
 void janet_lib_table(JanetTable *env) {
-    janet_cfuns(env, NULL, table_cfuns);
+    janet_core_cfuns(env, NULL, table_cfuns);
 }

src/core/thread.c

@@ -0,0 +1,650 @@
+/*
+* Copyright (c) 2019 Calvin Rose
+*
+* Permission is hereby granted, free of charge, to any person obtaining a copy
+* of this software and associated documentation files (the "Software"), to
+* deal in the Software without restriction, including without limitation the
+* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+* sell copies of the Software, and to permit persons to whom the Software is
+* furnished to do so, subject to the following conditions:
+*
+* The above copyright notice and this permission notice shall be included in
+* all copies or substantial portions of the Software.
+*
+* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+* IN THE SOFTWARE.
+*/
+
+#ifndef JANET_AMALG
+#include <janet.h>
+#include "gc.h"
+#include "util.h"
+#include "state.h"
+#endif
+
+#ifdef JANET_THREADS
+
+#ifdef JANET_WINDOWS
+#include <windows.h>
+#else
+#include <setjmp.h>
+#include <time.h>
+#include <pthread.h>
+#endif
+
+/* typedefed in janet.h */
+struct JanetMailbox {
+
+    /* Synchronization */
+#ifdef JANET_WINDOWS
+    CRITICAL_SECTION lock;
+    CONDITION_VARIABLE cond;
+#else
+    pthread_mutex_t lock;
+    pthread_cond_t cond;
+#endif
+
+    /* Receiving messages - (only by owner thread)  */
+    JanetTable *decode;
+
+    /* Setup procedure - requires a parent mailbox
+     * to receive thunk from */
+    JanetMailbox *parent;
+
+    /* Memory management - reference counting */
+    int refCount;
+    int closed;
+
+    /* Store messages */
+    uint16_t messageCapacity;
+    uint16_t messageCount;
+    uint16_t messageFirst;
+    uint16_t messageNext;
+
+    /* Buffers to store messages. These buffers are manually allocated, so
+     * are not owned by any thread's GC. */
+    JanetBuffer messages[];
+};
+
+static JANET_THREAD_LOCAL JanetMailbox *janet_vm_mailbox = NULL;
+static JANET_THREAD_LOCAL JanetThread *janet_vm_thread_current = NULL;
+
+static JanetMailbox *janet_mailbox_create(JanetMailbox *parent, int refCount, uint16_t capacity) {
+    JanetMailbox *mailbox = malloc(sizeof(JanetMailbox) + sizeof(JanetBuffer) * capacity);
+    if (NULL == mailbox) {
+        JANET_OUT_OF_MEMORY;
+    }
+#ifdef JANET_WINDOWS
+    InitializeCriticalSection(&mailbox->lock);
+    InitializeConditionVariable(&mailbox->cond);
+#else
+    pthread_mutex_init(&mailbox->lock, NULL);
+    pthread_cond_init(&mailbox->cond, NULL);
+#endif
+    mailbox->refCount = refCount;
+    mailbox->closed = 0;
+    mailbox->parent = parent;
+    mailbox->messageCount = 0;
+    mailbox->messageCapacity = capacity;
+    mailbox->messageFirst = 0;
+    mailbox->messageNext = 0;
+    for (uint16_t i = 0; i < capacity; i++) {
+        janet_buffer_init(mailbox->messages + i, 0);
+    }
+    return mailbox;
+}
+
+static void janet_mailbox_destroy(JanetMailbox *mailbox) {
+#ifdef JANET_WINDOWS
+    DeleteCriticalSection(&mailbox->lock);
+#else
+    pthread_mutex_destroy(&mailbox->lock);
+    pthread_cond_destroy(&mailbox->cond);
+#endif
+    for (uint16_t i = 0; i < mailbox->messageCapacity; i++) {
+        janet_buffer_deinit(mailbox->messages + i);
+    }
+    free(mailbox);
+}
+
+static void janet_mailbox_lock(JanetMailbox *mailbox) {
+#ifdef JANET_WINDOWS
+    EnterCriticalSection(&mailbox->lock);
+#else
+    pthread_mutex_lock(&mailbox->lock);
+#endif
+}
+
+static void janet_mailbox_unlock(JanetMailbox *mailbox) {
+#ifdef JANET_WINDOWS
+    LeaveCriticalSection(&mailbox->lock);
+#else
+    pthread_mutex_unlock(&mailbox->lock);
+#endif
+}
+
+/* Assumes you have the mailbox lock already */
+static void janet_mailbox_ref_with_lock(JanetMailbox *mailbox, int delta) {
+    mailbox->refCount += delta;
+    if (mailbox->refCount <= 0) {
+        janet_mailbox_unlock(mailbox);
+        janet_mailbox_destroy(mailbox);
+    } else {
+        janet_mailbox_unlock(mailbox);
+    }
+}
+
+static void janet_mailbox_ref(JanetMailbox *mailbox, int delta) {
+    janet_mailbox_lock(mailbox);
+    janet_mailbox_ref_with_lock(mailbox, delta);
+}
+
+static void janet_close_thread(JanetThread *thread) {
+    if (thread->mailbox) {
+        janet_mailbox_ref(thread->mailbox, -1);
+        thread->mailbox = NULL;
+    }
+}
+
+static int thread_gc(void *p, size_t size) {
+    (void) size;
+    JanetThread *thread = (JanetThread *)p;
+    janet_close_thread(thread);
+    return 0;
+}
+
+static int thread_mark(void *p, size_t size) {
+    (void) size;
+    JanetThread *thread = (JanetThread *)p;
+    if (thread->encode) {
+        janet_mark(janet_wrap_table(thread->encode));
+    }
+    return 0;
+}
+
+/* Abstract waiting for timeout across windows/posix */
+typedef struct {
+    int timedwait;
+    int nowait;
+#ifdef JANET_WINDOWS
+    DWORD interval;
+    DWORD ticksLeft;
+#else
+    struct timespec ts;
+#endif
+} JanetWaiter;
+
+static void janet_waiter_init(JanetWaiter *waiter, double sec) {
+    waiter->timedwait = 0;
+    waiter->nowait = 0;
+
+    if (sec == 0.0 || isnan(sec)) {
+        waiter->nowait = 1;
+        return;
+    }
+    waiter->timedwait = sec > 0.0;
+
+    /* Set maximum wait time to 30 days */
+    if (sec > (60.0 * 60.0 * 24.0 * 30.0)) {
+        sec = 60.0 * 60.0 * 24.0 * 30.0;
+    }
+
+#ifdef JANET_WINDOWS
+    if (waiter->timedwait) {
+        waiter->ticksLeft = waiter->interval = (DWORD) floor(1000.0 * sec);
+    }
+#else
+    if (waiter->timedwait) {
+        /* N seconds -> timespec of (now + sec) */
+        struct timespec now;
+        clock_gettime(CLOCK_REALTIME, &now);
+        time_t tvsec = (time_t) floor(sec);
+        long tvnsec = (long) floor(1000000000.0 * (sec - ((double) tvsec)));
+        tvsec += now.tv_sec;
+        tvnsec += now.tv_nsec;
+        if (tvnsec >= 1000000000L) {
+            tvnsec -= 1000000000L;
+            tvsec += 1;
+        }
+        waiter->ts.tv_sec = tvsec;
+        waiter->ts.tv_nsec = tvnsec;
+    }
+#endif
+}
+
+static int janet_waiter_wait(JanetWaiter *wait, JanetMailbox *mailbox) {
+    if (wait->nowait) return 1;
+#ifdef JANET_WINDOWS
+    if (wait->timedwait) {
+        if (wait->ticksLeft == 0) return 1;
+        DWORD startTime = GetTickCount();
+        int status = !SleepConditionVariableCS(&mailbox->cond, &mailbox->lock, wait->ticksLeft);
+        DWORD dTick = GetTickCount() - startTime;
+        /* Be careful about underflow */
+        wait->ticksLeft = dTick > wait->ticksLeft ? 0 : dTick;
+        return status;
+    } else {
+        SleepConditionVariableCS(&mailbox->cond, &mailbox->lock, INFINITE);
+        return 0;
+    }
+#else
+    if (wait->timedwait) {
+        return pthread_cond_timedwait(&mailbox->cond, &mailbox->lock, &wait->ts);
+    } else {
+        pthread_cond_wait(&mailbox->cond, &mailbox->lock);
+        return 0;
+    }
+#endif
+}
+
+static void janet_mailbox_wakeup(JanetMailbox *mailbox) {
+#ifdef JANET_WINDOWS
+    WakeConditionVariable(&mailbox->cond);
+#else
+    pthread_cond_signal(&mailbox->cond);
+#endif
+}
+
+static int mailbox_at_capacity(JanetMailbox *mailbox) {
+    return mailbox->messageCount >= mailbox->messageCapacity;
+}
+
+/* Returns 1 if could not send (encode error or timeout), 2 for mailbox closed, and
+ * 0 otherwise. Will not panic.  */
+int janet_thread_send(JanetThread *thread, Janet msg, double timeout) {
+
+    /* Ensure mailbox is not closed. */
+    JanetMailbox *mailbox = thread->mailbox;
+    if (NULL == mailbox) return 2;
+    janet_mailbox_lock(mailbox);
+    if (mailbox->closed) {
+        janet_mailbox_ref_with_lock(mailbox, -1);
+        thread->mailbox = NULL;
+        return 2;
+    }
+
+    /* Back pressure */
+    if (mailbox_at_capacity(mailbox)) {
+        JanetWaiter wait;
+        janet_waiter_init(&wait, timeout);
+
+        if (wait.nowait) {
+            janet_mailbox_unlock(mailbox);
+            return 1;
+        }
+
+        /* Retry loop, as there can be multiple writers */
+        while (mailbox_at_capacity(mailbox)) {
+            if (janet_waiter_wait(&wait, mailbox)) {
+                janet_mailbox_unlock(mailbox);
+                janet_mailbox_wakeup(mailbox);
+                return 1;
+            }
+        }
+    }
+
+    /* Hack to capture all panics from marshalling. This works because
+     * we know janet_marshal won't mess with other essential global state. */
+    jmp_buf buf;
+    jmp_buf *old_buf = janet_vm_jmp_buf;
+    janet_vm_jmp_buf = &buf;
+    int32_t oldmcount = mailbox->messageCount;
+
+    int ret = 0;
+    if (setjmp(buf)) {
+        ret = 1;
+        mailbox->messageCount = oldmcount;
+    } else {
+        JanetBuffer *msgbuf = mailbox->messages + mailbox->messageNext;
+        msgbuf->count = 0;
+
+        /* Start panic zone */
+        janet_marshal(msgbuf, msg, thread->encode, 0);
+        /* End panic zone */
+
+        mailbox->messageNext = (mailbox->messageNext + 1) % mailbox->messageCapacity;
+        mailbox->messageCount++;
+    }
+
+    /* Cleanup */
+    janet_vm_jmp_buf = old_buf;
+    janet_mailbox_unlock(mailbox);
+
+    /* Potentially wake up a blocked thread */
+    janet_mailbox_wakeup(mailbox);
+
+    return ret;
+}
+
+/* Returns 0 on successful message. Returns 1 if timedout */
+int janet_thread_receive(Janet *msg_out, double timeout) {
+    JanetMailbox *mailbox = janet_vm_mailbox;
+    janet_mailbox_lock(mailbox);
+
+    /* For timeouts */
+    JanetWaiter wait;
+    janet_waiter_init(&wait, timeout);
+
+    for (;;) {
+
+        /* Check for messages waiting for us */
+        if (mailbox->messageCount > 0) {
+
+            /* Hack to capture all panics from marshalling. This works because
+             * we know janet_marshal won't mess with other essential global state. */
+            jmp_buf buf;
+            jmp_buf *old_buf = janet_vm_jmp_buf;
+            janet_vm_jmp_buf = &buf;
+
+            /* Handle errors */
+            if (setjmp(buf)) {
+                /* Cleanup jmp_buf, keep lock */
+                janet_vm_jmp_buf = old_buf;
+            } else {
+                JanetBuffer *msgbuf = mailbox->messages + mailbox->messageFirst;
+                mailbox->messageCount--;
+                mailbox->messageFirst = (mailbox->messageFirst + 1) % mailbox->messageCapacity;
+
+                /* Read from beginning of channel */
+                const uint8_t *nextItem = NULL;
+                Janet item = janet_unmarshal(
+                                 msgbuf->data, msgbuf->count,
+                                 0, mailbox->decode, &nextItem);
+                *msg_out = item;
+
+                /* Cleanup */
+                janet_vm_jmp_buf = old_buf;
+                janet_mailbox_unlock(mailbox);
+
+                /* Potentially wake up pending threads */
+                janet_mailbox_wakeup(mailbox);
+
+                return 0;
+            }
+        }
+
+        if (wait.nowait || mailbox->refCount <= 1) {
+            janet_mailbox_unlock(mailbox);
+            return 1;
+        }
+
+        /* Wait for next message */
+        if (janet_waiter_wait(&wait, mailbox)) {
+            janet_mailbox_unlock(mailbox);
+            return 1;
+        }
+    }
+
+}
+
+static int janet_thread_getter(void *p, Janet key, Janet *out);
+
+static JanetAbstractType Thread_AT = {
+    "core/thread",
+    thread_gc,
+    thread_mark,
+    janet_thread_getter,
+    NULL,
+    NULL,
+    NULL,
+    NULL
+};
+
+static JanetThread *janet_make_thread(JanetMailbox *mailbox, JanetTable *encode) {
+    JanetThread *thread = janet_abstract(&Thread_AT, sizeof(JanetThread));
+    thread->mailbox = mailbox;
+    thread->encode = encode;
+    return thread;
+}
+
+JanetThread *janet_getthread(const Janet *argv, int32_t n) {
+    return (JanetThread *) janet_getabstract(argv, n, &Thread_AT);
+}
+
+static JanetTable *janet_get_core_table(const char *name) {
+    JanetTable *env = janet_core_env(NULL);
+    Janet out = janet_wrap_nil();
+    JanetBindingType bt = janet_resolve(env, janet_csymbol(name), &out);
+    if (bt == JANET_BINDING_NONE) return NULL;
+    if (!janet_checktype(out, JANET_TABLE)) return NULL;
+    return janet_unwrap_table(out);
+}
+
+/* Runs in new thread */
+static int thread_worker(JanetMailbox *mailbox) {
+    JanetFiber *fiber = NULL;
+    Janet out;
+
+    /* Use the mailbox we were given */
+    janet_vm_mailbox = mailbox;
+
+    /* Init VM */
+    janet_init();
+
+    /* Get dictionaries for default encode/decode */
+    JanetTable *encode = janet_get_core_table("make-image-dict");
+    mailbox->decode = janet_get_core_table("load-image-dict");
+
+    /* Create parent thread */
+    JanetThread *parent = janet_make_thread(mailbox->parent, encode);
+    janet_mailbox_ref(mailbox->parent, -1);
+    mailbox->parent = NULL; /* only used to create the thread */
+    Janet parentv = janet_wrap_abstract(parent);
+
+    /* Unmarshal the function */
+    Janet funcv;
+    int status = janet_thread_receive(&funcv, -1.0);
+
+    if (status) goto error;
+    if (!janet_checktype(funcv, JANET_FUNCTION)) goto error;
+    JanetFunction *func = janet_unwrap_function(funcv);
+
+    /* Arity check */
+    if (func->def->min_arity > 1 || func->def->max_arity < 1) {
+        goto error;
+    }
+
+    /* Call function */
+    Janet argv[1] = { parentv };
+    fiber = janet_fiber(func, 64, 1, argv);
+    JanetSignal sig = janet_continue(fiber, janet_wrap_nil(), &out);
+    if (sig != JANET_SIGNAL_OK) {
+        janet_eprintf("in thread %v: ", janet_wrap_abstract(janet_make_thread(mailbox, encode)));
+        janet_stacktrace(fiber, out);
+    }
+
+    /* Normal exit */
+    janet_deinit();
+    return 0;
+
+    /* Fail to set something up */
+error:
+    janet_eprintf("\nthread failed to start\n");
+    janet_deinit();
+    return 1;
+}
+
+#ifdef JANET_WINDOWS
+
+static DWORD janet_create_thread_wrapper(void *param) {
+    thread_worker((JanetMailbox *)param);
+    return 0;
+}
+
+static int janet_thread_start_child(JanetThread *thread) {
+    HANDLE handle = CreateThread(NULL, 0, janet_create_thread_wrapper, thread->mailbox, 0, NULL);
+    int ret = NULL == handle;
+    /* Does not kill thread, simply detatches */
+    if (!ret) CloseHandle(handle);
+    return ret;
+}
+
+#else
+
+static void *janet_pthread_wrapper(void *param) {
+    thread_worker((JanetMailbox *)param);
+    return NULL;
+}
+
+static int janet_thread_start_child(JanetThread *thread) {
+    pthread_t handle;
+    int error = pthread_create(&handle, NULL, janet_pthread_wrapper, thread->mailbox);
+    if (error) {
+        return 1;
+    } else {
+        pthread_detach(handle);
+        return 0;
+    }
+}
+
+#endif
+
+/*
+ * Setup/Teardown
+ */
+
+void janet_threads_init(void) {
+    if (NULL == janet_vm_mailbox) {
+        janet_vm_mailbox = janet_mailbox_create(NULL, 1, 10);
+    }
+}
+
+void janet_threads_deinit(void) {
+    janet_mailbox_lock(janet_vm_mailbox);
+    janet_vm_mailbox->closed = 1;
+    janet_mailbox_ref_with_lock(janet_vm_mailbox, -1);
+    janet_vm_mailbox = NULL;
+    janet_vm_thread_current = NULL;
+}
+
+/*
+ * Cfuns
+ */
+
+static Janet cfun_thread_current(int32_t argc, Janet *argv) {
+    (void) argv;
+    janet_fixarity(argc, 0);
+    if (NULL == janet_vm_thread_current) {
+        janet_vm_thread_current = janet_make_thread(janet_vm_mailbox, janet_get_core_table("make-image-dict"));
+        janet_mailbox_ref(janet_vm_mailbox, 1);
+        janet_gcroot(janet_wrap_abstract(janet_vm_thread_current));
+    }
+    return janet_wrap_abstract(janet_vm_thread_current);
+}
+
+static Janet cfun_thread_new(int32_t argc, Janet *argv) {
+    janet_arity(argc, 0, 1);
+    int32_t cap = janet_optinteger(argv, argc, 0, 10);
+    if (cap < 1 || cap > UINT16_MAX) {
+        janet_panicf("bad slot #1, expected integer in range [1, 65535], got %d", cap);
+    }
+    JanetTable *encode = janet_get_core_table("make-image-dict");
+    JanetMailbox *mailbox = janet_mailbox_create(janet_vm_mailbox, 2, (uint16_t) cap);
+
+    /* one for created thread, one for ->parent reference in new mailbox */
+    janet_mailbox_ref(janet_vm_mailbox, 2);
+
+    JanetThread *thread = janet_make_thread(mailbox, encode);
+    if (janet_thread_start_child(thread)) {
+        janet_mailbox_ref(mailbox, -1); /* mailbox reference */
+        janet_mailbox_ref(janet_vm_mailbox, -1); /* ->parent reference */
+        janet_panic("could not start thread");
+    }
+    return janet_wrap_abstract(thread);
+}
+
+static Janet cfun_thread_send(int32_t argc, Janet *argv) {
+    janet_arity(argc, 2, 3);
+    JanetThread *thread = janet_getthread(argv, 0);
+    int status = janet_thread_send(thread, argv[1], janet_optnumber(argv, argc, 2, 1.0));
+    switch (status) {
+        default:
+            break;
+        case 1:
+            janet_panicf("failed to send message %v", argv[1]);
+        case 2:
+            janet_panic("thread mailbox is closed");
+    }
+    return argv[0];
+}
+
+static Janet cfun_thread_receive(int32_t argc, Janet *argv) {
+    janet_arity(argc, 0, 1);
+    double wait = janet_optnumber(argv, argc, 0, 1.0);
+    Janet out;
+    int status = janet_thread_receive(&out, wait);
+    switch (status) {
+        default:
+            break;
+        case 1:
+            janet_panicf("timeout after %f seconds", wait);
+    }
+    return out;
+}
+
+static Janet cfun_thread_close(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    JanetThread *thread = janet_getthread(argv, 0);
+    janet_close_thread(thread);
+    return janet_wrap_nil();
+}
+
+static const JanetMethod janet_thread_methods[] = {
+    {"send", cfun_thread_send},
+    {"close", cfun_thread_close},
+    {NULL, NULL}
+};
+
+static int janet_thread_getter(void *p, Janet key, Janet *out) {
+    (void) p;
+    if (!janet_checktype(key, JANET_KEYWORD)) return 0;
+    return janet_getmethod(janet_unwrap_keyword(key), janet_thread_methods, out);
+}
+
+static const JanetReg threadlib_cfuns[] = {
+    {
+        "thread/current", cfun_thread_current,
+        JDOC("(thread/current)\n\n"
+             "Get the current running thread.")
+    },
+    {
+        "thread/new", cfun_thread_new,
+        JDOC("(thread/new &opt capacity)\n\n"
+             "Start a new thread. The thread will wait for a message containing the function used to start the thread, which should be passed to the thread "
+             "via thread/send. If capacity is provided, that is how many messages can be stored in the thread's mailbox before blocking senders. "
+             "The capacity must be between 1 and 65535 inclusive, and defaults to 10. "
+             "Returns a handle to the new thread.")
+    },
+    {
+        "thread/send", cfun_thread_send,
+        JDOC("(thread/send thread msg)\n\n"
+             "Send a message to the thread. This will never block and returns thread immediately. "
+             "Will throw an error if there is a problem sending the message.")
+    },
+    {
+        "thread/receive", cfun_thread_receive,
+        JDOC("(thread/receive &opt timeout)\n\n"
+             "Get a message sent to this thread. If timeout is provided, an error will be thrown after the timeout has elapsed but "
+             "no messages are received.")
+    },
+    {
+        "thread/close", cfun_thread_close,
+        JDOC("(thread/close thread)\n\n"
+             "Close a thread, unblocking it and ending communication with it. Note that closing "
+             "a thread is idempotent and does not cancel the thread's operation. Returns nil.")
+    },
+    {NULL, NULL, NULL}
+};
+
+/* Module entry point */
+void janet_lib_thread(JanetTable *env) {
+    janet_core_cfuns(env, NULL, threadlib_cfuns);
+    janet_register_abstract_type(&Thread_AT);
+}
+
+#endif

src/core/tuple.c

@@ -21,7 +21,7 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "symcache.h"
 #include "gc.h"
 #include "util.h"
@@ -31,12 +31,12 @@
  * which should be filled with Janets. The memory will not be collected until
  * janet_tuple_end is called. */
 Janet *janet_tuple_begin(int32_t length) {
-    char *data = janet_gcalloc(JANET_MEMORY_TUPLE, 4 * sizeof(int32_t) + length * sizeof(Janet));
-    Janet *tuple = (Janet *)(data + (4 * sizeof(int32_t)));
-    janet_tuple_length(tuple) = length;
-    janet_tuple_sm_start(tuple) = -1;
-    janet_tuple_sm_end(tuple) = -1;
-    return tuple;
+    size_t size = sizeof(JanetTupleHead) + (length * sizeof(Janet));
+    JanetTupleHead *head = janet_gcalloc(JANET_MEMORY_TUPLE, size);
+    head->sm_line = -1;
+    head->sm_column = -1;
+    head->length = length;
+    return (Janet *)(head->data);
 }
 
 /* Finish building a tuple */
@@ -93,58 +93,87 @@ int janet_tuple_compare(const Janet *lhs, const Janet *rhs) {
 
 /* C Functions */
 
+static Janet cfun_tuple_brackets(int32_t argc, Janet *argv) {
+    const Janet *tup = janet_tuple_n(argv, argc);
+    janet_tuple_flag(tup) |= JANET_TUPLE_FLAG_BRACKETCTOR;
+    return janet_wrap_tuple(tup);
+}
+
 static Janet cfun_tuple_slice(int32_t argc, Janet *argv) {
-    JanetRange range = janet_getslice(argc, argv);
     JanetView view = janet_getindexed(argv, 0);
+    JanetRange range = janet_getslice(argc, argv);
     return janet_wrap_tuple(janet_tuple_n(view.items + range.start, range.end - range.start));
 }
 
-static Janet cfun_tuple_prepend(int32_t argc, Janet *argv) {
-    janet_arity(argc, 1, -1);
-    JanetView view = janet_getindexed(argv, 0);
-    Janet *n = janet_tuple_begin(view.len - 1 + argc);
-    memcpy(n - 1 + argc, view.items, sizeof(Janet) * view.len);
-    for (int32_t i = 1; i < argc; i++) {
-        n[argc - i - 1] = argv[i];
+static Janet cfun_tuple_type(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    const Janet *tup = janet_gettuple(argv, 0);
+    if (janet_tuple_flag(tup) & JANET_TUPLE_FLAG_BRACKETCTOR) {
+        return janet_ckeywordv("brackets");
+    } else {
+        return janet_ckeywordv("parens");
     }
-    return janet_wrap_tuple(janet_tuple_end(n));
 }
 
-static Janet cfun_tuple_append(int32_t argc, Janet *argv) {
-    janet_arity(argc, 1, -1);
-    JanetView view = janet_getindexed(argv, 0);
-    Janet *n = janet_tuple_begin(view.len - 1 + argc);
-    memcpy(n, view.items, sizeof(Janet) * view.len);
-    memcpy(n + view.len, argv + 1, sizeof(Janet) * (argc - 1));
-    return janet_wrap_tuple(janet_tuple_end(n));
+static Janet cfun_tuple_sourcemap(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    const Janet *tup = janet_gettuple(argv, 0);
+    Janet contents[2];
+    contents[0] = janet_wrap_integer(janet_tuple_head(tup)->sm_line);
+    contents[1] = janet_wrap_integer(janet_tuple_head(tup)->sm_column);
+    return janet_wrap_tuple(janet_tuple_n(contents, 2));
+}
+
+static Janet cfun_tuple_setmap(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 3);
+    const Janet *tup = janet_gettuple(argv, 0);
+    janet_tuple_head(tup)->sm_line = janet_getinteger(argv, 1);
+    janet_tuple_head(tup)->sm_column = janet_getinteger(argv, 2);
+    return argv[0];
 }
 
 static const JanetReg tuple_cfuns[] = {
+    {
+        "tuple/brackets", cfun_tuple_brackets,
+        JDOC("(tuple/brackets & xs)\n\n"
+             "Creates a new bracketed tuple containing the elements xs.")
+    },
     {
         "tuple/slice", cfun_tuple_slice,
         JDOC("(tuple/slice arrtup [,start=0 [,end=(length arrtup)]])\n\n"
-                "Take a sub sequence of an array or tuple from index start "
-                "inclusive to index end exclusive. If start or end are not provided, "
-                "they default to 0 and the length of arrtup respectively."
-                "Returns the new tuple.")
+             "Take a sub sequence of an array or tuple from index start "
+             "inclusive to index end exclusive. If start or end are not provided, "
+             "they default to 0 and the length of arrtup respectively. "
+             "'start' and 'end' can also be negative to indicate indexing "
+             "from the end of the input. Note that index -1 is synonymous with "
+             "index '(length arrtup)' to allow a full negative slice range. "
+             "Returns the new tuple.")
     },
     {
-        "tuple/append", cfun_tuple_append,
-        JDOC("(tuple/append tup & items)\n\n"
-                "Returns a new tuple that is the result of appending "
-                "each element in items to tup.")
+        "tuple/type", cfun_tuple_type,
+        JDOC("(tuple/type tup)\n\n"
+             "Checks how the tuple was constructed. Will return the keyword "
+             ":brackets if the tuple was parsed with brackets, and :parens "
+             "otherwise. The two types of tuples will behave the same most of "
+             "the time, but will print differently and be treated differently by "
+             "the compiler.")
     },
     {
-        "tuple/prepend", cfun_tuple_prepend,
-        JDOC("(tuple/prepend tup & items)\n\n"
-                "Prepends each element in items to tuple and "
-                "returns a new tuple. Items are prepended such that the "
-                "last element in items is the first element in the new tuple.")
+        "tuple/sourcemap", cfun_tuple_sourcemap,
+        JDOC("(tuple/sourcemap tup)\n\n"
+             "Returns the sourcemap metadata attached to a tuple, "
+             " which is another tuple (line, column).")
+    },
+    {
+        "tuple/setmap", cfun_tuple_setmap,
+        JDOC("(tuple/setmap tup line column)\n\n"
+             "Set the sourcemap metadata on a tuple. line and column indicate "
+             "should be integers.")
     },
     {NULL, NULL, NULL}
 };
 
 /* Load the tuple module */
 void janet_lib_tuple(JanetTable *env) {
-    janet_cfuns(env, NULL, tuple_cfuns);
+    janet_core_cfuns(env, NULL, tuple_cfuns);
 }

src/core/typedarray.c

@@ -0,0 +1,580 @@
+/*
+* Copyright (c) 2019 Calvin Rose & contributors
+*
+* Permission is hereby granted, free of charge, to any person obtaining a copy
+* of this software and associated documentation files (the "Software"), to
+* deal in the Software without restriction, including without limitation the
+* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+* sell copies of the Software, and to permit persons to whom the Software is
+* furnished to do so, subject to the following conditions:
+*
+* The above copyright notice and this permission notice shall be included in
+* all copies or substantial portions of the Software.
+*
+* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+* IN THE SOFTWARE.
+*/
+
+#ifndef JANET_AMALG
+#include <janet.h>
+#include "util.h"
+#endif
+
+#ifdef JANET_TYPED_ARRAY
+
+static char *ta_type_names[] = {
+    "uint8",
+    "int8",
+    "uint16",
+    "int16",
+    "uint32",
+    "int32",
+    "uint64",
+    "int64",
+    "float32",
+    "float64",
+    "?"
+};
+
+static size_t ta_type_sizes[] = {
+    sizeof(uint8_t),
+    sizeof(int8_t),
+    sizeof(uint16_t),
+    sizeof(int16_t),
+    sizeof(uint32_t),
+    sizeof(int32_t),
+    sizeof(uint64_t),
+    sizeof(int64_t),
+    sizeof(float),
+    sizeof(double),
+    0
+};
+
+#define TA_COUNT_TYPES (JANET_TARRAY_TYPE_F64 + 1)
+#define TA_ATOM_MAXSIZE 8
+#define TA_FLAG_BIG_ENDIAN 1
+
+static JanetTArrayType get_ta_type_by_name(const uint8_t *name) {
+    for (int i = 0; i < TA_COUNT_TYPES; i++) {
+        if (!janet_cstrcmp(name, ta_type_names[i]))
+            return i;
+    }
+    janet_panicf("invalid typed array type %S", name);
+    return 0;
+}
+
+static JanetTArrayBuffer *ta_buffer_init(JanetTArrayBuffer *buf, size_t size) {
+    buf->data = NULL;
+    if (size > 0) {
+        buf->data = (uint8_t *)calloc(size, sizeof(uint8_t));
+        if (buf->data == NULL) {
+            JANET_OUT_OF_MEMORY;
+        }
+    }
+    buf->size = size;
+#ifdef JANET_BIG_ENDIAN
+    buf->flags = TA_FLAG_BIG_ENDIAN;
+#else
+    buf->flags = 0;
+#endif
+    return buf;
+}
+
+static int ta_buffer_gc(void *p, size_t s) {
+    (void) s;
+    JanetTArrayBuffer *buf = (JanetTArrayBuffer *)p;
+    free(buf->data);
+    return 0;
+}
+
+static void ta_buffer_marshal(void *p, JanetMarshalContext *ctx) {
+    JanetTArrayBuffer *buf = (JanetTArrayBuffer *)p;
+    janet_marshal_abstract(ctx, p);
+    janet_marshal_size(ctx, buf->size);
+    janet_marshal_int(ctx, buf->flags);
+    janet_marshal_bytes(ctx, buf->data, buf->size);
+}
+
+static void *ta_buffer_unmarshal(JanetMarshalContext *ctx) {
+    JanetTArrayBuffer *buf = janet_unmarshal_abstract(ctx, sizeof(JanetTArrayBuffer));
+    size_t size = janet_unmarshal_size(ctx);
+    int32_t flags = janet_unmarshal_int(ctx);
+    ta_buffer_init(buf, size);
+    buf->flags = flags;
+    janet_unmarshal_bytes(ctx, buf->data, size);
+    return buf;
+}
+
+static const JanetAbstractType ta_buffer_type = {
+    "ta/buffer",
+    ta_buffer_gc,
+    NULL,
+    NULL,
+    NULL,
+    ta_buffer_marshal,
+    ta_buffer_unmarshal,
+    NULL
+};
+
+static int ta_mark(void *p, size_t s) {
+    (void) s;
+    JanetTArrayView *view = (JanetTArrayView *)p;
+    janet_mark(janet_wrap_abstract(view->buffer));
+    return 0;
+}
+
+static void ta_view_marshal(void *p, JanetMarshalContext *ctx) {
+    JanetTArrayView *view = (JanetTArrayView *)p;
+    size_t offset = (view->buffer->data - view->as.u8);
+    janet_marshal_abstract(ctx, p);
+    janet_marshal_size(ctx, view->size);
+    janet_marshal_size(ctx, view->stride);
+    janet_marshal_int(ctx, view->type);
+    janet_marshal_size(ctx, offset);
+    janet_marshal_janet(ctx, janet_wrap_abstract(view->buffer));
+}
+
+static void *ta_view_unmarshal(JanetMarshalContext *ctx) {
+    size_t offset;
+    int32_t atype;
+    Janet buffer;
+    JanetTArrayView *view = janet_unmarshal_abstract(ctx, sizeof(JanetTArrayView));
+    view->size = janet_unmarshal_size(ctx);
+    view->stride = janet_unmarshal_size(ctx);
+    atype = janet_unmarshal_int(ctx);
+    if (atype < 0 || atype >= TA_COUNT_TYPES)
+        janet_panic("bad typed array type");
+    view->type = atype;
+    offset = janet_unmarshal_size(ctx);
+    buffer = janet_unmarshal_janet(ctx);
+    if (!janet_checktype(buffer, JANET_ABSTRACT) ||
+            (janet_abstract_type(janet_unwrap_abstract(buffer)) != &ta_buffer_type)) {
+        janet_panicf("expected typed array buffer");
+    }
+    view->buffer = (JanetTArrayBuffer *)janet_unwrap_abstract(buffer);
+    size_t buf_need_size = offset + (ta_type_sizes[view->type]) * ((view->size - 1) * view->stride + 1);
+    if (view->buffer->size < buf_need_size)
+        janet_panic("bad typed array offset in marshalled data");
+    view->as.u8 = view->buffer->data + offset;
+    return view;
+}
+
+static JanetMethod tarray_view_methods[6];
+
+static int ta_getter(void *p, Janet key, Janet *out) {
+    size_t index, i;
+    JanetTArrayView *array = p;
+    if (janet_checktype(key, JANET_KEYWORD)) {
+        return janet_getmethod(janet_unwrap_keyword(key), tarray_view_methods, out);
+    }
+    if (!janet_checksize(key)) janet_panic("expected size as key");
+    index = (size_t) janet_unwrap_number(key);
+    i = index * array->stride;
+    if (index >= array->size) {
+        return 0;
+    } else {
+        switch (array->type) {
+            case JANET_TARRAY_TYPE_U8:
+                *out = janet_wrap_number(array->as.u8[i]);
+                break;
+            case JANET_TARRAY_TYPE_S8:
+                *out = janet_wrap_number(array->as.s8[i]);
+                break;
+            case JANET_TARRAY_TYPE_U16:
+                *out = janet_wrap_number(array->as.u16[i]);
+                break;
+            case JANET_TARRAY_TYPE_S16:
+                *out = janet_wrap_number(array->as.s16[i]);
+                break;
+            case JANET_TARRAY_TYPE_U32:
+                *out = janet_wrap_number(array->as.u32[i]);
+                break;
+            case JANET_TARRAY_TYPE_S32:
+                *out = janet_wrap_number(array->as.s32[i]);
+                break;
+#ifdef JANET_INT_TYPES
+            case JANET_TARRAY_TYPE_U64:
+                *out = janet_wrap_u64(array->as.u64[i]);
+                break;
+            case JANET_TARRAY_TYPE_S64:
+                *out = janet_wrap_s64(array->as.s64[i]);
+                break;
+#endif
+            case JANET_TARRAY_TYPE_F32:
+                *out = janet_wrap_number_safe(array->as.f32[i]);
+                break;
+            case JANET_TARRAY_TYPE_F64:
+                *out = janet_wrap_number_safe(array->as.f64[i]);
+                break;
+            default:
+                janet_panicf("cannot get from typed array of type %s",
+                             ta_type_names[array->type]);
+                break;
+        }
+    }
+    return 1;
+}
+
+static void ta_setter(void *p, Janet key, Janet value) {
+    size_t index, i;
+    if (!janet_checksize(key)) janet_panic("expected size as key");
+    index = (size_t) janet_unwrap_number(key);
+    JanetTArrayView *array = p;
+    i = index * array->stride;
+    if (index >= array->size) {
+        janet_panic("index out of bounds");
+    }
+    if (!janet_checktype(value, JANET_NUMBER) &&
+            array->type != JANET_TARRAY_TYPE_U64 &&
+            array->type != JANET_TARRAY_TYPE_S64) {
+        janet_panic("expected number value");
+    }
+    switch (array->type) {
+        case JANET_TARRAY_TYPE_U8:
+            array->as.u8[i] = (uint8_t) janet_unwrap_number(value);
+            break;
+        case JANET_TARRAY_TYPE_S8:
+            array->as.s8[i] = (int8_t) janet_unwrap_number(value);
+            break;
+        case JANET_TARRAY_TYPE_U16:
+            array->as.u16[i] = (uint16_t) janet_unwrap_number(value);
+            break;
+        case JANET_TARRAY_TYPE_S16:
+            array->as.s16[i] = (int16_t) janet_unwrap_number(value);
+            break;
+        case JANET_TARRAY_TYPE_U32:
+            array->as.u32[i] = (uint32_t) janet_unwrap_number(value);
+            break;
+        case JANET_TARRAY_TYPE_S32:
+            array->as.s32[i] = (int32_t) janet_unwrap_number(value);
+            break;
+#ifdef JANET_INT_TYPES
+        case JANET_TARRAY_TYPE_U64:
+            array->as.u64[i] = janet_unwrap_u64(value);
+            break;
+        case JANET_TARRAY_TYPE_S64:
+            array->as.s64[i] = janet_unwrap_s64(value);
+            break;
+#endif
+        case JANET_TARRAY_TYPE_F32:
+            array->as.f32[i] = (float) janet_unwrap_number(value);
+            break;
+        case JANET_TARRAY_TYPE_F64:
+            array->as.f64[i] = janet_unwrap_number(value);
+            break;
+        default:
+            janet_panicf("cannot set typed array of type %s",
+                         ta_type_names[array->type]);
+            break;
+    }
+}
+
+static const JanetAbstractType ta_view_type = {
+    "ta/view",
+    NULL,
+    ta_mark,
+    ta_getter,
+    ta_setter,
+    ta_view_marshal,
+    ta_view_unmarshal,
+    NULL
+};
+
+JanetTArrayBuffer *janet_tarray_buffer(size_t size) {
+    JanetTArrayBuffer *buf = janet_abstract(&ta_buffer_type, sizeof(JanetTArrayBuffer));
+    ta_buffer_init(buf, size);
+    return buf;
+}
+
+JanetTArrayView *janet_tarray_view(
+    JanetTArrayType type,
+    size_t size,
+    size_t stride,
+    size_t offset,
+    JanetTArrayBuffer *buffer) {
+
+    JanetTArrayView *view = janet_abstract(&ta_view_type, sizeof(JanetTArrayView));
+
+    if ((stride < 1) || (size < 1)) janet_panic("stride and size should be > 0");
+    size_t buf_size = offset + ta_type_sizes[type] * ((size - 1) * stride + 1);
+
+    if (NULL == buffer) {
+        buffer = janet_abstract(&ta_buffer_type, sizeof(JanetTArrayBuffer));
+        ta_buffer_init(buffer, buf_size);
+    }
+
+    if (buffer->size < buf_size) {
+        janet_panicf("bad buffer size, %i bytes allocated < %i required",
+                     buffer->size,
+                     buf_size);
+    }
+
+    view->buffer = buffer;
+    view->stride = stride;
+    view->size = size;
+    view->as.u8 = buffer->data + offset;
+    view->type = type;
+
+    return view;
+}
+
+JanetTArrayBuffer *janet_gettarray_buffer(const Janet *argv, int32_t n) {
+    return janet_getabstract(argv, n, &ta_buffer_type);
+}
+
+JanetTArrayView *janet_gettarray_any(const Janet *argv, int32_t n) {
+    return janet_getabstract(argv, n, &ta_view_type);
+}
+
+JanetTArrayView *janet_gettarray_view(const Janet *argv, int32_t n, JanetTArrayType type) {
+    JanetTArrayView *view = janet_getabstract(argv, n, &ta_view_type);
+    if (view->type != type) {
+        janet_panicf("bad slot #%d, expected typed array of type %s, got %v",
+                     n, ta_type_names[type], argv[n]);
+    }
+    return view;
+}
+
+static Janet cfun_typed_array_new(int32_t argc, Janet *argv) {
+    janet_arity(argc, 2, 5);
+    size_t offset = 0;
+    size_t stride = 1;
+    JanetTArrayBuffer *buffer = NULL;
+    const uint8_t *keyw = janet_getkeyword(argv, 0);
+    JanetTArrayType type = get_ta_type_by_name(keyw);
+    size_t size = janet_getsize(argv, 1);
+    if (argc > 2)
+        stride = janet_getsize(argv, 2);
+    if (argc > 3)
+        offset = janet_getsize(argv, 3);
+    if (argc > 4) {
+        if (!janet_checktype(argv[4], JANET_ABSTRACT)) {
+            janet_panicf("bad slot #%d, expected ta/view|ta/buffer, got %v",
+                         4, argv[4]);
+        }
+        void *p = janet_unwrap_abstract(argv[4]);
+        if (janet_abstract_type(p) == &ta_view_type) {
+            JanetTArrayView *view = (JanetTArrayView *)p;
+            offset = (view->buffer->data - view->as.u8) + offset * ta_type_sizes[view->type];
+            stride *= view->stride;
+            buffer = view->buffer;
+        } else {
+            buffer = p;
+        }
+    }
+    JanetTArrayView *view = janet_tarray_view(type, size, stride, offset, buffer);
+    return janet_wrap_abstract(view);
+}
+
+static JanetTArrayView *ta_is_view(Janet x) {
+    if (!janet_checktype(x, JANET_ABSTRACT)) return NULL;
+    void *abst = janet_unwrap_abstract(x);
+    if (janet_abstract_type(abst) != &ta_view_type) return NULL;
+    return (JanetTArrayView *)abst;
+}
+
+static Janet cfun_typed_array_buffer(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    JanetTArrayView *view;
+    if ((view = ta_is_view(argv[0]))) {
+        return janet_wrap_abstract(view->buffer);
+    }
+    size_t size = janet_getsize(argv, 0);
+    JanetTArrayBuffer *buf = janet_tarray_buffer(size);
+    return janet_wrap_abstract(buf);
+}
+
+static Janet cfun_typed_array_size(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    JanetTArrayView *view;
+    if ((view = ta_is_view(argv[0]))) {
+        return janet_wrap_number((double) view->size);
+    }
+    JanetTArrayBuffer *buf = (JanetTArrayBuffer *)janet_getabstract(argv, 0, &ta_buffer_type);
+    return janet_wrap_number((double) buf->size);
+}
+
+static Janet cfun_typed_array_properties(int32_t argc, Janet *argv) {
+    janet_fixarity(argc, 1);
+    JanetTArrayView *view;
+    if ((view = ta_is_view(argv[0]))) {
+        JanetTArrayView *view = janet_unwrap_abstract(argv[0]);
+        JanetKV *props = janet_struct_begin(6);
+        ptrdiff_t boffset = view->as.u8 - view->buffer->data;
+        janet_struct_put(props, janet_ckeywordv("size"),
+                         janet_wrap_number((double) view->size));
+        janet_struct_put(props, janet_ckeywordv("byte-offset"),
+                         janet_wrap_number((double) boffset));
+        janet_struct_put(props, janet_ckeywordv("stride"),
+                         janet_wrap_number((double) view->stride));
+        janet_struct_put(props, janet_ckeywordv("type"),
+                         janet_ckeywordv(ta_type_names[view->type]));
+        janet_struct_put(props, janet_ckeywordv("type-size"),
+                         janet_wrap_number((double) ta_type_sizes[view->type]));
+        janet_struct_put(props, janet_ckeywordv("buffer"),
+                         janet_wrap_abstract(view->buffer));
+        return janet_wrap_struct(janet_struct_end(props));
+    } else {
+        JanetTArrayBuffer *buffer = janet_gettarray_buffer(argv, 0);
+        JanetKV *props = janet_struct_begin(2);
+        janet_struct_put(props, janet_ckeywordv("size"),
+                         janet_wrap_number((double) buffer->size));
+        janet_struct_put(props, janet_ckeywordv("big-endian"),
+                         janet_wrap_boolean(buffer->flags & TA_FLAG_BIG_ENDIAN));
+        return janet_wrap_struct(janet_struct_end(props));
+    }
+}
+
+static Janet cfun_typed_array_slice(int32_t argc, Janet *argv) {
+    janet_arity(argc, 1, 3);
+    JanetTArrayView *src = janet_getabstract(argv, 0, &ta_view_type);
+    JanetRange range;
+    int32_t length = (int32_t)src->size;
+    if (argc == 1) {
+        range.start = 0;
+        range.end = length;
+    } else if (argc == 2) {
+        range.start = janet_gethalfrange(argv, 1, length, "start");
+        range.end = length;
+    } else {
+        range.start = janet_gethalfrange(argv, 1, length, "start");
+        range.end = janet_gethalfrange(argv, 2, length, "end");
+        if (range.end < range.start)
+            range.end = range.start;
+    }
+    JanetArray *array = janet_array(range.end - range.start);
+    if (array->data) {
+        for (int32_t i = range.start; i < range.end; i++) {
+            if (!ta_getter(src, janet_wrap_number(i), &array->data[i - range.start]))
+                array->data[i - range.start] = janet_wrap_nil();
+        }
+    }
+    array->count = range.end - range.start;
+    return janet_wrap_array(array);
+}
+
+static Janet cfun_typed_array_copy_bytes(int32_t argc, Janet *argv) {
+    janet_arity(argc, 4, 5);
+    JanetTArrayView *src = janet_getabstract(argv, 0, &ta_view_type);
+    size_t index_src = janet_getsize(argv, 1);
+    JanetTArrayView *dst = janet_getabstract(argv, 2, &ta_view_type);
+    size_t index_dst = janet_getsize(argv, 3);
+    size_t count = (argc == 5) ? janet_getsize(argv, 4) : 1;
+    size_t src_atom_size = ta_type_sizes[src->type];
+    size_t dst_atom_size = ta_type_sizes[dst->type];
+    size_t step_src = src->stride * src_atom_size;
+    size_t step_dst = dst->stride * dst_atom_size;
+    size_t pos_src = (src->as.u8 - src->buffer->data) + (index_src * step_src);
+    size_t pos_dst = (dst->as.u8 - dst->buffer->data) + (index_dst * step_dst);
+    uint8_t *ps = src->buffer->data + pos_src, * pd = dst->buffer->data + pos_dst;
+    if ((pos_dst + (count - 1)*step_dst + src_atom_size <= dst->buffer->size) &&
+            (pos_src + (count - 1)*step_src + src_atom_size <= src->buffer->size)) {
+        for (size_t i = 0; i < count; i++) {
+            memmove(pd, ps, src_atom_size);
+            pd += step_dst;
+            ps += step_src;
+        }
+    } else {
+        janet_panic("typed array copy out of bounds");
+    }
+    return janet_wrap_nil();
+}
+
+static Janet cfun_typed_array_swap_bytes(int32_t argc, Janet *argv) {
+    janet_arity(argc, 4, 5);
+    JanetTArrayView *src = janet_getabstract(argv, 0, &ta_view_type);
+    size_t index_src = janet_getsize(argv, 1);
+    JanetTArrayView *dst = janet_getabstract(argv, 2, &ta_view_type);
+    size_t index_dst = janet_getsize(argv, 3);
+    size_t count = (argc == 5) ? janet_getsize(argv, 4) : 1;
+    size_t src_atom_size = ta_type_sizes[src->type];
+    size_t dst_atom_size = ta_type_sizes[dst->type];
+    size_t step_src = src->stride * src_atom_size;
+    size_t step_dst = dst->stride * dst_atom_size;
+    size_t pos_src = (src->as.u8 - src->buffer->data) + (index_src * step_src);
+    size_t pos_dst = (dst->as.u8 - dst->buffer->data) + (index_dst * step_dst);
+    uint8_t *ps = src->buffer->data + pos_src, * pd = dst->buffer->data + pos_dst;
+    uint8_t temp[TA_ATOM_MAXSIZE];
+    if ((pos_dst + (count - 1)*step_dst + src_atom_size <= dst->buffer->size) &&
+            (pos_src + (count - 1)*step_src + src_atom_size <= src->buffer->size)) {
+        for (size_t i = 0; i < count; i++) {
+            memcpy(temp, ps, src_atom_size);
+            memcpy(ps, pd, src_atom_size);
+            memcpy(pd, temp, src_atom_size);
+            pd += step_dst;
+            ps += step_src;
+        }
+    } else {
+        janet_panic("typed array swap out of bounds");
+    }
+    return janet_wrap_nil();
+}
+
+static const JanetReg ta_cfuns[] = {
+    {
+        "tarray/new", cfun_typed_array_new,
+        JDOC("(tarray/new type size &opt stride offset tarray|buffer)\n\n"
+             "Create new typed array.")
+    },
+    {
+        "tarray/buffer", cfun_typed_array_buffer,
+        JDOC("(tarray/buffer array|size)\n\n"
+             "Return typed array buffer or create a new buffer.")
+    },
+    {
+        "tarray/length", cfun_typed_array_size,
+        JDOC("(tarray/length array|buffer)\n\n"
+             "Return typed array or buffer size.")
+    },
+    {
+        "tarray/properties", cfun_typed_array_properties,
+        JDOC("(tarray/properties array)\n\n"
+             "Return typed array properties as a struct.")
+    },
+    {
+        "tarray/copy-bytes", cfun_typed_array_copy_bytes,
+        JDOC("(tarray/copy-bytes src sindex dst dindex &opt count)\n\n"
+             "Copy count elements (default 1) of src array from index sindex "
+             "to dst array at position dindex "
+             "memory can overlap.")
+    },
+    {
+        "tarray/swap-bytes", cfun_typed_array_swap_bytes,
+        JDOC("(tarray/swap-bytes src sindex dst dindex &opt count)\n\n"
+             "Swap count elements (default 1) between src array from index sindex "
+             "and dst array at position dindex "
+             "memory can overlap.")
+    },
+    {
+        "tarray/slice", cfun_typed_array_slice,
+        JDOC("(tarray/slice tarr &opt start end)\n\n"
+             "Takes a slice of a typed array from start to end. The range is half "
+             "open, [start, end). Indexes can also be negative, indicating indexing "
+             "from the end of the end of the typed array. By default, start is 0 and end is "
+             "the size of the typed array. Returns a new janet array.")
+    },
+    {NULL, NULL, NULL}
+};
+
+static JanetMethod tarray_view_methods[] = {
+    {"length", cfun_typed_array_size},
+    {"properties", cfun_typed_array_properties},
+    {"copy-bytes", cfun_typed_array_copy_bytes},
+    {"swap-bytes", cfun_typed_array_swap_bytes},
+    {"slice", cfun_typed_array_slice},
+    {NULL, NULL}
+};
+
+/* Module entry point */
+void janet_lib_typed_array(JanetTable *env) {
+    janet_core_cfuns(env, NULL, ta_cfuns);
+    janet_register_abstract_type(&ta_buffer_type);
+    janet_register_abstract_type(&ta_view_type);
+}
+
+#endif

src/core/util.c

@@ -23,7 +23,7 @@
 #include <inttypes.h>
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #include "util.h"
 #include "state.h"
 #include "gc.h"
@@ -42,7 +42,6 @@ const char *const janet_type_names[16] = {
     "number",
     "nil",
     "boolean",
-    "boolean",
     "fiber",
     "string",
     "symbol",
@@ -54,7 +53,8 @@ const char *const janet_type_names[16] = {
     "buffer",
     "function",
     "cfunction",
-    "abstract"
+    "abstract",
+    "pointer"
 };
 
 const char *const janet_signal_names[14] = {
@@ -210,10 +210,10 @@ int janet_cstrcmp(const uint8_t *str, const char *other) {
  * have a string as its first element, and the struct must be sorted
  * lexicographically by that element. */
 const void *janet_strbinsearch(
-        const void *tab,
-        size_t tabcount,
-        size_t itemsize,
-        const uint8_t *key) {
+    const void *tab,
+    size_t tabcount,
+    size_t itemsize,
+    const uint8_t *key) {
     size_t low = 0;
     size_t hi = tabcount;
     const char *t = (const char *)tab;
@@ -270,12 +270,13 @@ void janet_cfuns(JanetTable *env, const char *regprefix, const JanetReg *cfuns)
             int32_t nmlen = 0;
             while (regprefix[reglen]) reglen++;
             while (cfuns->name[nmlen]) nmlen++;
-            uint8_t *longname_buffer =
-                janet_string_begin(reglen + 1 + nmlen);
+            int32_t symlen = reglen + 1 + nmlen;
+            uint8_t *longname_buffer = malloc(symlen);
             memcpy(longname_buffer, regprefix, reglen);
             longname_buffer[reglen] = '/';
             memcpy(longname_buffer + reglen + 1, cfuns->name, nmlen);
-            longname = janet_wrap_symbol(janet_string_end(longname_buffer));
+            longname = janet_wrap_symbol(janet_symbol(longname_buffer, symlen));
+            free(longname_buffer);
         }
         Janet fun = janet_wrap_cfunction(cfuns->cfun);
         janet_def(env, cfuns->name, fun, cfuns->documentation);
@@ -284,6 +285,68 @@ void janet_cfuns(JanetTable *env, const char *regprefix, const JanetReg *cfuns)
     }
 }
 
+/* Abstract type introspection */
+
+static const JanetAbstractType type_wrap = {
+    "core/type-info",
+    NULL,
+    NULL,
+    NULL,
+    NULL,
+    NULL,
+    NULL,
+    NULL
+};
+
+typedef struct {
+    const JanetAbstractType *at;
+} JanetAbstractTypeWrap;
+
+void janet_register_abstract_type(const JanetAbstractType *at) {
+    JanetAbstractTypeWrap *abstract = (JanetAbstractTypeWrap *)
+                                      janet_abstract(&type_wrap, sizeof(JanetAbstractTypeWrap));
+    abstract->at = at;
+    Janet sym = janet_csymbolv(at->name);
+    if (!(janet_checktype(janet_table_get(janet_vm_registry, sym), JANET_NIL))) {
+        janet_panicf("cannot register abstract type %s, "
+                     "a type with the same name exists", at->name);
+    }
+    janet_table_put(janet_vm_registry, sym, janet_wrap_abstract(abstract));
+}
+
+const JanetAbstractType *janet_get_abstract_type(Janet key) {
+    Janet twrap = janet_table_get(janet_vm_registry, key);
+    if (janet_checktype(twrap, JANET_NIL)) {
+        return NULL;
+    }
+    if (!janet_checktype(twrap, JANET_ABSTRACT) ||
+            (janet_abstract_type(janet_unwrap_abstract(twrap)) != &type_wrap)) {
+        janet_panic("expected abstract type");
+    }
+    JanetAbstractTypeWrap *w = (JanetAbstractTypeWrap *)janet_unwrap_abstract(twrap);
+    return w->at;
+}
+
+#ifndef JANET_BOOTSTRAP
+void janet_core_def(JanetTable *env, const char *name, Janet x, const void *p) {
+    (void) p;
+    Janet key = janet_csymbolv(name);
+    janet_table_put(env, key, x);
+    if (janet_checktype(x, JANET_CFUNCTION)) {
+        janet_table_put(janet_vm_registry, x, key);
+    }
+}
+
+void janet_core_cfuns(JanetTable *env, const char *regprefix, const JanetReg *cfuns) {
+    (void) regprefix;
+    while (cfuns->name) {
+        Janet fun = janet_wrap_cfunction(cfuns->cfun);
+        janet_core_def(env, cfuns->name, fun, cfuns->documentation);
+        cfuns++;
+    }
+}
+#endif
+
 /* Resolve a symbol in the environment */
 JanetBindingType janet_resolve(JanetTable *env, const uint8_t *sym, Janet *out) {
     Janet ref;
@@ -293,8 +356,8 @@ JanetBindingType janet_resolve(JanetTable *env, const uint8_t *sym, Janet *out)
         return JANET_BINDING_NONE;
     entry_table = janet_unwrap_table(entry);
     if (!janet_checktype(
-        janet_table_get(entry_table, janet_ckeywordv("macro")),
-        JANET_NIL)) {
+                janet_table_get(entry_table, janet_ckeywordv("macro")),
+                JANET_NIL)) {
         *out = janet_table_get(entry_table, janet_ckeywordv("value"));
         return JANET_BINDING_MACRO;
     }
@@ -307,6 +370,14 @@ JanetBindingType janet_resolve(JanetTable *env, const uint8_t *sym, Janet *out)
     return JANET_BINDING_DEF;
 }
 
+/* Resolve a symbol in the core environment. */
+Janet janet_resolve_core(const char *name) {
+    JanetTable *env = janet_core_env(NULL);
+    Janet out = janet_wrap_nil();
+    janet_resolve(env, janet_csymbol(name), &out);
+    return out;
+}
+
 /* Read both tuples and arrays as c pointers + int32_t length. Return 1 if the
  * view can be constructed, 0 if an invalid type. */
 int janet_indexed_view(Janet seq, const Janet **data, int32_t *len) {
@@ -316,7 +387,7 @@ int janet_indexed_view(Janet seq, const Janet **data, int32_t *len) {
         return 1;
     } else if (janet_checktype(seq, JANET_TUPLE)) {
         *data = janet_unwrap_tuple(seq);
-        *len = janet_tuple_length(janet_unwrap_struct(seq));
+        *len = janet_tuple_length(janet_unwrap_tuple(seq));
         return 1;
     }
     return 0;
@@ -370,42 +441,10 @@ int janet_checkint64(Janet x) {
     return janet_checkint64range(dval);
 }
 
-/* Useful for inspecting values while debugging */
-void janet_inspect(Janet x) {
-    printf("<type=%s, ", janet_type_names[janet_type(x)]);
-
-#ifdef JANET_BIG_ENDIAN
-    printf("be ");
-#else
-    printf("le ");
-#endif
-
-#ifdef JANET_NANBOX_64
-    printf("nanbox64 raw=0x%.16" PRIx64 ", ", x.u64);
-#endif
-
-#ifdef JANET_NANBOX_32
-    printf("nanbox32 type=0x%.8" PRIx32 ", ", x.tagged.type);
-    printf("payload=%" PRId32 ", ", x.tagged.payload.integer);
-#endif
-
-    switch (janet_type(x)) {
-        case JANET_NIL:
-            printf("value=nil");
-            break;
-        case JANET_NUMBER:
-            printf("number=%.17g", janet_unwrap_number(x));
-            break;
-        case JANET_TRUE:
-            printf("value=true");
-            break;
-        case JANET_FALSE:
-            printf("value=false");
-            break;
-        default:
-            printf("pointer=%p", janet_unwrap_pointer(x));
-            break;
-    }
-
-    printf(">\n");
+int janet_checksize(Janet x) {
+    if (!janet_checktype(x, JANET_NUMBER))
+        return 0;
+    double dval = janet_unwrap_number(x);
+    return dval == (double)((size_t) dval) &&
+           dval <= SIZE_MAX;
 }

src/core/util.h

@@ -23,13 +23,39 @@
 #ifndef JANET_UTIL_H_defined
 #define JANET_UTIL_H_defined
 
+#include <stdio.h>
+#include <errno.h>
+
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
+#endif
+
+/* Handle runtime errors */
+#ifndef janet_exit
+#include <stdio.h>
+#define janet_exit(m) do { \
+    printf("C runtime error at line %d in file %s: %s\n",\
+        __LINE__,\
+        __FILE__,\
+        (m));\
+    exit(1);\
+} while (0)
+#endif
+
+#define janet_assert(c, m) do { \
+    if (!(c)) janet_exit((m)); \
+} while (0)
+
+/* What to do when out of memory */
+#ifndef JANET_OUT_OF_MEMORY
+#include <stdio.h>
+#define JANET_OUT_OF_MEMORY do { printf("janet out of memory\n"); exit(1); } while (0)
 #endif
 
 /* Omit docstrings in some builds */
-#ifdef JANET_NO_BOOTSTRAP
+#ifndef JANET_BOOTSTRAP
 #define JDOC(x) NULL
+#define JANET_NO_BOOTSTRAP
 #else
 #define JDOC(x) x
 #endif
@@ -47,10 +73,26 @@ Janet janet_dict_get(const JanetKV *buckets, int32_t cap, Janet key);
 void janet_memempty(JanetKV *mem, int32_t count);
 void *janet_memalloc_empty(int32_t count);
 const void *janet_strbinsearch(
-        const void *tab,
-        size_t tabcount,
-        size_t itemsize,
-        const uint8_t *key);
+    const void *tab,
+    size_t tabcount,
+    size_t itemsize,
+    const uint8_t *key);
+void janet_buffer_format(
+    JanetBuffer *b,
+    const char *strfrmt,
+    int32_t argstart,
+    int32_t argc,
+    Janet *argv);
+
+/* Inside the janet core, defining globals is different
+ * at bootstrap time and normal runtime */
+#ifdef JANET_BOOTSTRAP
+#define janet_core_def janet_def
+#define janet_core_cfuns janet_cfuns
+#else
+void janet_core_def(JanetTable *env, const char *name, Janet x, const void *p);
+void janet_core_cfuns(JanetTable *env, const char *regprefix, const JanetReg *cfuns);
+#endif
 
 /* Initialize builtin libraries */
 void janet_lib_io(JanetTable *env);
@@ -69,6 +111,17 @@ void janet_lib_asm(JanetTable *env);
 #endif
 void janet_lib_compile(JanetTable *env);
 void janet_lib_debug(JanetTable *env);
+#ifdef JANET_PEG
 void janet_lib_peg(JanetTable *env);
+#endif
+#ifdef JANET_TYPED_ARRAY
+void janet_lib_typed_array(JanetTable *env);
+#endif
+#ifdef JANET_INT_TYPES
+void janet_lib_inttypes(JanetTable *env);
+#endif
+#ifdef JANET_THREADS
+void janet_lib_thread(JanetTable *env);
+#endif
 
 #endif

src/core/value.c

@@ -21,7 +21,7 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #endif
 
 /*
@@ -35,27 +35,28 @@ int janet_equals(Janet x, Janet y) {
         result = 0;
     } else {
         switch (janet_type(x)) {
-        case JANET_NIL:
-        case JANET_TRUE:
-        case JANET_FALSE:
-            result = 1;
-            break;
-        case JANET_NUMBER:
-            result = (janet_unwrap_number(x) == janet_unwrap_number(y));
-            break;
-        case JANET_STRING:
-            result = janet_string_equal(janet_unwrap_string(x), janet_unwrap_string(y));
-            break;
-        case JANET_TUPLE:
-            result = janet_tuple_equal(janet_unwrap_tuple(x), janet_unwrap_tuple(y));
-            break;
-        case JANET_STRUCT:
-            result = janet_struct_equal(janet_unwrap_struct(x), janet_unwrap_struct(y));
-            break;
-        default:
-            /* compare pointers */
-            result = (janet_unwrap_pointer(x) == janet_unwrap_pointer(y));
-            break;
+            case JANET_NIL:
+                result = 1;
+                break;
+            case JANET_BOOLEAN:
+                result = (janet_unwrap_boolean(x) == janet_unwrap_boolean(y));
+                break;
+            case JANET_NUMBER:
+                result = (janet_unwrap_number(x) == janet_unwrap_number(y));
+                break;
+            case JANET_STRING:
+                result = janet_string_equal(janet_unwrap_string(x), janet_unwrap_string(y));
+                break;
+            case JANET_TUPLE:
+                result = janet_tuple_equal(janet_unwrap_tuple(x), janet_unwrap_tuple(y));
+                break;
+            case JANET_STRUCT:
+                result = janet_struct_equal(janet_unwrap_struct(x), janet_unwrap_struct(y));
+                break;
+            default:
+                /* compare pointers */
+                result = (janet_unwrap_pointer(x) == janet_unwrap_pointer(y));
+                break;
         }
     }
     return result;
@@ -65,41 +66,38 @@ int janet_equals(Janet x, Janet y) {
 int32_t janet_hash(Janet x) {
     int32_t hash = 0;
     switch (janet_type(x)) {
-    case JANET_NIL:
-        hash = 0;
-        break;
-    case JANET_FALSE:
-        hash = 1;
-        break;
-    case JANET_TRUE:
-        hash = 2;
-        break;
-    case JANET_STRING:
-    case JANET_SYMBOL:
-    case JANET_KEYWORD:
-        hash = janet_string_hash(janet_unwrap_string(x));
-        break;
-    case JANET_TUPLE:
-        hash = janet_tuple_hash(janet_unwrap_tuple(x));
-        break;
-    case JANET_STRUCT:
-        hash = janet_struct_hash(janet_unwrap_struct(x));
-        break;
-    default:
-        /* TODO - test performance with different hash functions */
-        if (sizeof(double) == sizeof(void *)) {
-            /* Assuming 8 byte pointer */
-            uint64_t i = janet_u64(x);
-            hash = (int32_t)(i & 0xFFFFFFFF);
-            /* Get a bit more entropy by shifting the low bits out */
-            hash >>= 3;
-            hash ^= (int32_t) (i >> 32);
-        } else {
-            /* Assuming 4 byte pointer (or smaller) */
-            hash = (int32_t) ((char *)janet_unwrap_pointer(x) - (char *)0);
-            hash >>= 2;
-        }
-        break;
+        case JANET_NIL:
+            hash = 0;
+            break;
+        case JANET_BOOLEAN:
+            hash = janet_unwrap_boolean(x);
+            break;
+        case JANET_STRING:
+        case JANET_SYMBOL:
+        case JANET_KEYWORD:
+            hash = janet_string_hash(janet_unwrap_string(x));
+            break;
+        case JANET_TUPLE:
+            hash = janet_tuple_hash(janet_unwrap_tuple(x));
+            break;
+        case JANET_STRUCT:
+            hash = janet_struct_hash(janet_unwrap_struct(x));
+            break;
+        default:
+            /* TODO - test performance with different hash functions */
+            if (sizeof(double) == sizeof(void *)) {
+                /* Assuming 8 byte pointer */
+                uint64_t i = janet_u64(x);
+                hash = (int32_t)(i & 0xFFFFFFFF);
+                /* Get a bit more entropy by shifting the low bits out */
+                hash >>= 3;
+                hash ^= (int32_t)(i >> 32);
+            } else {
+                /* Assuming 4 byte pointer (or smaller) */
+                hash = (int32_t)((char *)janet_unwrap_pointer(x) - (char *)0);
+                hash >>= 2;
+            }
+            break;
     }
     return hash;
 }
@@ -111,15 +109,15 @@ int janet_compare(Janet x, Janet y) {
     if (janet_type(x) == janet_type(y)) {
         switch (janet_type(x)) {
             case JANET_NIL:
-            case JANET_FALSE:
-            case JANET_TRUE:
                 return 0;
+            case JANET_BOOLEAN:
+                return janet_unwrap_boolean(x) - janet_unwrap_boolean(y);
             case JANET_NUMBER:
                 /* Check for NaNs to ensure total order */
                 if (janet_unwrap_number(x) != janet_unwrap_number(x))
                     return janet_unwrap_number(y) != janet_unwrap_number(y)
-                        ? 0
-                        : -1;
+                           ? 0
+                           : -1;
                 if (janet_unwrap_number(y) != janet_unwrap_number(y))
                     return 1;
 
@@ -147,13 +145,22 @@ int janet_compare(Janet x, Janet y) {
     return (janet_type(x) < janet_type(y)) ? -1 : 1;
 }
 
+static int32_t getter_checkint(Janet key, int32_t max) {
+    if (!janet_checkint(key)) goto bad;
+    int32_t ret = janet_unwrap_integer(key);
+    if (ret < 0) goto bad;
+    if (ret >= max) goto bad;
+    return ret;
+bad:
+    janet_panicf("expected integer key in range [0, %d), got %v", max, key);
+}
+
 /* Gets a value and returns. Can panic. */
-Janet janet_get(Janet ds, Janet key) {
+Janet janet_in(Janet ds, Janet key) {
     Janet value;
     switch (janet_type(ds)) {
         default:
             janet_panicf("expected %T, got %v", JANET_TFLAG_LENGTHABLE, ds);
-            value = janet_wrap_nil();
             break;
         case JANET_STRUCT:
             value = janet_struct_get(janet_unwrap_struct(ds), key);
@@ -161,75 +168,106 @@ Janet janet_get(Janet ds, Janet key) {
         case JANET_TABLE:
             value = janet_table_get(janet_unwrap_table(ds), key);
             break;
-        case JANET_ARRAY:
-            {
-                JanetArray *array = janet_unwrap_array(ds);
-                int32_t index;
-                if (!janet_checkint(key))
-                    janet_panic("expected integer key");
-                index = janet_unwrap_integer(key);
-                if (index < 0 || index >= array->count) {
-                    value = janet_wrap_nil();
-                } else {
-                    value = array->data[index];
-                }
-                break;
-            }
-        case JANET_TUPLE:
-            {
-                const Janet *tuple = janet_unwrap_tuple(ds);
-                int32_t index;
-                if (!janet_checkint(key))
-                    janet_panic("expected integer key");
-                index = janet_unwrap_integer(key);
-                if (index < 0 || index >= janet_tuple_length(tuple)) {
-                    value = janet_wrap_nil();
-                } else {
-                    value = tuple[index];
-                }
-                break;
-            }
-        case JANET_BUFFER:
-            {
-                JanetBuffer *buffer = janet_unwrap_buffer(ds);
-                int32_t index;
-                if (!janet_checkint(key))
-                    janet_panic("expected integer key");
-                index = janet_unwrap_integer(key);
-                if (index < 0 || index >= buffer->count) {
-                    value = janet_wrap_nil();
-                } else {
-                    value = janet_wrap_integer(buffer->data[index]);
-                }
-                break;
-            }
+        case JANET_ARRAY: {
+            JanetArray *array = janet_unwrap_array(ds);
+            int32_t index = getter_checkint(key, array->count);
+            value = array->data[index];
+            break;
+        }
+        case JANET_TUPLE: {
+            const Janet *tuple = janet_unwrap_tuple(ds);
+            int32_t len = janet_tuple_length(tuple);
+            value = tuple[getter_checkint(key, len)];
+            break;
+        }
+        case JANET_BUFFER: {
+            JanetBuffer *buffer = janet_unwrap_buffer(ds);
+            int32_t index = getter_checkint(key, buffer->count);
+            value = janet_wrap_integer(buffer->data[index]);
+            break;
+        }
         case JANET_STRING:
         case JANET_SYMBOL:
-        case JANET_KEYWORD:
-            {
-                const uint8_t *str = janet_unwrap_string(ds);
-                int32_t index;
-                if (!janet_checkint(key))
-                    janet_panic("expected integer key");
-                index = janet_unwrap_integer(key);
-                if (index < 0 || index >= janet_string_length(str)) {
-                    value = janet_wrap_nil();
-                } else {
-                    value = janet_wrap_integer(str[index]);
-                }
-                break;
+        case JANET_KEYWORD: {
+            const uint8_t *str = janet_unwrap_string(ds);
+            int32_t index = getter_checkint(key, janet_string_length(str));
+            value = janet_wrap_integer(str[index]);
+            break;
+        }
+        case JANET_ABSTRACT: {
+            JanetAbstractType *type = (JanetAbstractType *)janet_abstract_type(janet_unwrap_abstract(ds));
+            if (type->get) {
+                if (!(type->get)(janet_unwrap_abstract(ds), key, &value))
+                    janet_panicf("key %v not found in %v ", key, ds);
+            } else {
+                janet_panicf("no getter for %v ", ds);
             }
+            break;
+        }
     }
     return value;
 }
 
+Janet janet_get(Janet ds, Janet key) {
+    JanetType t = janet_type(ds);
+    switch (t) {
+        default:
+            return janet_wrap_nil();
+        case JANET_STRING:
+        case JANET_SYMBOL:
+        case JANET_KEYWORD: {
+            if (!janet_checkint(key)) return janet_wrap_nil();
+            int32_t index = janet_unwrap_integer(key);
+            if (index < 0) return janet_wrap_nil();
+            const uint8_t *str = janet_unwrap_string(ds);
+            if (index >= janet_string_length(str)) return janet_wrap_nil();
+            return janet_wrap_integer(str[index]);
+        }
+        case JANET_ABSTRACT: {
+            Janet value;
+            void *abst = janet_unwrap_abstract(ds);
+            JanetAbstractType *type = (JanetAbstractType *)janet_abstract_type(abst);
+            if (!type->get) return janet_wrap_nil();
+            if ((type->get)(abst, key, &value))
+                return value;
+            return janet_wrap_nil();
+        }
+        case JANET_ARRAY:
+        case JANET_TUPLE:
+        case JANET_BUFFER: {
+            if (!janet_checkint(key)) return janet_wrap_nil();
+            int32_t index = janet_unwrap_integer(key);
+            if (index < 0) return janet_wrap_nil();
+            if (t == JANET_ARRAY) {
+                JanetArray *a = janet_unwrap_array(ds);
+                if (index >= a->count) return janet_wrap_nil();
+                return a->data[index];
+            } else if (t == JANET_BUFFER) {
+                JanetBuffer *b = janet_unwrap_buffer(ds);
+                if (index >= b->count) return janet_wrap_nil();
+                return janet_wrap_integer(b->data[index]);
+            } else {
+                const Janet *t = janet_unwrap_tuple(ds);
+                if (index >= janet_tuple_length(t)) return janet_wrap_nil();
+                return t[index];
+            }
+        }
+        case JANET_TABLE: {
+            return janet_table_get(janet_unwrap_table(ds), key);
+        }
+        case JANET_STRUCT: {
+            const JanetKV *st = janet_unwrap_struct(ds);
+            return janet_struct_get(st, key);
+        }
+    }
+}
+
 Janet janet_getindex(Janet ds, int32_t index) {
     Janet value;
     if (index < 0) janet_panic("expected non-negative index");
     switch (janet_type(ds)) {
         default:
             janet_panicf("expected %T, got %v", JANET_TFLAG_LENGTHABLE, ds);
-            value = janet_wrap_nil();
             break;
         case JANET_STRING:
         case JANET_SYMBOL:
@@ -267,6 +305,16 @@ Janet janet_getindex(Janet ds, int32_t index) {
         case JANET_STRUCT:
             value = janet_struct_get(janet_unwrap_struct(ds), janet_wrap_integer(index));
             break;
+        case JANET_ABSTRACT: {
+            JanetAbstractType *type = (JanetAbstractType *)janet_abstract_type(janet_unwrap_abstract(ds));
+            if (type->get) {
+                if (!(type->get)(janet_unwrap_abstract(ds), janet_wrap_integer(index), &value))
+                    value = janet_wrap_nil();
+            } else {
+                janet_panicf("no getter for %v ", ds);
+            }
+            break;
+        }
     }
     return value;
 }
@@ -275,7 +323,6 @@ int32_t janet_length(Janet x) {
     switch (janet_type(x)) {
         default:
             janet_panicf("expected %T, got %v", JANET_TFLAG_LENGTHABLE, x);
-            return 0;
         case JANET_STRING:
         case JANET_SYMBOL:
         case JANET_KEYWORD:
@@ -290,6 +337,38 @@ int32_t janet_length(Janet x) {
             return janet_struct_length(janet_unwrap_struct(x));
         case JANET_TABLE:
             return janet_unwrap_table(x)->count;
+        case JANET_ABSTRACT: {
+            Janet argv[1] = { x };
+            Janet len = janet_mcall("length", 1, argv);
+            if (!janet_checkint(len))
+                janet_panicf("invalid integer length %v", len);
+            return janet_unwrap_integer(len);
+        }
+    }
+}
+
+Janet janet_lengthv(Janet x) {
+    switch (janet_type(x)) {
+        default:
+            janet_panicf("expected %T, got %v", JANET_TFLAG_LENGTHABLE, x);
+        case JANET_STRING:
+        case JANET_SYMBOL:
+        case JANET_KEYWORD:
+            return janet_wrap_integer(janet_string_length(janet_unwrap_string(x)));
+        case JANET_ARRAY:
+            return janet_wrap_integer(janet_unwrap_array(x)->count);
+        case JANET_BUFFER:
+            return janet_wrap_integer(janet_unwrap_buffer(x)->count);
+        case JANET_TUPLE:
+            return janet_wrap_integer(janet_tuple_length(janet_unwrap_tuple(x)));
+        case JANET_STRUCT:
+            return janet_wrap_integer(janet_struct_length(janet_unwrap_struct(x)));
+        case JANET_TABLE:
+            return janet_wrap_integer(janet_unwrap_table(x)->count);
+        case JANET_ABSTRACT: {
+            Janet argv[1] = { x };
+            return janet_mcall("length", 1, argv);
+        }
     }
 }
 
@@ -297,36 +376,41 @@ void janet_putindex(Janet ds, int32_t index, Janet value) {
     switch (janet_type(ds)) {
         default:
             janet_panicf("expected %T, got %v",
-                    JANET_TFLAG_ARRAY | JANET_TFLAG_BUFFER | JANET_TFLAG_TABLE, ds);
+                         JANET_TFLAG_ARRAY | JANET_TFLAG_BUFFER | JANET_TFLAG_TABLE, ds);
+        case JANET_ARRAY: {
+            JanetArray *array = janet_unwrap_array(ds);
+            if (index >= array->count) {
+                janet_array_ensure(array, index + 1, 2);
+                array->count = index + 1;
+            }
+            array->data[index] = value;
             break;
-        case JANET_ARRAY:
-            {
-                JanetArray *array = janet_unwrap_array(ds);
-                if (index >= array->count) {
-                    janet_array_ensure(array, index + 1, 2);
-                    array->count = index + 1;
-                }
-                array->data[index] = value;
-                break;
+        }
+        case JANET_BUFFER: {
+            JanetBuffer *buffer = janet_unwrap_buffer(ds);
+            if (!janet_checkint(value))
+                janet_panicf("can only put integers in buffers, got %v", value);
+            if (index >= buffer->count) {
+                janet_buffer_ensure(buffer, index + 1, 2);
+                buffer->count = index + 1;
             }
-        case JANET_BUFFER:
-            {
-                JanetBuffer *buffer = janet_unwrap_buffer(ds);
-                if (!janet_checkint(value))
-                    janet_panicf("can only put integers in buffers, got %v", value);
-                if (index >= buffer->count) {
-                    janet_buffer_ensure(buffer, index + 1, 2);
-                    buffer->count = index + 1;
-                }
-                buffer->data[index] = janet_unwrap_integer(value);
-                break;
-            }
-        case JANET_TABLE:
-            {
-                JanetTable *table = janet_unwrap_table(ds);
-                janet_table_put(table, janet_wrap_integer(index), value);
-                break;
+            buffer->data[index] = (uint8_t)(janet_unwrap_integer(value) & 0xFF);
+            break;
+        }
+        case JANET_TABLE: {
+            JanetTable *table = janet_unwrap_table(ds);
+            janet_table_put(table, janet_wrap_integer(index), value);
+            break;
+        }
+        case JANET_ABSTRACT: {
+            JanetAbstractType *type = (JanetAbstractType *)janet_abstract_type(janet_unwrap_abstract(ds));
+            if (type->put) {
+                (type->put)(janet_unwrap_abstract(ds), janet_wrap_integer(index), value);
+            } else {
+                janet_panicf("no setter for %v ", ds);
             }
+            break;
+        }
     }
 }
 
@@ -334,38 +418,38 @@ void janet_put(Janet ds, Janet key, Janet value) {
     switch (janet_type(ds)) {
         default:
             janet_panicf("expected %T, got %v",
-                    JANET_TFLAG_ARRAY | JANET_TFLAG_BUFFER | JANET_TFLAG_TABLE, ds);
+                         JANET_TFLAG_ARRAY | JANET_TFLAG_BUFFER | JANET_TFLAG_TABLE, ds);
+        case JANET_ARRAY: {
+            JanetArray *array = janet_unwrap_array(ds);
+            int32_t index = getter_checkint(key, INT32_MAX - 1);
+            if (index >= array->count) {
+                janet_array_setcount(array, index + 1);
+            }
+            array->data[index] = value;
             break;
-        case JANET_ARRAY:
-            {
-                int32_t index;
-                JanetArray *array = janet_unwrap_array(ds);
-                if (!janet_checkint(key)) janet_panicf("expected integer key, got %v", key);
-                index = janet_unwrap_integer(key);
-                if (index < 0 || index == INT32_MAX) janet_panicf("bad integer key, got %v", key);
-                if (index >= array->count) {
-                    janet_array_setcount(array, index + 1);
-                }
-                array->data[index] = value;
-                break;
-            }
-        case JANET_BUFFER:
-            {
-                int32_t index;
-                JanetBuffer *buffer = janet_unwrap_buffer(ds);
-                if (!janet_checkint(key)) janet_panicf("expected integer key, got %v", key);
-                index = janet_unwrap_integer(key);
-                if (index < 0 || index == INT32_MAX) janet_panicf("bad integer key, got %v", key);
-                if (!janet_checkint(value))
-                    janet_panicf("can only put integers in buffers, got %v", value);
-                if (index >= buffer->count) {
-                    janet_buffer_setcount(buffer, index + 1);
-                }
-                buffer->data[index] = (uint8_t) (janet_unwrap_integer(value) & 0xFF);
-                break;
+        }
+        case JANET_BUFFER: {
+            JanetBuffer *buffer = janet_unwrap_buffer(ds);
+            int32_t index = getter_checkint(key, INT32_MAX - 1);
+            if (!janet_checkint(value))
+                janet_panicf("can only put integers in buffers, got %v", value);
+            if (index >= buffer->count) {
+                janet_buffer_setcount(buffer, index + 1);
             }
+            buffer->data[index] = (uint8_t)(janet_unwrap_integer(value) & 0xFF);
+            break;
+        }
         case JANET_TABLE:
             janet_table_put(janet_unwrap_table(ds), key, value);
             break;
+        case JANET_ABSTRACT: {
+            JanetAbstractType *type = (JanetAbstractType *)janet_abstract_type(janet_unwrap_abstract(ds));
+            if (type->put) {
+                (type->put)(janet_unwrap_abstract(ds), key, value);
+            } else {
+                janet_panicf("no setter for %v ", ds);
+            }
+            break;
+        }
     }
 }

src/core/vector.c

@@ -22,6 +22,7 @@
 
 #ifndef JANET_AMALG
 #include "vector.h"
+#include "util.h"
 #endif
 
 /* Grow the buffer dynamically. Used for push operations. */
@@ -29,33 +30,11 @@ void *janet_v_grow(void *v, int32_t increment, int32_t itemsize) {
     int32_t dbl_cur = (NULL != v) ? 2 * janet_v__cap(v) : 0;
     int32_t min_needed = janet_v_count(v) + increment;
     int32_t m = dbl_cur > min_needed ? dbl_cur : min_needed;
-    int32_t *p = (int32_t *) realloc(v ? janet_v__raw(v) : 0, itemsize * m + sizeof(int32_t)*2);
-    if (NULL != p) {
-        if (!v) p[1] = 0;
-        p[0] = m;
-        return p + 2;
-   } else {
-       {
-           JANET_OUT_OF_MEMORY;
-       }
-       return (void *) (2 * sizeof(int32_t));
-   }
-}
-
-/* Clone a buffer. */
-void *janet_v_copymem(void *v, int32_t itemsize) {
-    int32_t *p;
-    if (NULL == v) return NULL;
-    p = malloc(2 * sizeof(int32_t) + itemsize * janet_v__cap(v));
-    if (NULL != p) {
-        memcpy(p, janet_v__raw(v), 2 * sizeof(int32_t) + itemsize * janet_v__cnt(v));
-        return p + 2;
-    } else {
-       {
-           JANET_OUT_OF_MEMORY;
-       }
-       return (void *) (2 * sizeof(int32_t));
-    }
+    size_t newsize = ((size_t) itemsize) * m + sizeof(int32_t) * 2;
+    int32_t *p = (int32_t *) janet_srealloc(v ? janet_v__raw(v) : 0, newsize);
+    if (!v) p[1] = 0;
+    p[0] = m;
+    return p + 2;
 }
 
 /* Convert a buffer to normal allocated memory (forget capacity) */
@@ -69,10 +48,10 @@ void *janet_v_flattenmem(void *v, int32_t itemsize) {
         memcpy(p, v, sizen);
         return p;
     } else {
-       {
-           JANET_OUT_OF_MEMORY;
-       }
-       return NULL;
+        {
+            JANET_OUT_OF_MEMORY;
+        }
+        return NULL;
     }
 }
 

src/core/vector.h

@@ -24,7 +24,7 @@
 #define JANET_VECTOR_H_defined
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <janet.h>
 #endif
 
 /*
@@ -33,16 +33,15 @@
 */
 
 /* This is mainly used code such as the assembler or compiler, which
- * need vector like data structures that are not garbage collected
- * and used only from C */
+ * need vector like data structures that are only garbage collected in case
+ * of an error, and normally rely on malloc/free. */
 
-#define janet_v_free(v)         (((v) != NULL) ? (free(janet_v__raw(v)), 0) : 0)
+#define janet_v_free(v)         (((v) != NULL) ? (janet_sfree(janet_v__raw(v)), 0) : 0)
 #define janet_v_push(v, x)      (janet_v__maybegrow(v, 1), (v)[janet_v__cnt(v)++] = (x))
 #define janet_v_pop(v)          (janet_v_count(v) ? janet_v__cnt(v)-- : 0)
 #define janet_v_count(v)        (((v) != NULL) ? janet_v__cnt(v) : 0)
 #define janet_v_last(v)         ((v)[janet_v__cnt(v) - 1])
 #define janet_v_empty(v)        (((v) != NULL) ? (janet_v__cnt(v) = 0) : 0)
-#define janet_v_copy(v)         (janet_v_copymem((v), sizeof(*(v))))
 #define janet_v_flatten(v)      (janet_v_flattenmem((v), sizeof(*(v))))
 
 #define janet_v__raw(v) ((int32_t *)(v) - 2)
@@ -55,7 +54,6 @@
 
 /* Actual functions defined in vector.c */
 void *janet_v_grow(void *v, int32_t increment, int32_t itemsize);
-void *janet_v_copymem(void *v, int32_t itemsize);
 void *janet_v_flattenmem(void *v, int32_t itemsize);
 
 #endif

src/core/wrap.c

@@ -21,12 +21,151 @@
 */
 
 #ifndef JANET_AMALG
-#include <janet/janet.h>
+#include <math.h>
+#include <janet.h>
+#include "util.h"
+#include "state.h"
 #endif
 
+/* Macro fills */
+
+JanetType(janet_type)(Janet x) {
+    return janet_type(x);
+}
+int (janet_checktype)(Janet x, JanetType type) {
+    return janet_checktype(x, type);
+}
+int (janet_checktypes)(Janet x, int typeflags) {
+    return janet_checktypes(x, typeflags);
+}
+int (janet_truthy)(Janet x) {
+    return janet_truthy(x);
+}
+
+const JanetKV *(janet_unwrap_struct)(Janet x) {
+    return janet_unwrap_struct(x);
+}
+const Janet *(janet_unwrap_tuple)(Janet x) {
+    return janet_unwrap_tuple(x);
+}
+JanetFiber *(janet_unwrap_fiber)(Janet x) {
+    return janet_unwrap_fiber(x);
+}
+JanetArray *(janet_unwrap_array)(Janet x) {
+    return janet_unwrap_array(x);
+}
+JanetTable *(janet_unwrap_table)(Janet x) {
+    return janet_unwrap_table(x);
+}
+JanetBuffer *(janet_unwrap_buffer)(Janet x) {
+    return janet_unwrap_buffer(x);
+}
+const uint8_t *(janet_unwrap_string)(Janet x) {
+    return janet_unwrap_string(x);
+}
+const uint8_t *(janet_unwrap_symbol)(Janet x) {
+    return janet_unwrap_symbol(x);
+}
+const uint8_t *(janet_unwrap_keyword)(Janet x) {
+    return janet_unwrap_keyword(x);
+}
+void *(janet_unwrap_abstract)(Janet x) {
+    return janet_unwrap_abstract(x);
+}
+void *(janet_unwrap_pointer)(Janet x) {
+    return janet_unwrap_pointer(x);
+}
+JanetFunction *(janet_unwrap_function)(Janet x) {
+    return janet_unwrap_function(x);
+}
+JanetCFunction(janet_unwrap_cfunction)(Janet x) {
+    return janet_unwrap_cfunction(x);
+}
+int (janet_unwrap_boolean)(Janet x) {
+    return janet_unwrap_boolean(x);
+}
+int32_t (janet_unwrap_integer)(Janet x) {
+    return janet_unwrap_integer(x);
+}
+
+#if defined(JANET_NANBOX_32) || defined(JANET_NANBOX_64)
+Janet(janet_wrap_nil)(void) {
+    return janet_wrap_nil();
+}
+Janet(janet_wrap_true)(void) {
+    return janet_wrap_true();
+}
+Janet(janet_wrap_false)(void) {
+    return janet_wrap_false();
+}
+Janet(janet_wrap_boolean)(int x) {
+    return janet_wrap_boolean(x);
+}
+Janet(janet_wrap_string)(const uint8_t *x) {
+    return janet_wrap_string(x);
+}
+Janet(janet_wrap_symbol)(const uint8_t *x) {
+    return janet_wrap_symbol(x);
+}
+Janet(janet_wrap_keyword)(const uint8_t *x) {
+    return janet_wrap_keyword(x);
+}
+Janet(janet_wrap_array)(JanetArray *x) {
+    return janet_wrap_array(x);
+}
+Janet(janet_wrap_tuple)(const Janet *x) {
+    return janet_wrap_tuple(x);
+}
+Janet(janet_wrap_struct)(const JanetKV *x) {
+    return janet_wrap_struct(x);
+}
+Janet(janet_wrap_fiber)(JanetFiber *x) {
+    return janet_wrap_fiber(x);
+}
+Janet(janet_wrap_buffer)(JanetBuffer *x) {
+    return janet_wrap_buffer(x);
+}
+Janet(janet_wrap_function)(JanetFunction *x) {
+    return janet_wrap_function(x);
+}
+Janet(janet_wrap_cfunction)(JanetCFunction x) {
+    return janet_wrap_cfunction(x);
+}
+Janet(janet_wrap_table)(JanetTable *x) {
+    return janet_wrap_table(x);
+}
+Janet(janet_wrap_abstract)(void *x) {
+    return janet_wrap_abstract(x);
+}
+Janet(janet_wrap_pointer)(void *x) {
+    return janet_wrap_pointer(x);
+}
+Janet(janet_wrap_integer)(int32_t x) {
+    return janet_wrap_integer(x);
+}
+#endif
+
+#ifndef JANET_NANBOX_32
+double (janet_unwrap_number)(Janet x) {
+    return janet_unwrap_number(x);
+}
+#endif
+
+#ifdef JANET_NANBOX_64
+Janet(janet_wrap_number)(double x) {
+    return janet_wrap_number(x);
+}
+#endif
+
+/*****/
+
 void *janet_memalloc_empty(int32_t count) {
     int32_t i;
     void *mem = malloc(count * sizeof(JanetKV));
+    janet_vm_next_collection += count * sizeof(JanetKV);
+    if (NULL == mem) {
+        JANET_OUT_OF_MEMORY;
+    }
     JanetKV *mmem = (JanetKV *)mem;
     for (i = 0; i < count; i++) {
         JanetKV *kv = mmem + i;
@@ -46,6 +185,12 @@ void janet_memempty(JanetKV *mem, int32_t count) {
 
 #ifdef JANET_NANBOX_64
 
+Janet janet_wrap_number_safe(double d) {
+    Janet ret;
+    ret.number = isnan(d) ? NAN : d;
+    return ret;
+}
+
 void *janet_nanbox_to_pointer(Janet x) {
     x.i64 &= JANET_NANBOX_PAYLOADBITS;
     return x.pointer;
@@ -86,6 +231,11 @@ Janet janet_wrap_number(double x) {
     return ret;
 }
 
+Janet janet_wrap_number_safe(double d) {
+    double x = isnan(d) ? NAN : d;
+    return janet_wrap_number(x);
+}
+
 Janet janet_nanbox32_from_tagi(uint32_t tag, int32_t integer) {
     Janet ret;
     ret.tagged.type = tag;
@@ -107,13 +257,11 @@ double janet_unwrap_number(Janet x) {
 
 #else
 
-/* Wrapper functions wrap a data type that is used from C into a
- * janet value, which can then be used in janet internal functions. Use
- * these functions sparingly, as these function will let the programmer
- * leak memory, where as the stack based API ensures that all values can
- * be collected by the garbage collector. */
+Janet janet_wrap_number_safe(double d) {
+    return janet_wrap_number(d);
+}
 
-Janet janet_wrap_nil() {
+Janet janet_wrap_nil(void) {
     Janet y;
     y.type = JANET_NIL;
     y.as.u64 = 0;
@@ -122,22 +270,22 @@ Janet janet_wrap_nil() {
 
 Janet janet_wrap_true(void) {
     Janet y;
-    y.type = JANET_TRUE;
-    y.as.u64 = 0;
+    y.type = JANET_BOOLEAN;
+    y.as.u64 = 1;
     return y;
 }
 
 Janet janet_wrap_false(void) {
     Janet y;
-    y.type = JANET_FALSE;
+    y.type = JANET_BOOLEAN;
     y.as.u64 = 0;
     return y;
 }
 
 Janet janet_wrap_boolean(int x) {
     Janet y;
-    y.type = x ? JANET_TRUE : JANET_FALSE;
-    y.as.u64 = 0;
+    y.type = JANET_BOOLEAN;
+    y.as.u64 = !!x;
     return y;
 }
 
@@ -163,7 +311,9 @@ JANET_WRAP_DEFINE(function, JanetFunction *, JANET_FUNCTION, pointer)
 JANET_WRAP_DEFINE(cfunction, JanetCFunction, JANET_CFUNCTION, pointer)
 JANET_WRAP_DEFINE(table, JanetTable *, JANET_TABLE, pointer)
 JANET_WRAP_DEFINE(abstract, void *, JANET_ABSTRACT, pointer)
+JANET_WRAP_DEFINE(pointer, void *, JANET_POINTER, pointer)
 
 #undef JANET_WRAP_DEFINE
 
 #endif
+

src/mainclient/init.janet

@@ -1,76 +0,0 @@
-# Copyright 2017-2019 (C) Calvin Rose
-
-(do
-
-  (var *should-repl* false)
-  (var *no-file* true)
-  (var *quiet* false)
-  (var *raw-stdin* false)
-  (var *handleopts* true)
-  (var *exit-on-error* true)
-
-  # Flag handlers
-  (def handlers :private
-    {"h" (fn [&]
-           (print "usage: " (get process/args 0) " [options] script args...")
-           (print
-             `Options are:
-  -h Show this help
-  -v Print the version string
-  -s Use raw stdin instead of getline like functionality
-  -e Execute a string of janet
-  -r Enter the repl after running all scripts
-  -p Keep on executing if there is a top level error (persistent)
-  -q Hide prompt, logo, and repl output (quiet)
-  -l Execute code in a file before running the main script
-  -- Stop handling options`)
-           (os/exit 0)
-           1)
-     "v" (fn [&] (print janet/version "-" janet/build) (os/exit 0) 1)
-     "s" (fn [&] (set *raw-stdin* true) (set *should-repl* true) 1)
-     "r" (fn [&] (set *should-repl* true) 1)
-     "p" (fn [&] (set *exit-on-error* false) 1)
-     "q" (fn [&] (set *quiet* true) 1)
-     "-" (fn [&] (set *handleopts* false) 1)
-     "l" (fn [i &]
-           (import* *env* (get process/args (+ i 1))
-                    :prefix "" :exit *exit-on-error*)
-           2)
-     "e" (fn [i &]
-           (set *no-file* false)
-           (eval-string (get process/args (+ i 1)))
-           2)})
-
-  (defn- dohandler [n i &]
-    (def h (get handlers n))
-    (if h (h i) (do (print "unknown flag -" n) ((get handlers "h")))))
-
-  # Process arguments
-  (var i 1)
-  (def lenargs (length process/args))
-  (while (< i lenargs)
-    (def arg (get process/args i))
-    (if (and *handleopts* (= "-" (string/slice arg 0 1)))
-      (+= i (dohandler (string/slice arg 1 2) i))
-      (do
-        (set *no-file* false)
-        (import* *env* arg :prefix "" :exit *exit-on-error*)
-        (set i lenargs))))
-
-  (when (or *should-repl* *no-file*)
-    (if-not *quiet*
-      (print "Janet " janet/version "-" janet/build "  Copyright (C) 2017-2019 Calvin Rose"))
-    (defn noprompt [_] "")
-    (defn getprompt [p]
-      (def offset (parser/where p))
-      (string "janet:" offset ":" (parser/state p) "> "))
-    (def prompter (if *quiet* noprompt getprompt))
-    (defn getstdin [prompt buf]
-      (file/write stdout prompt)
-      (file/flush stdout)
-      (file/read stdin :line buf))
-    (def getter (if *raw-stdin* getstdin getline))
-    (defn getchunk [buf p]
-      (getter (prompter p) buf))
-    (def onsig (if *quiet* (fn [x &] x) nil))
-    (repl getchunk onsig)))

src/mainclient/line.c

@@ -20,23 +20,26 @@
 * IN THE SOFTWARE.
 */
 
+#ifndef JANET_AMALG
 #include "line.h"
+#endif
 
 /* Common */
 Janet janet_line_getter(int32_t argc, Janet *argv) {
-    janet_fixarity(argc, 2);
-    const uint8_t *str = janet_getstring(argv, 0);
-    JanetBuffer *buf = janet_getbuffer(argv, 1);
+    janet_arity(argc, 0, 2);
+    const char *str = (argc >= 1) ? (const char *) janet_getstring(argv, 0) : "";
+    JanetBuffer *buf = (argc >= 2) ? janet_getbuffer(argv, 1) : janet_buffer(10);
     janet_line_get(str, buf);
-    return argv[0];
+    return janet_wrap_buffer(buf);
 }
 
 static void simpleline(JanetBuffer *buffer) {
+    FILE *in = janet_dynfile("in", stdin);
     buffer->count = 0;
     int c;
     for (;;) {
-        c = fgetc(stdin);
-        if (feof(stdin) || c < 0) {
+        c = fgetc(in);
+        if (feof(in) || c < 0) {
             break;
         }
         janet_buffer_push_u8(buffer, (uint8_t) c);
@@ -55,8 +58,10 @@ void janet_line_deinit() {
     ;
 }
 
-void janet_line_get(const uint8_t *p, JanetBuffer *buffer) {
-    fputs((const char *)p, stdout);
+void janet_line_get(const char *p, JanetBuffer *buffer) {
+    FILE *out = janet_dynfile("out", stdout);
+    fputs(p, out);
+    fflush(out);
     simpleline(buffer);
 }
 
@@ -84,17 +89,18 @@ https://github.com/antirez/linenoise/blob/master/linenoise.c
 /* static state */
 #define JANET_LINE_MAX 1024
 #define JANET_HISTORY_MAX 100
-static int israwmode = 0;
-static const char *prompt = "> ";
-static int plen = 2;
-static char buf[JANET_LINE_MAX];
-static int len = 0;
-static int pos = 0;
-static int cols = 80;
-static char *history[JANET_HISTORY_MAX];
-static int history_count = 0;
-static int historyi = 0;
-static struct termios termios_start;
+static JANET_THREAD_LOCAL int gbl_israwmode = 0;
+static JANET_THREAD_LOCAL const char *gbl_prompt = "> ";
+static JANET_THREAD_LOCAL int gbl_plen = 2;
+static JANET_THREAD_LOCAL char gbl_buf[JANET_LINE_MAX];
+static JANET_THREAD_LOCAL int gbl_len = 0;
+static JANET_THREAD_LOCAL int gbl_pos = 0;
+static JANET_THREAD_LOCAL int gbl_cols = 80;
+static JANET_THREAD_LOCAL char *gbl_history[JANET_HISTORY_MAX];
+static JANET_THREAD_LOCAL int gbl_history_count = 0;
+static JANET_THREAD_LOCAL int gbl_historyi = 0;
+static JANET_THREAD_LOCAL int gbl_sigint_flag = 0;
+static JANET_THREAD_LOCAL struct termios gbl_termios_start;
 
 /* Unsupported terminal list from linenoise */
 static const char *badterms[] = {
@@ -117,16 +123,15 @@ static char *sdup(const char *s) {
 static int rawmode() {
     struct termios t;
     if (!isatty(STDIN_FILENO)) goto fatal;
-    if (tcgetattr(STDIN_FILENO, &termios_start) == -1) goto fatal;
-    t = termios_start;
+    if (tcgetattr(STDIN_FILENO, &gbl_termios_start) == -1) goto fatal;
+    t = gbl_termios_start;
     t.c_iflag &= ~(BRKINT | ICRNL | INPCK | ISTRIP | IXON);
-    t.c_oflag &= ~(OPOST);
     t.c_cflag |= (CS8);
     t.c_lflag &= ~(ECHO | ICANON | IEXTEN | ISIG);
     t.c_cc[VMIN] = 1;
     t.c_cc[VTIME] = 0;
     if (tcsetattr(STDIN_FILENO, TCSAFLUSH, &t) < 0) goto fatal;
-    israwmode = 1;
+    gbl_israwmode = 1;
     return 0;
 fatal:
     errno = ENOTTY;
@@ -135,8 +140,8 @@ fatal:
 
 /* Disable raw mode */
 static void norawmode() {
-    if (israwmode && tcsetattr(STDIN_FILENO, TCSAFLUSH, &termios_start) != -1)
-        israwmode = 0;
+    if (gbl_israwmode && tcsetattr(STDIN_FILENO, TCSAFLUSH, &gbl_termios_start) != -1)
+        gbl_israwmode = 0;
 }
 
 static int curpos() {
@@ -144,8 +149,8 @@ static int curpos() {
     int cols, rows;
     unsigned int i = 0;
     if (write(STDOUT_FILENO, "\x1b[6n", 4) != 4) return -1;
-    while (i < sizeof(buf)-1) {
-        if (read(STDIN_FILENO, buf+i, 1) != 1) break;
+    while (i < sizeof(buf) - 1) {
+        if (read(STDIN_FILENO, buf + i, 1) != 1) break;
         if (buf[i] == 'R') break;
         i++;
     }
@@ -166,8 +171,10 @@ static int getcols() {
         if (cols == -1) goto failed;
         if (cols > start) {
             char seq[32];
-            snprintf(seq, 32, "\x1b[%dD", cols-start);
-            if (write(STDOUT_FILENO, seq, strlen(seq)) == -1) {}
+            snprintf(seq, 32, "\x1b[%dD", cols - start);
+            if (write(STDOUT_FILENO, seq, strlen(seq)) == -1) {
+                exit(1);
+            }
         }
         return cols;
     } else {
@@ -178,7 +185,9 @@ failed:
 }
 
 static void clear() {
-    if (write(STDOUT_FILENO,"\x1b[H\x1b[2J",7) <= 0) {}
+    if (write(STDOUT_FILENO, "\x1b[H\x1b[2J", 7) <= 0) {
+        exit(1);
+    }
 }
 
 static void refresh() {
@@ -186,38 +195,40 @@ static void refresh() {
     JanetBuffer b;
 
     /* Keep cursor position on screen */
-    char *_buf = buf;
-    int _len = len;
-    int _pos = pos;
-    while ((plen + _pos) >= cols) {
+    char *_buf = gbl_buf;
+    int _len = gbl_len;
+    int _pos = gbl_pos;
+    while ((gbl_plen + _pos) >= gbl_cols) {
         _buf++;
         _len--;
         _pos--;
     }
-    while ((plen + _len) > cols) {
+    while ((gbl_plen + _len) > gbl_cols) {
         _len--;
     }
 
     janet_buffer_init(&b, 0);
-    /* Cursor to left edge, prompt and buffer */
+    /* Cursor to left edge, gbl_prompt and buffer */
     janet_buffer_push_u8(&b, '\r');
-    janet_buffer_push_cstring(&b, prompt);
+    janet_buffer_push_cstring(&b, gbl_prompt);
     janet_buffer_push_bytes(&b, (uint8_t *) _buf, _len);
     /* Erase to right */
     janet_buffer_push_cstring(&b, "\x1b[0K");
     /* Move cursor to original position. */
-    snprintf(seq, 64,"\r\x1b[%dC", (int)(_pos + plen));
+    snprintf(seq, 64, "\r\x1b[%dC", (int)(_pos + gbl_plen));
     janet_buffer_push_cstring(&b, seq);
-    if (write(STDOUT_FILENO, b.data, b.count) == -1) {}
+    if (write(STDOUT_FILENO, b.data, b.count) == -1) {
+        exit(1);
+    }
     janet_buffer_deinit(&b);
 }
 
 static int insert(char c) {
-    if (len < JANET_LINE_MAX - 1) {
-        if (len == pos) {
-            buf[pos++] = c;
-            buf[++len] = '\0';
-            if (plen + len < cols) {
+    if (gbl_len < JANET_LINE_MAX - 1) {
+        if (gbl_len == gbl_pos) {
+            gbl_buf[gbl_pos++] = c;
+            gbl_buf[++gbl_len] = '\0';
+            if (gbl_plen + gbl_len < gbl_cols) {
                 /* Avoid a full update of the line in the
                  * trivial case. */
                 if (write(STDOUT_FILENO, &c, 1) == -1) return -1;
@@ -225,9 +236,9 @@ static int insert(char c) {
                 refresh();
             }
         } else {
-            memmove(buf + pos + 1, buf + pos, len - pos);
-            buf[pos++] = c;
-            buf[++len] = '\0';
+            memmove(gbl_buf + gbl_pos + 1, gbl_buf + gbl_pos, gbl_len - gbl_pos);
+            gbl_buf[gbl_pos++] = c;
+            gbl_buf[++gbl_len] = '\0';
             refresh();
         }
     }
@@ -235,21 +246,21 @@ static int insert(char c) {
 }
 
 static void historymove(int delta) {
-    if (history_count > 1) {
-        free(history[historyi]);
-        history[historyi] = sdup(buf);
+    if (gbl_history_count > 1) {
+        free(gbl_history[gbl_historyi]);
+        gbl_history[gbl_historyi] = sdup(gbl_buf);
 
-        historyi += delta;
-        if (historyi < 0) {
-            historyi = 0;
+        gbl_historyi += delta;
+        if (gbl_historyi < 0) {
+            gbl_historyi = 0;
             return;
-        } else if (historyi >= history_count) {
-            historyi = history_count - 1;
+        } else if (gbl_historyi >= gbl_history_count) {
+            gbl_historyi = gbl_history_count - 1;
             return;
         }
-        strncpy(buf, history[historyi], JANET_LINE_MAX - 1);
-        pos = len = strlen(buf);
-        buf[len] = '\0';
+        strncpy(gbl_buf, gbl_history[gbl_historyi], JANET_LINE_MAX - 1);
+        gbl_pos = gbl_len = strlen(gbl_buf);
+        gbl_buf[gbl_len] = '\0';
 
         refresh();
     }
@@ -257,62 +268,70 @@ static void historymove(int delta) {
 
 static void addhistory() {
     int i, len;
-    char *newline = sdup(buf);
+    char *newline = sdup(gbl_buf);
     if (!newline) return;
-    len = history_count;
+    len = gbl_history_count;
     if (len < JANET_HISTORY_MAX) {
-        history[history_count++] = newline;
+        gbl_history[gbl_history_count++] = newline;
         len++;
     } else {
-        free(history[JANET_HISTORY_MAX - 1]);
+        free(gbl_history[JANET_HISTORY_MAX - 1]);
     }
     for (i = len - 1; i > 0; i--) {
-        history[i] = history[i - 1];
+        gbl_history[i] = gbl_history[i - 1];
     }
-    history[0] = newline;
+    gbl_history[0] = newline;
 }
 
 static void replacehistory() {
-    char *newline = sdup(buf);
+    char *newline = sdup(gbl_buf);
     if (!newline) return;
-    free(history[0]);
-    history[0] = newline;
+    free(gbl_history[0]);
+    gbl_history[0] = newline;
 }
 
 static void kleft() {
-    if (pos > 0) {
-        pos--;
+    if (gbl_pos > 0) {
+        gbl_pos--;
         refresh();
     }
 }
 
 static void kright() {
-    if (pos != len) {
-        pos++;
+    if (gbl_pos != gbl_len) {
+        gbl_pos++;
         refresh();
     }
 }
 
 static void kbackspace() {
-    if (pos > 0) {
-        memmove(buf + pos - 1, buf + pos, len - pos);
-        pos--;
-        buf[--len] = '\0';
+    if (gbl_pos > 0) {
+        memmove(gbl_buf + gbl_pos - 1, gbl_buf + gbl_pos, gbl_len - gbl_pos);
+        gbl_pos--;
+        gbl_buf[--gbl_len] = '\0';
+        refresh();
+    }
+}
+
+static void kdelete() {
+    if (gbl_pos != gbl_len) {
+        memmove(gbl_buf + gbl_pos, gbl_buf + gbl_pos + 1, gbl_len - gbl_pos);
+        gbl_buf[--gbl_len] = '\0';
         refresh();
     }
 }
 
 static int line() {
-    cols = getcols();
-    plen = 0;
-    len = 0;
-    pos = 0;
-    while (prompt[plen]) plen++;
-    buf[0] = '\0';
+    gbl_cols = getcols();
+    gbl_plen = 0;
+    gbl_len = 0;
+    gbl_pos = 0;
+    while (gbl_prompt[gbl_plen]) gbl_plen++;
+    gbl_buf[0] = '\0';
 
     addhistory();
 
-    if (write(STDOUT_FILENO, prompt, plen) == -1) return -1;
+    if (write(STDOUT_FILENO, gbl_prompt, gbl_plen) == -1) return -1;
     for (;;) {
         char c;
         int nread;
@@ -321,103 +340,107 @@ static int line() {
         nread = read(STDIN_FILENO, &c, 1);
         if (nread <= 0) return -1;
 
-        switch(c) {
-        default:
-            if (insert(c)) return -1;
-            break;
-        case 9:     /* tab */
-            if (insert(' ')) return -1;
-            if (insert(' ')) return -1;
-            break;
-        case 13:    /* enter */
-            return 0;
-        case 3:     /* ctrl-c */
-            errno = EAGAIN;
-            return -1;
-        case 127:   /* backspace */
-        case 8:     /* ctrl-h */
-            kbackspace();
-            break;
-        case 4:     /* ctrl-d, eof */
-            return -1;
-        case 2:     /* ctrl-b */
-            kleft();
-            break;
-        case 6:     /* ctrl-f */
-            kright();
-            break;
-        case 21:
-            buf[0] = '\0';
-            pos = len = 0;
-            refresh();
-            break;
-        case 26: /* ctrl-z */
-            norawmode();
-            kill(getpid(), SIGSTOP);
-            rawmode();
-            refresh();
-            break;
-        case 12:
-            clear();
-            refresh();
-            break;
-        case 27:    /* escape sequence */
-            /* Read the next two bytes representing the escape sequence.
-             * Use two calls to handle slow terminals returning the two
-             * chars at different times. */
-            if (read(STDIN_FILENO, seq, 1) == -1) break;
-            if (read(STDIN_FILENO, seq + 1, 1) == -1) break;
-            if (seq[0] == '[') {
-                if (seq[1] >= '0' && seq[1] <= '9') {
-                    /* Extended escape, read additional byte. */
-                    if (read(STDIN_FILENO, seq + 2, 1) == -1) break;
-                    if (seq[2] == '~') {
-                        switch(seq[1]) {
-                        default:
-                            break;
+        switch (c) {
+            default:
+                if (insert(c)) return -1;
+                break;
+            case 9:     /* tab */
+                if (insert(' ')) return -1;
+                if (insert(' ')) return -1;
+                break;
+            case 13:    /* enter */
+                return 0;
+            case 3:     /* ctrl-c */
+                errno = EAGAIN;
+                gbl_sigint_flag = 1;
+                return -1;
+            case 127:   /* backspace */
+            case 8:     /* ctrl-h */
+                kbackspace();
+                break;
+            case 4:     /* ctrl-d, eof */
+                return -1;
+            case 2:     /* ctrl-b */
+                kleft();
+                break;
+            case 6:     /* ctrl-f */
+                kright();
+                break;
+            case 21:
+                gbl_buf[0] = '\0';
+                gbl_pos = gbl_len = 0;
+                refresh();
+                break;
+            case 26: /* ctrl-z */
+                norawmode();
+                kill(getpid(), SIGSTOP);
+                rawmode();
+                refresh();
+                break;
+            case 12:
+                clear();
+                refresh();
+                break;
+            case 27:    /* escape sequence */
+                /* Read the next two bytes representing the escape sequence.
+                 * Use two calls to handle slow terminals returning the two
+                 * chars at different times. */
+                if (read(STDIN_FILENO, seq, 1) == -1) break;
+                if (read(STDIN_FILENO, seq + 1, 1) == -1) break;
+                if (seq[0] == '[') {
+                    if (seq[1] >= '0' && seq[1] <= '9') {
+                        /* Extended escape, read additional byte. */
+                        if (read(STDIN_FILENO, seq + 2, 1) == -1) break;
+                        if (seq[2] == '~') {
+                            switch (seq[1]) {
+                                case '3': /* delete */
+                                    kdelete();
+                                    break;
+                                default:
+                                    break;
+                            }
+                        }
+                    } else {
+                        switch (seq[1]) {
+                            default:
+                                break;
+                            case 'A':
+                                historymove(1);
+                                break;
+                            case 'B':
+                                historymove(-1);
+                                break;
+                            case 'C': /* Right */
+                                kright();
+                                break;
+                            case 'D': /* Left */
+                                kleft();
+                                break;
+                            case 'H':
+                                gbl_pos = 0;
+                                refresh();
+                                break;
+                            case 'F':
+                                gbl_pos = gbl_len;
+                                refresh();
+                                break;
                         }
                     }
-                } else {
+                } else if (seq[0] == 'O') {
                     switch (seq[1]) {
-                    default:
-                        break;
-                    case 'A':
-                        historymove(1);
-                        break;
-                    case 'B':
-                        historymove(-1);
-                        break;
-                    case 'C': /* Right */
-                        kright();
-                        break;
-                    case 'D': /* Left */
-                        kleft();
-                        break;
-                    case 'H':
-                        pos = 0;
-                        refresh();
-                        break;
-                    case 'F':
-                        pos = len;
-                        refresh();
-                        break;
+                        default:
+                            break;
+                        case 'H':
+                            gbl_pos = 0;
+                            refresh();
+                            break;
+                        case 'F':
+                            gbl_pos = gbl_len;
+                            refresh();
+                            break;
                     }
                 }
-            } else if (seq[0] == 'O') {
-                switch (seq[1]) {
-                default:
-                    break;
-                case 'H':
-                    pos = 0;
-                    refresh();
-                    break;
-                case 'F':
-                    pos = len;
-                    refresh();
-                    break;
-                }
-            }
-            break;
+                break;
         }
     }
     return 0;
@@ -430,9 +453,9 @@ void janet_line_init() {
 void janet_line_deinit() {
     int i;
     norawmode();
-    for (i = 0; i < history_count; i++)
-        free(history[i]);
-    historyi = 0;
+    for (i = 0; i < gbl_history_count; i++)
+        free(gbl_history[i]);
+    gbl_historyi = 0;
 }
 
 static int checktermsupport() {
@@ -444,10 +467,11 @@ static int checktermsupport() {
     return 1;
 }
 
-void janet_line_get(const uint8_t *p, JanetBuffer *buffer) {
-    prompt = (const char *)p;
+void janet_line_get(const char *p, JanetBuffer *buffer) {
+    gbl_prompt = p;
     buffer->count = 0;
-    historyi = 0;
+    gbl_historyi = 0;
+    FILE *out = janet_dynfile("out", stdout);
     if (!isatty(STDIN_FILENO) || !checktermsupport()) {
         simpleline(buffer);
         return;
@@ -458,15 +482,20 @@ void janet_line_get(const uint8_t *p, JanetBuffer *buffer) {
     }
     if (line()) {
         norawmode();
-        fputc('\n', stdout);
+        if (gbl_sigint_flag) {
+            raise(SIGINT);
+        } else {
+            fputc('\n', out);
+        }
         return;
     }
+    fflush(stdin);
     norawmode();
-    fputc('\n', stdout);
-    janet_buffer_ensure(buffer, len + 1, 2);
-    memcpy(buffer->data, buf, len);
-    buffer->data[len] = '\n';
-    buffer->count = len + 1;
+    fputc('\n', out);
+    janet_buffer_ensure(buffer, gbl_len + 1, 2);
+    memcpy(buffer->data, gbl_buf, gbl_len);
+    buffer->data[gbl_len] = '\n';
+    buffer->count = gbl_len + 1;
     replacehistory();
 }
 

src/mainclient/line.h

@@ -23,12 +23,14 @@
 #ifndef JANET_LINE_H_defined
 #define JANET_LINE_H_defined
 
-#include <janet/janet.h>
+#ifndef JANET_AMALG
+#include <janet.h>
+#endif
 
 void janet_line_init();
 void janet_line_deinit();
 
-void janet_line_get(const uint8_t *p, JanetBuffer *buffer);
+void janet_line_get(const char *p, JanetBuffer *buffer);
 Janet janet_line_getter(int32_t argc, Janet *argv);
 
 #endif

src/mainclient/main.c

@@ -20,34 +20,63 @@
 * IN THE SOFTWARE.
 */
 
-#include <janet/janet.h>
+#ifndef JANET_AMALG
+#include <janet.h>
 #include "line.h"
+#endif
 
-extern const unsigned char *janet_gen_init;
-extern int32_t janet_gen_init_size;
+#ifdef _WIN32
+#include <windows.h>
+#include <shlwapi.h>
+#ifndef ENABLE_VIRTUAL_TERMINAL_PROCESSING
+#define ENABLE_VIRTUAL_TERMINAL_PROCESSING 0x0004
+#endif
+#endif
 
 int main(int argc, char **argv) {
     int i, status;
     JanetArray *args;
     JanetTable *env;
 
+#ifdef _WIN32
+    /* Enable color console on windows 10 console and utf8 output. */
+    HANDLE hOut = GetStdHandle(STD_OUTPUT_HANDLE);
+    DWORD dwMode = 0;
+    GetConsoleMode(hOut, &dwMode);
+    dwMode |= ENABLE_VIRTUAL_TERMINAL_PROCESSING;
+    SetConsoleMode(hOut, dwMode);
+    SetConsoleOutputCP(65001);
+#endif
+
     /* Set up VM */
     janet_init();
-    env = janet_core_env();
+
+    /* Replace original getline with new line getter */
+    JanetTable *replacements = janet_table(0);
+    janet_table_put(replacements, janet_csymbolv("getline"), janet_wrap_cfunction(janet_line_getter));
+    janet_line_init();
+
+    /* Get core env */
+    env = janet_core_env(replacements);
 
     /* Create args tuple */
     args = janet_array(argc);
-    for (i = 0; i < argc; i++)
+    for (i = 1; i < argc; i++)
         janet_array_push(args, janet_cstringv(argv[i]));
-    janet_def(env, "process/args", janet_wrap_array(args), "Command line arguments.");
 
-    /* Expose line getter */
-    janet_def(env, "getline", janet_wrap_cfunction(janet_line_getter), NULL);
-    janet_register("getline", janet_line_getter);
-    janet_line_init();
+    /* Save current executable path to (dyn :executable) */
+    janet_table_put(env, janet_ckeywordv("executable"), janet_cstringv(argv[0]));
 
     /* Run startup script */
-    status = janet_dobytes(env, janet_gen_init, janet_gen_init_size, "init.janet", NULL);
+    Janet mainfun, out;
+    janet_resolve(env, janet_csymbol("cli-main"), &mainfun);
+    Janet mainargs[1] = { janet_wrap_array(args) };
+    JanetFiber *fiber = janet_fiber(janet_unwrap_function(mainfun), 64, 1, mainargs);
+    fiber->env = env;
+    status = janet_continue(fiber, janet_wrap_nil(), &out);
+    if (status != JANET_SIGNAL_OK) {
+        janet_stacktrace(fiber, out);
+    }
 
     /* Deinitialize vm */
     janet_deinit();

src/webclient/main.c

@@ -20,7 +20,7 @@
 * IN THE SOFTWARE.
 */
 
-#include <janet/janet.h>
+#include <janet.h>
 #include <emscripten.h>
 
 extern const unsigned char *janet_gen_webinit;
@@ -44,7 +44,7 @@ static int enter_loop(void) {
     Janet ret;
     JanetSignal status = janet_continue(repl_fiber, janet_wrap_nil(), &ret);
     if (status == JANET_SIGNAL_ERROR) {
-        janet_stacktrace(repl_fiber, "runtime", ret);
+        janet_stacktrace(repl_fiber, ret);
         janet_deinit();
         repl_fiber = NULL;
         return 1;
@@ -70,7 +70,7 @@ void repl_init(void) {
     janet_init();
     janet_register("repl-yield", repl_yield);
     janet_register("js", cfun_js);
-    env = janet_core_env();
+    env = janet_core_env(NULL);
 
     janet_def(env, "repl-yield", janet_wrap_cfunction(repl_yield), NULL);
     janet_def(env, "js", janet_wrap_cfunction(cfun_js), NULL);

src/webclient/webinit.janet

@@ -1,11 +1,12 @@
 # Copyright 2017-2019 (C) Calvin Rose
 
-(print (string "Janet " janet/version "-" janet/build "  Copyright (C) 2017-2018 Calvin Rose"))
+(print (string "Janet " janet/version "-" janet/build "  Copyright (C) 2017-2019 Calvin Rose"))
 
 (fiber/new (fn webrepl []
-  (repl (fn get-line [buf p]
-          (def offset (parser/where p))
-          (def prompt (string "janet:" offset ":" (parser/state p) "> "))
-          (repl-yield prompt buf)
-          (yield)
-          buf))))
+             (setdyn :pretty-format "%.20P")
+             (repl (fn get-line [buf p]
+                     (def [offset] (parser/where p))
+                     (def prompt (string "janet:" offset ":" (parser/state p :delimiters) "> "))
+                     (repl-yield prompt buf)
+                     (yield)
+                     buf))))

test/amalg/main.c

@@ -0,0 +1,36 @@
+/*
+* Copyright (c) 2019 Calvin Rose
+*
+* Permission is hereby granted, free of charge, to any person obtaining a copy
+* of this software and associated documentation files (the "Software"), to
+* deal in the Software without restriction, including without limitation the
+* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+* sell copies of the Software, and to permit persons to whom the Software is
+* furnished to do so, subject to the following conditions:
+*
+* The above copyright notice and this permission notice shall be included in
+* all copies or substantial portions of the Software.
+*
+* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+* IN THE SOFTWARE.
+*/
+
+/* A simple client for checking if the amalgamated Janet source compiles
+ * correctly. */
+
+#include <janet.h>
+
+int main(int argc, const char *argv[]) {
+    (void) argc;
+    (void) argv;
+    janet_init();
+    JanetTable *env = janet_core_env(NULL);
+    janet_dostring(env, "(print `hello, world!`)", "main", NULL);
+    janet_deinit();
+    return 0;
+}

test/helper.janet

@@ -26,6 +26,11 @@
   (def errsym (keyword (gensym)))
   ~(assert (= ,errsym (try (do ,;forms) ([_] ,errsym))) ,msg))
 
+(defmacro assert-no-error
+  [msg & forms]
+  (def errsym (keyword (gensym)))
+  ~(assert (not= ,errsym (try (do ,;forms) ([_] ,errsym))) ,msg))
+
 (defn start-suite [x]
  (set suite-num x)
  (print "\nRunning test suite " x " tests...\n  "))

test/install/.gitignore

@@ -0,0 +1,9 @@
+/build
+.cache
+.manifests
+json.*
+jhydro.*
+circlet.*
+argparse.*
+sqlite3.*
+path.*

test/install/project.janet

@@ -0,0 +1,10 @@
+(declare-project
+  :name "testmod")
+
+(declare-native
+    :name "testmod"
+    :source @["testmod.c"])
+
+(declare-executable
+  :name "testexec"
+  :entry "testexec.janet")

test/install/test/test1.janet

@@ -0,0 +1,3 @@
+(import build/testmod :as testmod)
+
+(if (not= 5 (testmod/get5)) (error "testmod/get5 failed"))