1
0
mirror of https://github.com/janet-lang/janet synced 2025-10-28 06:07:43 +00:00

Compare commits

...

87 Commits

Author SHA1 Message Date
Calvin Rose
0d3c6abee8 POLLER -> POLLERR 2020-04-18 19:15:59 -04:00
Calvin Rose
4a693222b4 Port net code to windows.
Use winsock2 and WSAPoll. Not the most high performance
solution but should work well.
2020-04-18 19:14:38 -04:00
Calvin Rose
0745c15d7b Fix return value from shell.c 2020-04-18 15:31:46 -05:00
Calvin Rose
2904c19ed9 Switch to poll from select.
Simpler and more flexible interface, and also lets
us use epoll more easily on linux, which is the most important
plantform to optimize for network performance.
2020-04-18 15:22:20 -05:00
Calvin Rose
4ac382e553 Add alias JANET_SIGNAL_EVENT. 2020-04-17 16:27:02 -05:00
Calvin Rose
596111c988 Merge branch 'master' into net 2020-04-17 15:08:26 -05:00
Calvin Rose
e202d30835 Use make format. 2020-04-17 13:39:23 -05:00
Calvin Rose
fbe903b277 Add janet_cfuns_prefix to janet.h
Makes adding functions to the current environment easier.
2020-04-17 13:37:52 -05:00
Calvin Rose
8a89e50c13 :octal-permissions -> :int-permissions (#347) 2020-04-16 19:05:00 -05:00
Calvin Rose
6cb0e0dcea Merge branch 'master' of github.com:janet-lang/janet 2020-04-16 19:02:58 -05:00
Calvin Rose
a147ea3e80 Use JANET_PRETTY_DICT_LIMIT. 2020-04-16 19:01:49 -05:00
Calvin Rose
557988e530 Merge pull request #344 from DavidKorczynski/master
Added a fuzzer and integration with OSS-Fuzz.
2020-04-16 18:50:07 -05:00
Calvin Rose
67fb2c212f Address #348
Remove extreneous data from lockfile.
2020-04-16 18:44:21 -05:00
Calvin Rose
3765b08cca Merge branch 'master' of github.com:janet-lang/janet 2020-04-16 12:11:59 -05:00
Calvin Rose
3eb84fcb13 Fix some typos, make jpm repl work without a project.janet. 2020-04-16 12:11:17 -05:00
Calvin Rose
bea76e8e08 Merge pull request #345 from sogaiu/checks-after-allocs
Check some *alloc return values
2020-04-15 19:45:39 -05:00
Calvin Rose
f5433dcaa4 Fix core getline that doesn't use replacement. 2020-04-15 19:45:17 -05:00
Calvin Rose
ef3b953a42 Fix docstrings. 2020-04-14 21:32:50 -05:00
Calvin Rose
605a205008 Range errors for slice-likes include negatives.
Makes for less confusing errors when calling something
like `(slice [] 0 -10)`.
2020-04-14 21:27:48 -05:00
Calvin Rose
058f63b440 Add sh-rule and sh-phony to jpm dialect.
Provides useful shorthand for writing rules that invoke
shell commands.
2020-04-14 20:43:53 -05:00
Calvin Rose
71882475d6 janet_formatb -> janet_formatbv, new janet_formatb
The old function was not very useable. In the likely
case that there is no external code using this
(not well documented/janet_formatc is more convenient), we
can change this.
2020-04-14 07:38:41 -05:00
sogaiu
a3d29a15df Check some *alloc return values 2020-04-14 10:22:45 +01:00
Calvin Rose
a09112404d Add better error message on unexpected eos.
Show innermost open delimiter
2020-04-13 23:18:27 -05:00
Calvin Rose
93fc11ea21 Add edefer.
Also improve error messages from vm internal errors.
(Show bad value, not its type).
2020-04-13 20:24:11 -05:00
davkor
4faa129b8e Added a first fuzzer. 2020-04-13 17:33:58 +01:00
Calvin Rose
6c4ed0409d Add emscripten check to features.h. 2020-04-12 14:13:55 -05:00
Calvin Rose
ea2811f14f Merge branch 'master' of github.com:janet-lang/janet 2020-04-11 13:42:34 -05:00
Calvin Rose
8bc2987a71 (struct ...) with duped keys will use last value. 2020-04-11 13:42:25 -05:00
Calvin Rose
1d13095d19 Merge pull request #340 from pepe/get-vs-in-last
Fix last for empty collection, add tests
2020-04-10 19:03:21 -05:00
Calvin Rose
5ed76f197a Differentiate error from resume and error from resumed fiber. 2020-04-10 18:29:10 -05:00
Calvin Rose
e1f4cadf41 Add debugger to the core repl.
Debugger functions are prefixed by periods.
2020-04-10 17:20:23 -05:00
Calvin Rose
3b0e6357ad Make Ctrl-G in repl show docstring for symbol.
Can be used to browse docs without poluting your repl session.
2020-04-10 11:36:23 -05:00
Calvin Rose
02f17bd4e4 Add sort-by and sorted-by. 2020-04-09 20:43:51 -05:00
Josef Pospíšil
b63a0796fd Fix last for empty collection, add tests 2020-04-09 14:35:57 +02:00
Calvin Rose
e6d4e729fb Keep reference alive so unmarshalled object not collected. 2020-04-06 17:24:52 -05:00
Calvin Rose
b75a22b753 Make JANET_FRAME_SIZE consistent across architectures.
This means unmarshalling fibers should work across arches.
2020-04-06 12:41:56 -05:00
Calvin Rose
72beeeeaaa Move funcenv verification to runtime.
Lazy verification makes it easier to not leave funcenvs
in an invalid state, as well as be more precise with the validation.
We needed to verify the FuncEnvs actually pointed to a stack frame if
they were of the "on-stack" variant. There was some minor checking
before, but it was not enough to prevent func envs from pointing to
memory that was off of the fiber stack, overlapping stack frames, etc.
2020-04-06 10:58:47 -05:00
Calvin Rose
c3c42ef56f Fix case for #336.
Also consider ascii 127 (delete) non-printable for string escapes.
2020-04-06 00:11:22 -05:00
Calvin Rose
a3c55681b2 Address #336 case 6 2020-04-05 21:39:39 -05:00
Calvin Rose
cc70388846 Merge pull request #338 from andrewchambers/unmarshalfuzz2
Make unmarshal fuzzer exercise more code paths.
2020-04-05 20:36:30 -05:00
Calvin Rose
fcc610f539 Address #336 case 4
Set funcenv fields to NULL before any possible panics.
2020-04-05 19:18:59 -05:00
Calvin Rose
5bbd507858 Address #336 case 3
Fix error condition for bad abstract types - don't return NULL, panic.
2020-04-05 17:38:14 -05:00
Andrew Chambers
45156c0c47 Make unmarshal fuzzer exercise more code paths. 2020-04-06 09:59:00 +12:00
Calvin Rose
553e38ffd6 Merge pull request #337 from andrewchambers/fuzzunmarshal
Setup some simple fuzz helpers for unmarshal.
2020-04-05 08:17:42 -05:00
Calvin Rose
c4ca0490ee Prevent unmarsal DOS in arrays,buffers,tables,and structs. 2020-04-05 08:16:40 -05:00
Calvin Rose
b145d47863 Address cases 1 and 2 of #336.
Mainly related to not checking ints < 0.
2020-04-05 08:01:18 -05:00
Calvin Rose
095827a261 Update CHANGELOG.md 2020-04-05 07:12:00 -05:00
Calvin Rose
87ecdb8112 Change \UXXXXXXXX -> \UXXXXXX and check codepoint max.
No need to add two extra leading zeros, as the max unicode
codepoint is 0x10FFFF.
2020-04-05 07:09:53 -05:00
Andrew Chambers
98b2fa4d64 Setup some simple fuzz helpers for unmarshal. 2020-04-05 23:05:18 +12:00
Calvin Rose
810ef7401c Update changelog and bump version to dev version. 2020-04-04 21:50:27 -05:00
Calvin Rose
ae70a03383 Address #306 - Add unicode escapes.
Unicode escapes have the same syntax as go - \uXXXX or \UXXXXXXXX.
2020-04-04 21:46:08 -05:00
Calvin Rose
081d132538 Address #321
Also improve docs for dofile and related functions.
2020-04-04 21:17:15 -05:00
Calvin Rose
bb5c478704 Switch to two digit sonames.
Janet's versioning scheme is not 'true' semantic versioning.
Minor versions can have and often do have breaking changes.
Although such breakages are mostly avoided, only limited effort is
made to prevent this, and no system is in place to verify this.
Thus, stricter version pinning is needed.
2020-04-04 18:30:18 -05:00
Calvin Rose
ff6601f29e Add version and soversion to meson libjanet. 2020-04-04 18:04:22 -05:00
Calvin Rose
320c6c6f05 Increase NSIS installer verbosity. 2020-04-04 13:58:27 -05:00
Calvin Rose
6b89da4bb2 Use -Wl,-install_name,... on macos. 2020-04-04 13:44:21 -05:00
Calvin Rose
5b82b9e101 Address compiler warning on macos. 2020-04-04 13:34:16 -05:00
Calvin Rose
1d0e862129 Update Makefile for pkg-config issues and soname. 2020-04-04 13:09:59 -05:00
Calvin Rose
f089b2001f Add several math functions to the math module. 2020-04-04 12:52:34 -05:00
Calvin Rose
9f8420bf50 Add jpm repl subcommand and post-deps macro for jpm.
This will allow more flexibility in writing jpm project files.
2020-04-03 19:33:54 -05:00
Calvin Rose
8275da63fb Address #331 - Add :octal-permissions 2020-04-03 18:29:45 -05:00
Calvin Rose
72696600d8 Add :deps opiton to declare-executable.
This allows the addition of custom dependencies.
2020-04-03 17:53:41 -05:00
Calvin Rose
1aeb317863 Revise, revise, revise, and proofread. 2020-04-03 17:04:05 -05:00
Calvin Rose
b49b510732 Update os/link docstring. 2020-04-03 16:58:45 -05:00
Calvin Rose
a0d61e45d5 Change os/perm-str to os/perm-string. 2020-04-03 15:23:29 -05:00
Calvin Rose
95f1ef7561 Add umask support for windows, and allow parsing mode strings. 2020-04-03 15:14:11 -05:00
Calvin Rose
edb2fab64c Merge branch 'master' of github.com:janet-lang/janet 2020-04-03 15:04:39 -05:00
Calvin Rose
464fb73d83 Add os/perm-int and os/perm-str.
This helps address #331. While we could also
make os/stat return an integer, we don't do that yet
for api breakage reasons.

This also lets us use this logic on other functions
that take permission strings.
2020-04-03 15:02:12 -05:00
Calvin Rose
6a4e63a17d Merge pull request #333 from andrewchambers/umask
Add os/umask.
2020-04-03 14:48:52 -05:00
Calvin Rose
168f94d29a Merge pull request #330 from DEADB17/patch-1
Correct typo and match wording for consistency
2020-04-03 14:46:15 -05:00
Andrew Chambers
3c2b1baff2 Add os/umask. 2020-04-02 23:33:50 +13:00
DEADB17
57b08a57a0 Corret typo and match wording for consistency 2020-03-31 23:32:17 -04:00
Calvin Rose
65403ec9fe Merge branch 'master' into net 2020-03-07 14:06:51 -06:00
Calvin Rose
90b3730a0a Merge branch 'master' into net 2020-03-07 13:34:13 -06:00
Calvin Rose
16202216b2 Address #291
When resuming a fiber with a child, the root fiber was set incorrectly.
2020-03-05 19:18:45 -06:00
Calvin Rose
8f1527712e Merge branch 'master' into net 2020-03-05 18:08:35 -06:00
Calvin Rose
01a79dc965 Remove extra functionality. 2020-02-20 20:10:03 -06:00
Calvin Rose
0df220780a Fix issues with #282
Bad handling of write errors, as well as janet_root_fiber().
2020-02-20 19:54:31 -06:00
Calvin Rose
f4a46ba6ea Add methods to streams.
This makes streams polymorphic with files in many cases.
printf family functions still need porting.
2020-02-12 09:32:41 -06:00
Calvin Rose
79bb9e54d5 Remove direct references to file descriptors.
If a descriptor is freed by the Janet code, other
uses of that descriptor, say in the event loop, need
to know that it has been closed.
2020-02-11 08:57:44 -06:00
Calvin Rose
135aff9e17 Add janet_loop() call to static binaries. 2020-02-09 20:02:35 -06:00
Calvin Rose
8ae6ae65a1 Merge branch 'master' into net 2020-02-09 20:00:58 -06:00
Calvin Rose
f4d7fd97f6 Working TCP echo server and client.
Required a few changes to APIs, namely janet_root_fiber()
to get topmost fiber that is active in the current scheduler.
This is distinct from janet_current_fiber(), which gets the bottom
most fiber in the fiber stack - it might have a parent, and so cannot
be reliably resumed.
This is the kind of situation that makes symmetric coroutines more
attractive.
2020-02-09 20:00:50 -06:00
Calvin Rose
7f1f684b21 Merge branch 'master' into net 2020-02-03 20:46:32 -06:00
Calvin Rose
eda61455d3 Work on tcp server code. 2020-02-03 09:29:51 -06:00
Calvin Rose
c5907258c3 Merge branch 'master' into net 2020-02-02 13:16:47 -06:00
Calvin Rose
c0d2140d14 Begin net/ module in core.
Humble beginnings.
2020-02-01 20:39:54 -06:00
44 changed files with 2066 additions and 368 deletions

View File

@@ -1,6 +1,28 @@
# Changelog
All notable changes to this project will be documented in this file.
## Unreleased - ???
- Fix bug in `getline`.
- Add `sh-rule` and `sh-phony` to jpm's dialect of Janet.
- Change C api's `janet_formatb` -> `janet_formatbv`.
- Add C `janet_formatb` to C api.
- Add `edefer` macro to core.
- A struct/table literal/constructor with duplicate keys will use the last value given.
Previously, this was inconsistent between tables and structs, literals and constructor functions.
- Add debugger to core. The debugger functions are only available
in a debug repl, and are prefixed by a `.`.
- Add `sort-by` and `sorted-by` to core.
- Support UTF-8 escapes in strings via `\uXXXX` or `\UXXXXXX`.
- Add `math/erf`
- Add `math/erfc`
- Add `math/log1p`
- Add `math/next`
- Add os/umask
- Add os/perm-int
- Add os/perm-string
- Add :octal-permissions option for os/stat.
- Add `jpm repl` subcommand, as well as `post-deps` macro in project.janet files.
## 1.8.1 - 2020-03-31
- Fix bugs for big endian systems
- Fix 1.8.0 regression on BSDs

View File

@@ -36,6 +36,7 @@ JANET_PATH?=$(LIBDIR)/janet
MANPATH?=$(PREFIX)/share/man/man1/
PKG_CONFIG_PATH?=$(LIBDIR)/pkgconfig
DEBUGGER=gdb
SONAME_SETTER=-Wl,-soname,
CFLAGS:=$(CFLAGS) -std=c99 -Wall -Wextra -Isrc/include -Isrc/conf -fPIC -O2 -fvisibility=hidden
LDFLAGS:=$(LDFLAGS) -rdynamic
@@ -47,6 +48,7 @@ LDCONFIG:=ldconfig "$(LIBDIR)"
UNAME:=$(shell uname -s)
ifeq ($(UNAME), Darwin)
CLIBS:=$(CLIBS) -ldl
SONAME_SETTER:=-Wl,-install_name,
LDCONFIG:=true
else ifeq ($(UNAME), Linux)
CLIBS:=$(CLIBS) -lrt -ldl
@@ -94,6 +96,7 @@ JANET_CORE_SOURCES=src/core/abstract.c \
src/core/io.c \
src/core/marsh.c \
src/core/math.c \
src/core/net.c \
src/core/os.c \
src/core/parse.c \
src/core/peg.c \
@@ -146,6 +149,8 @@ build/janet.c: build/janet_boot src/boot/boot.janet
##### Amalgamation #####
########################
SONAME=libjanet.so.1.9
build/shell.c: src/mainclient/shell.c
cp $< $@
@@ -165,7 +170,7 @@ $(JANET_TARGET): build/janet.o build/shell.o
$(CC) $(LDFLAGS) $(CFLAGS) -o $@ $^ $(CLIBS)
$(JANET_LIBRARY): build/janet.o build/shell.o
$(CC) $(LDFLAGS) $(CFLAGS) -shared -o $@ $^ $(CLIBS)
$(CC) $(LDFLAGS) $(CFLAGS) $(SONAME_SETTER)$(SONAME) -shared -o $@ $^ $(CLIBS)
$(JANET_STATIC_LIBRARY): build/janet.o build/shell.o
$(AR) rcs $@ $^
@@ -228,8 +233,6 @@ build/doc.html: $(JANET_TARGET) tools/gendoc.janet
##### Installation #####
########################
SONAME=libjanet.so.1
.INTERMEDIATE: build/janet.pc
build/janet.pc: $(JANET_TARGET)
echo 'prefix=$(PREFIX)' > $@
@@ -242,7 +245,7 @@ build/janet.pc: $(JANET_TARGET)
echo "Description: Library for the Janet programming language." >> $@
$(JANET_TARGET) -e '(print "Version: " janet/version)' >> $@
echo 'Cflags: -I$${includedir}' >> $@
echo 'Libs: -L$${libdir} -ljanet $(LDFLAGS)' >> $@
echo 'Libs: -L$${libdir} -ljanet' >> $@
echo 'Libs.private: $(CLIBS)' >> $@
install: $(JANET_TARGET) build/janet.pc

View File

@@ -34,7 +34,7 @@
(defmacro rule
"Add a rule to the rule graph."
[target deps & body]
~(,rule-impl ,target ,deps (fn [] nil ,;body)))
~(,rule-impl ,target ,deps (fn [] ,;body)))
(defmacro phony
"Add a phony rule to the rule graph. A phony rule will run every time
@@ -43,6 +43,16 @@
[target deps & body]
~(,rule-impl ,target ,deps (fn [] nil ,;body) true))
(defmacro sh-rule
"Add a rule that invokes a shell command, and fails if the command returns non-zero."
[target deps & body]
~(,rule-impl ,target ,deps (fn [] (,assert (,zero? (,os/shell (,string ,;body)))))))
(defmacro sh-phony
"Add a phony rule that invokes a shell command, and fails if the command returns non-zero."
[target deps & body]
~(,rule-impl ,target ,deps (fn [] (,assert (,zero? (,os/shell (,string ,;body))))) true))
(defn add-dep
"Add a dependency to an existing rule. Useful for extending phony
rules or extending the dependency graph of existing rules."
@@ -177,26 +187,47 @@
[into x]
(when x
(proto-flatten into (table/getproto x))
(loop [k :keys x]
(put into k (x k))))
(merge-into into x))
into)
(defn import-rules
"Import another file that defines more rules. This ruleset
is merged into the current ruleset."
[path]
(defn make-jpm-env
"Build an environment table with jpm functions preloaded."
[&opt no-deps]
(def env (make-env))
(unless (os/stat path :mode)
(error (string "cannot open " path)))
(put env :jpm-no-deps no-deps)
(loop [k :keys _env :when (symbol? k)]
(unless ((_env k) :private) (put env k (_env k))))
env)
(defn require-jpm
"Require a jpm file project file. This is different from a normal require
in that code is loaded in the jpm environment."
[path &opt no-deps]
(unless (os/stat path :mode)
(error (string "cannot open " path)))
(def env (make-jpm-env no-deps))
(def currenv (proto-flatten @{} (fiber/getenv (fiber/current))))
(loop [k :keys currenv :when (keyword? k)]
(put env k (currenv k)))
(dofile path :env env :exit true)
env)
(defn import-rules
"Import another file that defines more rules. This ruleset
is merged into the current ruleset."
[path &opt no-deps]
(def env (require-jpm path no-deps))
(when-let [rules (env :rules)] (merge-into (getrules) rules))
env)
(defmacro post-deps
"Run code at the top level if jpm dependencies are installed. Build
code that imports dependencies should be wrapped with this macro, as project.janet
needs to be able to run successfully even without dependencies installed."
[& body]
(unless (dyn :jpm-no-deps)
~',(reduce |(eval $1) nil body)))
#
# OS and shell helpers
#
@@ -509,11 +540,14 @@ int main(int argc, const char **argv) {
fiber->env = temptab;
Janet out;
JanetSignal result = janet_continue(fiber, janet_wrap_nil(), &out);
if (result) {
if (result != JANET_SIGNAL_OK && result != JANET_SIGNAL_EVENT) {
janet_stacktrace(fiber, out);
janet_deinit();
return result;
}
#ifdef JANET_NET
janet_loop();
#endif
janet_deinit();
return 0;
}
@@ -699,7 +733,7 @@ int main(int argc, const char **argv) {
(def dep-urls (map |(if (string? $) $ ($ :repo)) d))
(unless (resolved r)
(when (all resolved dep-urls)
(array/push ordered-packages p)
(array/push ordered-packages {:repo r :sha s})
(set made-progress true)
(put resolved r true))))
(unless made-progress
@@ -799,13 +833,15 @@ int main(int argc, const char **argv) {
is marshalled into bytecode which is then embedded in a final executable for distribution.\n\n
This executable can be installed as well to the --binpath given."
[&keys {:install install :name name :entry entry :headers headers
:cflags cflags :lflags lflags}]
:cflags cflags :lflags lflags :deps deps}]
(def name (if is-win (string name ".exe") name))
(def dest (string "build" sep name))
(create-executable @{:cflags cflags :lflags lflags} entry dest)
(add-dep "build" dest)
(when headers
(each h headers (add-dep dest h)))
(when deps
(each d deps (add-dep dest d)))
(when install
(install-rule dest (dyn :binpath JANET_BINPATH))))
@@ -912,8 +948,8 @@ int main(int argc, const char **argv) {
'(* "--" '(some (if-not "=" 1)) (+ (* "=" '(any 1)) -1))))
(defn- local-rule
[rule]
(import-rules "./project.janet")
[rule &opt no-deps]
(import-rules "./project.janet" no-deps)
(do-rule rule))
(defn- help
@@ -948,6 +984,8 @@ Subcommands are:
name is lockfile.janet.
load-lockfile (lockfile) : Install modules from a lockfile in a reproducible way. The
default lockfile name is lockfile.janet.
repl : Run a repl in the context of the current project.janet file. This lets you run rules and
otherwise debug the current project.janet file.
Keys are:
--modpath : The directory to install modules to. Defaults to $JANET_MODPATH, $JANET_PATH, or (dyn :syspath)
@@ -961,15 +999,16 @@ Keys are:
--pkglist : URL of git repository for package listing. Defaults to $JANET_PKGLIST or https://github.com/janet-lang/pkgs.git
Flags are:
--nocolor : Disable color in the jpm repl.
--verbose : Print shell commands as they are executed.
--test : If passed to jpm install, runs tests before installing. Will run tests recursively on dependencies.
`))
(defn- show-help
(defn show-help
[]
(print help))
(defn- show-paths
(defn show-paths
[]
(print "binpath: " (dyn :binpath JANET_BINPATH))
(print "modpath: " (dyn :modpath JANET_MODPATH))
@@ -977,21 +1016,21 @@ Flags are:
(print "headerpath: " (dyn :headerpath JANET_HEADERPATH))
(print "syspath: " (dyn :syspath)))
(defn- build
(defn build
[]
(local-rule "build"))
(defn- clean
(defn clean
[]
(local-rule "clean"))
(defn- install
(defn install
[&opt repo]
(if repo
(install-git repo)
(local-rule "install")))
(defn- test
(defn test
[]
(local-rule "test"))
@@ -1001,25 +1040,46 @@ Flags are:
(uninstall what)
(local-rule "uninstall")))
(defn- deps
(defn deps
[]
(local-rule "install-deps"))
(local-rule "install-deps" true))
(defn- list-rules
[]
(import-rules "./project.janet")
(defn list-rules
[&opt ctx]
(import-rules "./project.janet" true)
(def ks (sort (seq [k :keys (dyn :rules)] k)))
(each k ks (print k)))
(defn- update-pkgs
(defn update-pkgs
[]
(install-git (dyn :pkglist default-pkglist)))
(defn- quickbin
(defn quickbin
[input output]
(create-executable @{} input output)
(do-rule output))
(defn jpm-repl
[]
(def env
(try
(require-jpm "./project.janet")
([err f]
(if (= "cannot open ./project.janet" err)
(put (make-jpm-env) :project {})
(propagate err f)))))
(setdyn :pretty-format (if-not (dyn :nocolor) "%.20Q" "%.20q"))
(setdyn :err-color (if-not (dyn :nocolor) true))
(def p (env :project))
(def name (p :name))
(if name (print "Project: " name))
(if-let [r (p :repo)] (print "Repository: " r))
(if-let [a (p :author)] (print "Author: " a))
(defn getchunk [buf p]
(def [line] (parser/where p))
(getline (string "jpm[" (or name "repl") "]:" line ":" (parser/state p :delimiters) "> ") buf env))
(repl getchunk nil env))
(def- subcommands
{"build" build
"clean" clean
@@ -1028,6 +1088,7 @@ Flags are:
"test" test
"help" help
"deps" deps
"repl" jpm-repl
"show-paths" show-paths
"clear-cache" clear-cache
"run" local-rule

View File

@@ -1,20 +1,18 @@
###
### A useful debugger library for Janet. Should be used
### inside a debug repl.
### inside a debug repl. This has been moved into the core.
###
(defn .fiber
"Get the current fiber being debugged."
[]
(if-let [entry (dyn '_fiber)]
(entry :value)
(dyn :fiber)))
(dyn :fiber))
(defn .stack
"Print the current fiber stack"
[]
(print)
(debug/stacktrace (.fiber) "")
(with-dyns [:err-color false] (debug/stacktrace (.fiber) ""))
(print))
(defn .frame

6
examples/tcpclient.janet Normal file
View File

@@ -0,0 +1,6 @@
(with [conn (net/connect "127.0.0.1" "8000")]
(printf "Connected to %q!" conn)
(:write conn "Echo...")
(print "Wrote to connection...")
(def res (:read conn 1024))
(pp res))

13
examples/tcpserver.janet Normal file
View File

@@ -0,0 +1,13 @@
(defn handler
"Simple handler for connections."
[stream]
(defer (:close stream)
(def id (gensym))
(def b @"")
(print "Connection " id "!")
(while (:read stream 1024 b)
(:write stream b)
(buffer/clear b))
(printf "Done %v!" id)))
(net/server "127.0.0.1" "8000" handler)

10
janet.1
View File

@@ -96,6 +96,10 @@ Delete everything before the cursor on the input line.
.BR Ctrl\-W
Delete one word before the cursor.
.TP 16
.BR Ctrl\-G
Show documentation for the current symbol under the cursor.
.TP 16
.BR Alt\-B/Alt\-F
Move cursor backwards and forwards one word.
@@ -148,6 +152,12 @@ Read raw input from stdin and forgo prompt history and other readline-like featu
Execute a string of Janet source. Source code is executed in the order it is encountered, so earlier
arguments are executed before later ones.
.TP
.BR \-d
Enable debug mode. On all terminating signals as well the debug signal, this will
cause the debugger to come up in the REPL. Same as calling (setdyn :debug true) in a
default repl.
.TP
.BR \-n
Disable ANSI colors in the repl. Has no effect if no repl is run.

9
jpm.1
View File

@@ -24,6 +24,10 @@ More interesting are the local commands. For more information on jpm usage, see
.SH FLAGS
.TP
.BR \-\-nocolor
Disable color in the jpm repl.
.TP
.BR \-\-verbose
Print detailed messages of what jpm is doing, including compilation commands and other shell commands.
@@ -154,6 +158,11 @@ The main function is the entry point of the program and will receive command lin
as function arguments. The entry file can import other modules, including native C modules, and
jpm will attempt to include the dependencies into the generated executable.
.TP
.BR repl
Load the current project.janet file and start a repl in it's environment. This lets a user better
debug the project file, as well as run rules manually.
.SH ENVIRONMENT
.B JANET_PATH

View File

@@ -20,7 +20,7 @@
project('janet', 'c',
default_options : ['c_std=c99', 'b_lundef=false', 'default_library=both'],
version : '1.8.1')
version : '1.9.0-dev')
# Global settings
janet_path = join_paths(get_option('prefix'), get_option('libdir'), 'janet')
@@ -59,6 +59,7 @@ conf.set('JANET_NO_DOCSTRINGS', not get_option('docstrings'))
conf.set('JANET_NO_SOURCEMAPS', not get_option('sourcemaps'))
conf.set('JANET_NO_ASSEMBLER', not get_option('assembler'))
conf.set('JANET_NO_PEG', not get_option('peg'))
conf.set('JANET_NO_NET', not get_option('net'))
conf.set('JANET_REDUCED_OS', get_option('reduced_os'))
conf.set('JANET_NO_TYPED_ARRAY', not get_option('typed_array'))
conf.set('JANET_NO_INT_TYPES', not get_option('int_types'))
@@ -112,6 +113,7 @@ core_src = [
'src/core/io.c',
'src/core/marsh.c',
'src/core/math.c',
'src/core/net.c',
'src/core/os.c',
'src/core/parse.c',
'src/core/peg.c',
@@ -167,6 +169,8 @@ janetc = custom_target('janetc',
libjanet = library('janet', janetc,
include_directories : incdir,
dependencies : [m_dep, dl_dep, thread_dep],
version: meson.project_version(),
soversion: version_parts[0] + '.' + version_parts[1],
install : true)
# Extra c flags - adding -fvisibility=hidden matches the Makefile and
@@ -218,6 +222,7 @@ test_files = [
'test/suite6.janet',
'test/suite7.janet',
'test/suite8.janet'
'test/suite9.janet'
]
foreach t : test_files
test(t, janet_nativeclient, args : files([t]), workdir : meson.current_source_dir())

View File

@@ -11,6 +11,7 @@ option('peg', type : 'boolean', value : true)
option('typed_array', type : 'boolean', value : true)
option('int_types', type : 'boolean', value : true)
option('prf', type : 'boolean', value : true)
option('net', type : 'boolean', value : true)
option('recursion_guard', type : 'integer', min : 10, max : 8000, value : 1024)
option('max_proto_depth', type : 'integer', min : 10, max : 8000, value : 200)

View File

@@ -32,7 +32,7 @@
(def buf (buffer "(" name))
(while (< index arglen)
(buffer/push-string buf " ")
(buffer/format buf "%p" (in args index))
(buffer/format buf "%j" (in args index))
(set index (+ index 1)))
(array/push modifiers (string buf ")\n\n" docstr))
# Build return value
@@ -301,7 +301,19 @@
,form
(if (= (,fiber/status ,f) :dead)
,r
(propagate ,r ,f)))))
(,propagate ,r ,f)))))
(defmacro edefer
"Run form after body in the case that body terminates abnormally (an error or user signal 0-4).
Otherwise, return last form in body."
[form & body]
(with-syms [f r]
~(do
(def ,f (,fiber/new (fn [] ,;body) :ti))
(def ,r (,resume ,f))
(if (= (,fiber/status ,f) :dead)
,r
(do ,form (,propagate ,r ,f))))))
(defmacro prompt
"Set up a checkpoint that can be returned to. Tag should be a value
@@ -314,7 +326,7 @@
(def [,target ,payload] ,res)
(if (,= ,tag ,target)
,payload
(propagate ,res ,fib)))))
(,propagate ,res ,fib)))))
(defmacro chr
"Convert a string of length 1 to its byte (ascii) value at compile time."
@@ -467,7 +479,7 @@
(for-template i start stop 1 < + body))
(defmacro eachk
"loop over each key in ds. returns nil."
"Loop over each key in ds. Returns nil."
[x ds & body]
(keys-template x ds false body))
@@ -489,7 +501,7 @@
that define something to loop over. They are formatted like:\n\n
\tbinding :verb object/expression\n\n
Where binding is a binding as passed to def, :verb is one of a set of keywords,
and object is any janet expression. The available verbs are:\n\n
and object is any expression. The available verbs are:\n\n
\t:iterate - repeatedly evaluate and bind to the expression while it is truthy.\n
\t:range - loop over a range. The object should be two element tuple with a start
and end value, and an optional positive step. The range is half open, [start, end).\n
@@ -648,7 +660,7 @@
(defn last
"Get the last element from an indexed data structure."
[xs]
(in xs (- (length xs) 1)))
(get xs (- (length xs) 1)))
###
###
@@ -656,41 +668,54 @@
###
###
(def sort
"(sort xs [, by])\n\nSort an array in-place. Uses quick-sort and is not a stable sort."
(do
(defn- sort-part
[a lo hi by]
(def pivot (in a hi))
(var i lo)
(for j lo hi
(def aj (in a j))
(when (by aj pivot)
(def ai (in a i))
(set (a i) aj)
(set (a j) ai)
(++ i)))
(set (a hi) (in a i))
(set (a i) pivot)
i)
(defn part
[a lo hi by]
(def pivot (in a hi))
(var i lo)
(for j lo hi
(def aj (in a j))
(when (by aj pivot)
(def ai (in a i))
(set (a i) aj)
(set (a j) ai)
(++ i)))
(set (a hi) (in a i))
(set (a i) pivot)
i)
(defn- sort-help
[a lo hi by]
(when (> hi lo)
(def piv (sort-part a lo hi by))
(sort-help a lo (- piv 1) by)
(sort-help a (+ piv 1) hi by))
a)
(defn sort-help
[a lo hi by]
(when (> hi lo)
(def piv (part a lo hi by))
(sort-help a lo (- piv 1) by)
(sort-help a (+ piv 1) hi by))
a)
(defn sort
"Sort an array in-place. Uses quick-sort and is not a stable sort."
[a &opt by]
(sort-help a 0 (- (length a) 1) (or by <)))
(fn sort [a &opt by]
(sort-help a 0 (- (length a) 1) (or by <)))))
(put _env 'sort-part nil)
(put _env 'sort-help nil)
(defn sort-by
"Returns a new sorted array that compares elements by invoking
a function on each element and comparing the result with <."
[f ind]
(sort ind (fn [x y] (< (f x) (f y)))))
(defn sorted
"Returns a new sorted array without modifying the old one."
[ind &opt by]
(sort (array/slice ind) by))
(defn sorted-by
"Returns a new sorted array that compares elements by invoking
a function on each element and comparing the result with <."
[f ind]
(sorted ind (fn [x y] (< (f x) (f y)))))
(defn reduce
"Reduce, also know as fold-left in many languages, transforms
an indexed type (array, tuple) with a function to produce a value."
@@ -942,7 +967,7 @@
(reduce fop x forms))
(defmacro -?>
"Short circuit threading macro. Inserts x as the last value in the first form
"Short circuit threading macro. Inserts x as the second value in the first form
in forms, and inserts the modified first form into the second form
in the same manner, and so on. The pipeline will return nil
if an intermediate value is nil.
@@ -958,7 +983,7 @@
(reduce fop x forms))
(defmacro -?>>
"Threading macro. Inserts x as the last value in the first form
"Short circuit threading macro. Inserts x as the last value in the first form
in forms, and inserts the modified first form into the second form
in the same manner, and so on. The pipeline will return nil
if an intermediate value is nil.
@@ -1457,10 +1482,10 @@
###
(defn- env-walk
[pred &opt env]
[pred &opt env local]
(default env (fiber/getenv (fiber/current)))
(def envs @[])
(do (var e env) (while e (array/push envs e) (set e (table/getproto e))))
(do (var e env) (while e (array/push envs e) (set e (table/getproto e)) (if local (break))))
(def ret-set @{})
(loop [envi :in envs
k :keys envi
@@ -1469,22 +1494,24 @@
(sort (keys ret-set)))
(defn all-bindings
"Get all symbols available in an enviroment. Defaults to the current
fiber's environment."
[&opt env]
(env-walk symbol? env))
"Get all symbols available in an environment. Defaults to the current
fiber's environment. If local is truthy, will not show inherited bindings
(from prototype tables)."
[&opt env local]
(env-walk symbol? env local))
(defn all-dynamics
"Get all dynamic bindings in an environment. Defaults to the current
fiber's environment."
[&opt env]
(env-walk keyword? env))
fiber's environment. If local is truthy, will not show inherited bindings
(from prototype tables)."
[&opt env local]
(env-walk keyword? env local))
(defn doc-format
"Reformat text to wrap at a given line."
[text]
[text &opt width]
(def maxcol (- (dyn :doc-width 80) 8))
(def maxcol (- (or width (dyn :doc-width 80)) 8))
(var buf @" ")
(var word @"")
(var current 0)
@@ -1676,7 +1703,7 @@
ret)
(defn all
"Returns true if all xs are truthy, otherwise the resulty of first
"Returns true if all xs are truthy, otherwise the result of first
falsey predicate value, (pred x)."
[pred xs]
(var ret true)
@@ -1890,8 +1917,9 @@
(eflush))
(defn run-context
"Run a context. This evaluates expressions of janet in an environment,
"Run a context. This evaluates expressions in an environment,
and is encapsulates the parsing, compilation, and evaluation.
Returns (in environment :exit-value environment) when complete.
opts is a table or struct of options. The options are as follows:\n\n\t
:chunks - callback to read into a buffer - default is getline\n\t
:on-parse-error - callback when parsing fails - default is bad-parse\n\t
@@ -1920,6 +1948,7 @@
(default on-parse-error bad-parse)
(default evaluator (fn evaluate [x &] (x)))
(default where "<anonymous>")
(default guard :ydt)
# Are we done yet?
(var going true)
@@ -1946,7 +1975,7 @@
(string err " on line " line ", column " column)
err))
(on-compile-error msg errf where))))
(or guard :a)))
guard))
(fiber/setenv f env)
(while (fiber/can-resume? f)
(def res (resume f resumeval))
@@ -2149,13 +2178,15 @@
@{})
(defn dofile
"Evaluate a file and return the resulting environment."
[path & args]
(def {:exit exit-on-error
:source source
:env env
:expander expander
:evaluator evaluator} (table ;args))
"Evaluate a file and return the resulting environment. :env, :expander, and
:evaluator are passed through to the underlying run-context call.
If exit is true, any top level errors will trigger a call to (os/exit 1)
after printing the error."
[path &keys
{:exit exit
:env env
:expander expander
:evaluator evaluator}]
(def f (if (= (type path) :core/file)
path
(file/open path :rb)))
@@ -2167,11 +2198,11 @@
(defn chunks [buf _] (file/read f 2048 buf))
(defn bp [&opt x y]
(def ret (bad-parse x y))
(if exit-on-error (os/exit 1))
(if exit (os/exit 1))
ret)
(defn bc [&opt x y z]
(def ret (bad-compile x y z))
(if exit-on-error (os/exit 1))
(if exit (os/exit 1))
ret)
(unless f
(error (string "could not find file " path)))
@@ -2183,7 +2214,7 @@
:on-status (fn [f x]
(when (not= (fiber/status f) :dead)
(debug/stacktrace f x)
(if exit-on-error (os/exit 1) (eflush))))
(if exit (os/exit 1) (eflush))))
:evaluator evaluator
:expander expander
:source (if path-is-file "<anonymous>" spath)}))
@@ -2244,18 +2275,171 @@
any errors encountered at the top level in the module will cause (os/exit 1)
to be called. Dynamic bindings will NOT be imported."
[path & args]
(def argm (map (fn [x]
(if (keyword? x)
x
(string x)))
args))
(def argm (map |(if (keyword? $) $ (string $)) args))
(tuple import* (string path) ;argm))
(defmacro use
"Similar to import, but imported bindings are not prefixed with a namespace
identifier. Can also import multiple modules in one shot."
[& modules]
~(do ,;(map (fn [x] ~(,import* ,(string x) :prefix "")) modules)))
~(do ,;(map |~(,import* ,(string $) :prefix "") modules)))
###
###
### Debugger
###
###
(defn .fiber
"Get the current fiber being debugged."
[]
(dyn :fiber))
(defn .signal
"Get the current signal being debugged."
[]
(dyn :signal))
(defn .stack
"Print the current fiber stack"
[]
(print)
(with-dyns [:err-color false] (debug/stacktrace (.fiber) (.signal)))
(print))
(defn .frame
"Show a stack frame"
[&opt n]
(def stack (debug/stack (.fiber)))
(in stack (or n 0)))
(defn .fn
"Get the current function"
[&opt n]
(in (.frame n) :function))
(defn .slots
"Get an array of slots in a stack frame"
[&opt n]
(in (.frame n) :slots))
(defn .slot
"Get the value of the nth slot."
[&opt nth frame-idx]
(in (.slots frame-idx) (or nth 0)))
(defn .disasm
"Gets the assembly for the current function."
[&opt n]
(def frame (.frame n))
(def func (frame :function))
(disasm func))
(defn .bytecode
"Get the bytecode for the current function."
[&opt n]
((.disasm n) 'bytecode))
(defn .ppasm
"Pretty prints the assembly for the current function"
[&opt n]
(def frame (.frame n))
(def func (frame :function))
(def dasm (disasm func))
(def bytecode (dasm 'bytecode))
(def pc (frame :pc))
(def sourcemap (dasm 'sourcemap))
(var last-loc [-2 -2])
(print "\n signal: " (.signal))
(print " function: " (dasm 'name) " [" (in dasm 'source "") "]")
(when-let [constants (dasm 'constants)]
(printf " constants: %.4q" constants))
(printf " slots: %.4q\n" (frame :slots))
(def padding (string/repeat " " 20))
(loop [i :range [0 (length bytecode)]
:let [instr (bytecode i)]]
(prin (if (= (tuple/type instr) :brackets) "*" " "))
(prin (if (= i pc) "> " " "))
(prinf "%.20s" (string (string/join (map string instr) " ") padding))
(when sourcemap
(let [[sl sc] (sourcemap i)
loc [sl sc]]
(when (not= loc last-loc)
(set last-loc loc)
(prin " # line " sl ", column " sc))))
(print))
(print))
(defn .source
"Show the source code for the function being debugged."
[&opt n]
(def frame (.frame n))
(def s (frame :source))
(def all-source (slurp s))
(print "\n" all-source "\n"))
(defn .breakall
"Set breakpoints on all instructions in the current function."
[&opt n]
(def fun (.fn n))
(def bytecode (.bytecode n))
(for i 0 (length bytecode)
(debug/fbreak fun i))
(print "Set " (length bytecode) " breakpoints in " fun))
(defn .clearall
"Clear all breakpoints on the current function."
[&opt n]
(def fun (.fn n))
(def bytecode (.bytecode n))
(for i 0 (length bytecode)
(debug/unfbreak fun i))
(print "Cleared " (length bytecode) " breakpoints in " fun))
(defn .break
"Set breakpoint at the current pc."
[]
(def frame (.frame))
(def fun (frame :function))
(def pc (frame :pc))
(debug/fbreak fun pc)
(print "Set breakpoint in " fun " at pc=" pc))
(defn .clear
"Clear the current breakpoint"
[]
(def frame (.frame))
(def fun (frame :function))
(def pc (frame :pc))
(debug/unfbreak fun pc)
(print "Cleared breakpoint in " fun " at pc=" pc))
(defn .next
"Go to the next breakpoint."
[&opt n]
(var res nil)
(for i 0 (or n 1)
(set res (resume (.fiber))))
res)
(defn .nextc
"Go to the next breakpoint, clearing the current breakpoint."
[&opt n]
(.clear)
(.next n))
(defn .step
"Execute the next n instructions."
[&opt n]
(var res nil)
(for i 0 (or n 1)
(set res (debug/step (.fiber))))
res)
(def- debugger-keys (filter (partial string/has-prefix? ".") (keys _env)))
(def- debugger-env @{})
(each k debugger-keys (put debugger-env k (_env k)) (put _env k nil))
(put _env 'debugger-keys nil)
###
###
@@ -2271,11 +2455,15 @@
the repl in."
[&opt chunks onsignal env]
(default env (make-env))
(default chunks (fn [buf p] (getline (string "repl:"
((parser/where p) 0)
":"
(parser/state p :delimiters) "> ")
buf env)))
(default chunks
(fn [buf p]
(getline
(string
"repl:"
((parser/where p) 0)
":"
(parser/state p :delimiters) "> ")
buf env)))
(defn make-onsignal
[e level]
@@ -2285,13 +2473,14 @@
(put nextenv :fiber f)
(put nextenv :debug-level level)
(put nextenv :signal x)
(merge-into nextenv debugger-env)
(debug/stacktrace f x)
(eflush)
(defn debugger-chunks [buf p]
(def status (parser/state p :delimiters))
(def c ((parser/where p) 0))
(def prompt (string "debug[" level "]:" c ":" status "> "))
(getline prompt buf nextenv))
(def prpt (string "debug[" level "]:" c ":" status "> "))
(getline prpt buf nextenv))
(print "entering debug[" level "] - (quit) to exit")
(flush)
(repl debugger-chunks (make-onsignal nextenv (+ 1 level)) nextenv)
@@ -2311,6 +2500,8 @@
:on-status (or onsignal (make-onsignal env 1))
:source "repl"}))
(put _env 'debugger-env nil)
###
###
### CLI Tool Main
@@ -2354,6 +2545,7 @@
(var *handleopts* true)
(var *exit-on-error* true)
(var *colorize* true)
(var *debug* false)
(var *compile-only* false)
(if-let [jp (os/getenv "JANET_PATH")] (setdyn :syspath jp))
@@ -2369,6 +2561,7 @@
-v : Print the version string
-s : Use raw stdin instead of getline like functionality
-e code : Execute a string of janet
-d : Set the debug flag in the repl
-r : Enter the repl after running all scripts
-p : Keep on executing if there is a top level error (persistent)
-q : Hide prompt, logo, and repl output (quiet)
@@ -2401,7 +2594,8 @@
"e" (fn [i &]
(set *no-file* false)
(eval-string (in args (+ i 1)))
2)})
2)
"d" (fn [&] (set *debug* true) 1)})
(defn- dohandler [n i &]
(def h (in handlers n))
@@ -2460,6 +2654,7 @@
(file/flush stdout)
(file/read stdin :line buf))
(def env (make-env))
(if *debug* (put env :debug true))
(def getter (if *raw-stdin* getstdin getline))
(defn getchunk [buf p]
(getter (prompter p) buf env))
@@ -2480,7 +2675,7 @@
###
###
(def root-env "The root environment used to create envionments with (make-env)" _env)
(def root-env "The root environment used to create environments with (make-env)" _env)
(do
(put _env 'boot/opts nil)
@@ -2559,6 +2754,7 @@
"src/core/io.c"
"src/core/marsh.c"
"src/core/math.c"
"src/core/net.c"
"src/core/os.c"
"src/core/parse.c"
"src/core/peg.c"

View File

@@ -27,10 +27,10 @@
#define JANETCONF_H
#define JANET_VERSION_MAJOR 1
#define JANET_VERSION_MINOR 8
#define JANET_VERSION_PATCH 1
#define JANET_VERSION_MINOR 9
#define JANET_VERSION_PATCH 0
#define JANET_VERSION_EXTRA ""
#define JANET_VERSION "1.8.1"
#define JANET_VERSION "1.9.0-dev"
/* #define JANET_BUILD "local" */
@@ -49,6 +49,7 @@
/* Other settings */
/* #define JANET_NO_ASSEMBLER */
/* #define JANET_NO_PEG */
/* #define JANET_NO_NET */
/* #define JANET_NO_TYPED_ARRAY */
/* #define JANET_NO_INT_TYPES */
/* #define JANET_NO_PRF */

View File

@@ -707,6 +707,9 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
if (janet_indexed_view(x, &arr, &count)) {
janet_asm_assert(&a, count == def->bytecode_length, "sourcemap must have the same length as the bytecode");
def->sourcemap = malloc(sizeof(JanetSourceMapping) * (size_t) count);
if (NULL == def->sourcemap) {
JANET_OUT_OF_MEMORY;
}
for (i = 0; i < count; i++) {
const Janet *tup;
Janet entry = arr[i];
@@ -730,6 +733,9 @@ static JanetAssembleResult janet_asm1(JanetAssembler *parent, Janet source, int
/* Set environments */
def->environments =
realloc(def->environments, def->environments_length * sizeof(int32_t));
if (NULL == def->environments) {
JANET_OUT_OF_MEMORY;
}
/* Verify the func def */
if (janet_verify(def)) {

View File

@@ -54,7 +54,7 @@ void janet_panicf(const char *format, ...) {
while (format[len]) len++;
janet_buffer_init(&buffer, len);
va_start(args, format);
janet_formatb(&buffer, format, args);
janet_formatbv(&buffer, format, args);
va_end(args);
ret = janet_string(buffer.data, buffer.count);
janet_buffer_deinit(&buffer);
@@ -235,18 +235,20 @@ size_t janet_getsize(const Janet *argv, int32_t n) {
int32_t janet_gethalfrange(const Janet *argv, int32_t n, int32_t length, const char *which) {
int32_t raw = janet_getinteger(argv, n);
if (raw < 0) raw += length + 1;
if (raw < 0 || raw > length)
janet_panicf("%s index %d out of range [0,%d]", which, raw, length);
return raw;
int32_t not_raw = raw;
if (not_raw < 0) not_raw += length + 1;
if (not_raw < 0 || not_raw > length)
janet_panicf("%s index %d out of range [%d,%d]", which, raw, -length - 1, length);
return not_raw;
}
int32_t janet_getargindex(const Janet *argv, int32_t n, int32_t length, const char *which) {
int32_t raw = janet_getinteger(argv, n);
if (raw < 0) raw += length;
if (raw < 0 || raw > length)
janet_panicf("%s index %d out of range [0,%d)", which, raw, length);
return raw;
int32_t not_raw = raw;
if (not_raw < 0) not_raw += length;
if (not_raw < 0 || not_raw > length)
janet_panicf("%s index %d out of range [%d,%d)", which, raw, -length, length);
return not_raw;
}
JanetView janet_getindexed(const Janet *argv, int32_t n) {

View File

@@ -435,7 +435,7 @@ static Janet janet_core_hash(int32_t argc, Janet *argv) {
static Janet janet_core_getline(int32_t argc, Janet *argv) {
FILE *in = janet_dynfile("in", stdin);
FILE *out = janet_dynfile("out", stdout);
janet_arity(argc, 0, 2);
janet_arity(argc, 0, 3);
JanetBuffer *buf = (argc >= 2) ? janet_getbuffer(argv, 1) : janet_buffer(10);
if (argc >= 1) {
const char *prompt = (const char *) janet_getstring(argv, 0);
@@ -646,7 +646,7 @@ static const JanetReg corelib_cfuns[] = {
"getline", janet_core_getline,
JDOC("(getline &opt prompt buf env)\n\n"
"Reads a line of input into a buffer, including the newline character, using a prompt. "
"An optional environment table can be provided for autocomplete. "
"An optional environment table can be provided for auto-complete. "
"Returns the modified buffer. "
"Use this function to implement a simple interface for a terminal program.")
},
@@ -680,7 +680,7 @@ static const JanetReg corelib_cfuns[] = {
"\t:all:\tthe value of path verbatim\n"
"\t:cur:\tthe current file, or (dyn :current-file)\n"
"\t:dir:\tthe directory containing the current file\n"
"\t:name:\tthe filename component of path, with extenion if given\n"
"\t:name:\tthe filename component of path, with extension if given\n"
"\t:native:\tthe extension used to load natives, .so or .dll\n"
"\t:sys:\tthe system path, or (syn :syspath)")
},
@@ -697,7 +697,7 @@ static const JanetReg corelib_cfuns[] = {
{
"slice", janet_core_slice,
JDOC("(slice x &opt start end)\n\n"
"Extract a sub-range of an indexed data strutrue or byte sequence.")
"Extract a sub-range of an indexed data structure or byte sequence.")
},
{
"signal", janet_core_signal,
@@ -999,6 +999,9 @@ static void janet_load_libs(JanetTable *env) {
#ifdef JANET_THREADS
janet_lib_thread(env);
#endif
#ifdef JANET_NET
janet_lib_net(env);
#endif
}
#ifdef JANET_BOOTSTRAP

View File

@@ -29,8 +29,12 @@
#define _POSIX_C_SOURCE 200112L
#endif
#if defined(WIN32) || defined(_WIN32)
#define WIN32_LEAN_AND_MEAN
#endif
/* Needed for realpath on linux */
#if !defined(_XOPEN_SOURCE) && defined(__linux__)
#if !defined(_XOPEN_SOURCE) && (defined(__linux__) || defined(__EMSCRIPTEN__))
#define _XOPEN_SOURCE 500
#endif

View File

@@ -218,6 +218,7 @@ int janet_fiber_funcframe(JanetFiber *fiber, JanetFunction *func) {
static void janet_env_detach(JanetFuncEnv *env) {
/* Check for closure environment */
if (env) {
janet_env_valid(env);
int32_t len = env->length;
size_t s = sizeof(Janet) * (size_t) len;
Janet *vmem = malloc(s);
@@ -244,10 +245,38 @@ static void janet_env_detach(JanetFuncEnv *env) {
}
}
/* Validate potentially untrusted func env (unmarshalled envs are difficult to verify) */
int janet_env_valid(JanetFuncEnv *env) {
if (env->offset < 0) {
int32_t real_offset = -(env->offset);
JanetFiber *fiber = env->as.fiber;
int32_t i = fiber->frame;
while (i > 0) {
JanetStackFrame *frame = (JanetStackFrame *)(fiber->data + i - JANET_FRAME_SIZE);
if (real_offset == i &&
frame->env == env &&
frame->func &&
frame->func->def->slotcount == env->length) {
env->offset = real_offset;
return 1;
}
i = frame->prevframe;
}
/* Invalid, set to empty off-stack variant. */
env->offset = 0;
env->length = 0;
env->as.values = NULL;
return 0;
} else {
return 1;
}
}
/* Detach a fiber from the env if the target fiber has stopped mutating */
void janet_env_maybe_detach(JanetFuncEnv *env) {
/* Check for detachable closure envs */
if (env->offset) {
janet_env_valid(env);
if (env->offset > 0) {
JanetFiberStatus s = janet_fiber_status(env->as.fiber);
int isFinished = s == JANET_STATUS_DEAD ||
s == JANET_STATUS_ERROR ||
@@ -376,6 +405,10 @@ JanetFiber *janet_current_fiber(void) {
return janet_vm_fiber;
}
JanetFiber *janet_root_fiber(void) {
return janet_vm_root_fiber;
}
/* CFuns */
static Janet cfun_fiber_getenv(int32_t argc, Janet *argv) {

View File

@@ -74,5 +74,6 @@ int janet_fiber_funcframe_tail(JanetFiber *fiber, JanetFunction *func);
void janet_fiber_cframe(JanetFiber *fiber, JanetCFunction cfun);
void janet_fiber_popframe(JanetFiber *fiber);
void janet_env_maybe_detach(JanetFuncEnv *env);
int janet_env_valid(JanetFuncEnv *env);
#endif

View File

@@ -193,7 +193,7 @@ static void janet_mark_funcenv(JanetFuncEnv *env) {
/* If closure env references a dead fiber, we can just copy out the stack frame we need so
* we don't need to keep around the whole dead fiber. */
janet_env_maybe_detach(env);
if (env->offset) {
if (env->offset > 0) {
/* On stack */
janet_mark_fiber(env->as.fiber);
} else {
@@ -389,6 +389,9 @@ void janet_collect(void) {
if (janet_vm_gc_suspend) return;
depth = JANET_RECURSION_GUARD;
orig_rootcount = janet_vm_root_count;
#ifdef JANET_NET
janet_net_markloop();
#endif
for (i = 0; i < orig_rootcount; i++)
janet_mark(janet_vm_roots[i]);
while (orig_rootcount < janet_vm_root_count) {

View File

@@ -502,7 +502,7 @@ void janet_dynprintf(const char *name, FILE *dflt_file, const char *format, ...)
int32_t len = 0;
while (format[len]) len++;
janet_buffer_init(&buffer, len);
janet_formatb(&buffer, format, args);
janet_formatbv(&buffer, format, args);
if (xtype == JANET_ABSTRACT) {
void *abstract = janet_unwrap_abstract(x);
if (janet_abstract_type(abstract) != &janet_file_type)
@@ -515,7 +515,7 @@ void janet_dynprintf(const char *name, FILE *dflt_file, const char *format, ...)
break;
}
case JANET_BUFFER:
janet_formatb(janet_unwrap_buffer(x), format, args);
janet_formatbv(janet_unwrap_buffer(x), format, args);
break;
}
va_end(args);

View File

@@ -42,26 +42,26 @@ typedef struct {
/* Lead bytes in marshaling protocol */
enum {
LB_REAL = 200,
LB_NIL,
LB_FALSE,
LB_TRUE,
LB_FIBER,
LB_INTEGER,
LB_STRING,
LB_SYMBOL,
LB_KEYWORD,
LB_ARRAY,
LB_TUPLE,
LB_TABLE,
LB_TABLE_PROTO,
LB_STRUCT,
LB_BUFFER,
LB_FUNCTION,
LB_REGISTRY,
LB_ABSTRACT,
LB_REFERENCE,
LB_FUNCENV_REF,
LB_FUNCDEF_REF
LB_NIL, /* 201 */
LB_FALSE, /* 202 */
LB_TRUE, /* 203 */
LB_FIBER, /* 204 */
LB_INTEGER, /* 205 */
LB_STRING, /* 206 */
LB_SYMBOL, /* 207 */
LB_KEYWORD, /* 208 */
LB_ARRAY, /* 209 */
LB_TUPLE, /* 210 */
LB_TABLE, /* 211 */
LB_TABLE_PROTO, /* 212 */
LB_STRUCT, /* 213 */
LB_BUFFER, /* 214 */
LB_FUNCTION, /* 215 */
LB_REGISTRY, /* 216 */
LB_ABSTRACT, /* 217 */
LB_REFERENCE, /* 218 */
LB_FUNCENV_REF, /* 219 */
LB_FUNCDEF_REF /* 220 */
} LeadBytes;
/* Helper to look inside an entry in an environment */
@@ -183,8 +183,9 @@ static void marshal_one_env(MarshalState *st, JanetFuncEnv *env, int flags) {
return;
}
}
janet_env_valid(env);
janet_v_push(st->seen_envs, env);
if (env->offset && (JANET_STATUS_ALIVE == janet_fiber_status(env->as.fiber))) {
if (env->offset > 0 && (JANET_STATUS_ALIVE == janet_fiber_status(env->as.fiber))) {
pushint(st, 0);
pushint(st, env->length);
Janet *values = env->as.fiber->data + env->offset;
@@ -200,7 +201,7 @@ static void marshal_one_env(MarshalState *st, JanetFuncEnv *env, int flags) {
janet_env_maybe_detach(env);
pushint(st, env->offset);
pushint(st, env->length);
if (env->offset) {
if (env->offset > 0) {
/* On stack variant */
marshal_one(st, janet_wrap_fiber(env->as.fiber), flags + 1);
} else {
@@ -634,6 +635,15 @@ static int32_t readint(UnmarshalState *st, const uint8_t **atdata) {
return ret;
}
/* Helper to read a natural number (int >= 0). */
static int32_t readnat(UnmarshalState *st, const uint8_t **atdata) {
int32_t ret = readint(st, atdata);
if (ret < 0) {
janet_panicf("expected integer >= 0, got %d", ret);
}
return ret;
}
/* Helper to read a size_t (up to 8 bytes unsigned). */
static uint64_t read64(UnmarshalState *st, const uint8_t **atdata) {
uint64_t ret;
@@ -702,30 +712,31 @@ static const uint8_t *unmarshal_one_env(
JanetFuncEnv *env = janet_gcalloc(JANET_MEMORY_FUNCENV, sizeof(JanetFuncEnv));
env->length = 0;
env->offset = 0;
env->as.values = NULL;
janet_v_push(st->lookup_envs, env);
int32_t offset = readint(st, &data);
int32_t length = readint(st, &data);
if (offset) {
int32_t offset = readnat(st, &data);
int32_t length = readnat(st, &data);
if (offset > 0) {
Janet fiberv;
/* On stack variant */
data = unmarshal_one(st, data, &fiberv, flags);
janet_asserttype(fiberv, JANET_FIBER);
env->as.fiber = janet_unwrap_fiber(fiberv);
/* Unmarshalling fiber may set values */
if (env->offset != 0 && env->offset != offset)
janet_panic("invalid funcenv offset");
if (env->length != 0 && env->length != length)
janet_panic("invalid funcenv length");
/* Negative offset indicates untrusted input */
env->offset = -offset;
} else {
/* Off stack variant */
if (length == 0) {
janet_panic("invalid funcenv length");
}
env->as.values = malloc(sizeof(Janet) * (size_t) length);
if (!env->as.values) {
JANET_OUT_OF_MEMORY;
}
env->offset = 0;
for (int32_t i = 0; i < length; i++)
data = unmarshal_one(st, data, env->as.values + i, flags);
}
env->offset = offset;
env->length = length;
*out = env;
}
@@ -770,6 +781,11 @@ static const uint8_t *unmarshal_one_def(
def->name = NULL;
def->source = NULL;
def->closure_bitset = NULL;
def->defs = NULL;
def->environments = NULL;
def->constants = NULL;
def->bytecode = NULL;
def->sourcemap = NULL;
janet_v_push(st->lookup_defs, def);
/* Set default lengths to zero */
@@ -780,18 +796,18 @@ static const uint8_t *unmarshal_one_def(
/* Read flags and other fixed values */
def->flags = readint(st, &data);
def->slotcount = readint(st, &data);
def->arity = readint(st, &data);
def->min_arity = readint(st, &data);
def->max_arity = readint(st, &data);
def->slotcount = readnat(st, &data);
def->arity = readnat(st, &data);
def->min_arity = readnat(st, &data);
def->max_arity = readnat(st, &data);
/* Read some lengths */
constants_length = readint(st, &data);
bytecode_length = readint(st, &data);
constants_length = readnat(st, &data);
bytecode_length = readnat(st, &data);
if (def->flags & JANET_FUNCDEF_FLAG_HASENVS)
environments_length = readint(st, &data);
environments_length = readnat(st, &data);
if (def->flags & JANET_FUNCDEF_FLAG_HASDEFS)
defs_length = readint(st, &data);
defs_length = readnat(st, &data);
/* Check name and source (optional) */
if (def->flags & JANET_FUNCDEF_FLAG_HASNAME) {
@@ -866,7 +882,7 @@ static const uint8_t *unmarshal_one_def(
for (int32_t i = 0; i < bytecode_length; i++) {
current += readint(st, &data);
def->sourcemap[i].line = current;
def->sourcemap[i].column = readint(st, &data);
def->sourcemap[i].column = readnat(st, &data);
}
} else {
def->sourcemap = NULL;
@@ -898,7 +914,7 @@ static const uint8_t *unmarshal_one_fiber(
JanetFiber **out,
int flags) {
/* Initialize a new fiber */
/* Initialize a new fiber with gc friendly defaults */
JanetFiber *fiber = janet_gcalloc(JANET_MEMORY_FIBER, sizeof(JanetFiber));
fiber->flags = 0;
fiber->frame = 0;
@@ -913,42 +929,41 @@ static const uint8_t *unmarshal_one_fiber(
/* Push fiber to seen stack */
janet_v_push(st->lookup, janet_wrap_fiber(fiber));
/* Set frame later so fiber can be GCed at anytime if unmarshalling fails */
int32_t frame = 0;
int32_t stack = 0;
int32_t stacktop = 0;
/* Read ints */
fiber->flags = readint(st, &data);
frame = readint(st, &data);
fiber->stackstart = readint(st, &data);
fiber->stacktop = readint(st, &data);
fiber->maxstack = readint(st, &data);
int32_t fiber_flags = readint(st, &data);
int32_t frame = readnat(st, &data);
int32_t fiber_stackstart = readnat(st, &data);
int32_t fiber_stacktop = readnat(st, &data);
int32_t fiber_maxstack = readnat(st, &data);
JanetTable *fiber_env = NULL;
/* Check for bad flags and ints */
if ((int32_t)(frame + JANET_FRAME_SIZE) > fiber->stackstart ||
fiber->stackstart > fiber->stacktop ||
fiber->stacktop > fiber->maxstack) {
if ((int32_t)(frame + JANET_FRAME_SIZE) > fiber_stackstart ||
fiber_stackstart > fiber_stacktop ||
fiber_stacktop > fiber_maxstack) {
janet_panic("fiber has incorrect stack setup");
}
/* Allocate stack memory */
fiber->capacity = fiber->stacktop + 10;
fiber->capacity = fiber_stacktop + 10;
fiber->data = malloc(sizeof(Janet) * fiber->capacity);
if (!fiber->data) {
JANET_OUT_OF_MEMORY;
}
for (int32_t i = 0; i < fiber->capacity; i++) {
fiber->data[i] = janet_wrap_nil();
}
/* get frames */
stack = frame;
stacktop = fiber->stackstart - JANET_FRAME_SIZE;
int32_t stack = frame;
int32_t stacktop = fiber_stackstart - JANET_FRAME_SIZE;
while (stack > 0) {
JanetFunction *func = NULL;
JanetFuncDef *def = NULL;
JanetFuncEnv *env = NULL;
int32_t frameflags = readint(st, &data);
int32_t prevframe = readint(st, &data);
int32_t pcdiff = readint(st, &data);
int32_t prevframe = readnat(st, &data);
int32_t pcdiff = readnat(st, &data);
/* Get frame items */
Janet *framestack = fiber->data + stack;
@@ -964,15 +979,7 @@ static const uint8_t *unmarshal_one_fiber(
/* Check env */
if (frameflags & JANET_STACKFRAME_HASENV) {
frameflags &= ~JANET_STACKFRAME_HASENV;
int32_t offset = stack;
int32_t length = stacktop - stack;
data = unmarshal_one_env(st, data, &env, flags + 1);
if (env->offset != 0 && env->offset != offset)
janet_panic("funcenv offset does not match fiber frame");
if (env->length != 0 && env->length != length)
janet_panic("funcenv length does not match fiber frame");
env->offset = offset;
env->length = length;
}
/* Error checking */
@@ -980,11 +987,11 @@ static const uint8_t *unmarshal_one_fiber(
if (expected_framesize != stacktop - stack) {
janet_panic("fiber stackframe size mismatch");
}
if (pcdiff < 0 || pcdiff >= def->bytecode_length) {
if (pcdiff >= def->bytecode_length) {
janet_panic("fiber stackframe has invalid pc");
}
if ((int32_t)(prevframe + JANET_FRAME_SIZE) > stack) {
janet_panic("fibre stackframe does not align with previous frame");
janet_panic("fiber stackframe does not align with previous frame");
}
/* Get stack items */
@@ -1007,25 +1014,32 @@ static const uint8_t *unmarshal_one_fiber(
}
/* Check for fiber env */
if (fiber->flags & JANET_FIBER_FLAG_HASENV) {
if (fiber_flags & JANET_FIBER_FLAG_HASENV) {
Janet envv;
fiber->flags &= ~JANET_FIBER_FLAG_HASENV;
fiber_flags &= ~JANET_FIBER_FLAG_HASENV;
data = unmarshal_one(st, data, &envv, flags + 1);
janet_asserttype(envv, JANET_TABLE);
fiber->env = janet_unwrap_table(envv);
fiber_env = janet_unwrap_table(envv);
}
/* Check for child fiber */
if (fiber->flags & JANET_FIBER_FLAG_HASCHILD) {
if (fiber_flags & JANET_FIBER_FLAG_HASCHILD) {
Janet fiberv;
fiber->flags &= ~JANET_FIBER_FLAG_HASCHILD;
fiber_flags &= ~JANET_FIBER_FLAG_HASCHILD;
data = unmarshal_one(st, data, &fiberv, flags + 1);
janet_asserttype(fiberv, JANET_FIBER);
fiber->child = janet_unwrap_fiber(fiberv);
}
/* Return data */
/* We have valid fiber, finally construct remaining fields. */
fiber->frame = frame;
fiber->flags = fiber_flags;
fiber->stackstart = fiber_stackstart;
fiber->stacktop = fiber_stacktop;
fiber->maxstack = fiber_maxstack;
fiber->env = fiber_env;
/* Return data */
*out = fiber;
return data;
}
@@ -1084,7 +1098,7 @@ static const uint8_t *unmarshal_one_abstract(UnmarshalState *st, const uint8_t *
Janet key;
data = unmarshal_one(st, data, &key, flags + 1);
const JanetAbstractType *at = janet_get_abstract_type(key);
if (at == NULL) return NULL;
if (at == NULL) goto oops;
if (at->unmarshal) {
JanetMarshalContext context = {NULL, st, flags, data, at};
*out = janet_wrap_abstract(at->unmarshal(&context));
@@ -1093,7 +1107,8 @@ static const uint8_t *unmarshal_one_abstract(UnmarshalState *st, const uint8_t *
}
return context.data;
}
return NULL;
oops:
janet_panic("invalid abstract type");
}
static const uint8_t *unmarshal_one(
@@ -1105,7 +1120,7 @@ static const uint8_t *unmarshal_one(
MARSH_STACKCHECK;
MARSH_EOS(st, data);
lead = data[0];
if (lead < 200) {
if (lead < LB_REAL) {
*out = janet_wrap_integer(readint(st, &data));
return data;
}
@@ -1159,7 +1174,7 @@ static const uint8_t *unmarshal_one(
case LB_KEYWORD:
case LB_REGISTRY: {
data++;
int32_t len = readint(st, &data);
int32_t len = readnat(st, &data);
MARSH_EOS(st, data - 1 + len);
if (lead == LB_STRING) {
const uint8_t *str = janet_string(data, len);
@@ -1219,7 +1234,11 @@ static const uint8_t *unmarshal_one(
/* Things that open with integers */
{
data++;
int32_t len = readint(st, &data);
int32_t len = readnat(st, &data);
/* DOS check */
if (lead != LB_REFERENCE) {
MARSH_EOS(st, data - 1 + len);
}
if (lead == LB_ARRAY) {
/* Array */
JanetArray *array = janet_array(len);
@@ -1251,7 +1270,7 @@ static const uint8_t *unmarshal_one(
*out = janet_wrap_struct(janet_struct_end(struct_));
janet_v_push(st->lookup, *out);
} else if (lead == LB_REFERENCE) {
if (len < 0 || len >= janet_v_count(st->lookup))
if (len >= janet_v_count(st->lookup))
janet_panicf("invalid reference %d", len);
*out = st->lookup[len];
} else {

View File

@@ -255,6 +255,10 @@ JANET_DEFINE_MATHOP(fabs, fabs)
JANET_DEFINE_MATHOP(floor, floor)
JANET_DEFINE_MATHOP(trunc, trunc)
JANET_DEFINE_MATHOP(round, round)
JANET_DEFINE_MATHOP(gamma, lgamma)
JANET_DEFINE_MATHOP(log1p, log1p)
JANET_DEFINE_MATHOP(erf, erf)
JANET_DEFINE_MATHOP(erfc, erfc)
#define JANET_DEFINE_MATH2OP(name, fop)\
static Janet janet_##name(int32_t argc, Janet *argv) {\
@@ -267,6 +271,7 @@ static Janet janet_##name(int32_t argc, Janet *argv) {\
JANET_DEFINE_MATH2OP(atan2, atan2)
JANET_DEFINE_MATH2OP(pow, pow)
JANET_DEFINE_MATH2OP(hypot, hypot)
JANET_DEFINE_MATH2OP(nextafter, nextafter)
static Janet janet_not(int32_t argc, Janet *argv) {
janet_fixarity(argc, 1);
@@ -438,6 +443,26 @@ static const JanetReg math_cfuns[] = {
JDOC("(math/exp2 x)\n\n"
"Returns 2 to the power of x.")
},
{
"math/log1p", janet_log1p,
JDOC("(math/log1p x)\n\n"
"Returns (log base e of x) + 1 more accurately than (+ (math/log x) 1)")
},
{
"math/gamma", janet_gamma,
JDOC("(math/gamma x)\n\n"
"Returns gamma(x).")
},
{
"math/erfc", janet_erfc,
JDOC("(math/erfc x)\n\n"
"Returns the complementary error function of x.")
},
{
"math/erf", janet_erf,
JDOC("(math/erf x)\n\n"
"Returns the error function of x.")
},
{
"math/expm1", janet_expm1,
JDOC("(math/expm1 x)\n\n"
@@ -453,6 +478,11 @@ static const JanetReg math_cfuns[] = {
JDOC("(math/round x)\n\n"
"Returns the integer nearest to x.")
},
{
"math/next", janet_nextafter,
JDOC("(math/next y)\n\n"
"Returns the next representable floating point value after x in the direction of y.")
},
{NULL, NULL, NULL}
};

703
src/core/net.c Normal file
View File

@@ -0,0 +1,703 @@
/*
* Copyright (c) 2020 Calvin Rose
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to
* deal in the Software without restriction, including without limitation the
* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
* sell copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*/
#ifndef JANET_AMALG
#include "features.h"
#include <janet.h>
#include "util.h"
#endif
#ifdef JANET_WINDOWS
#include <windows.h>
#include <winsock2.h>
#include <ws2tcpip.h>
#pragma comment (lib, "Ws2_32.lib")
#pragma comment (lib, "Mswsock.lib")
#pragma comment (lib, "Advapi32.lib")
#else
#include <unistd.h>
#include <signal.h>
#include <sys/ioctl.h>
#include <sys/fcntl.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <poll.h>
#include <netdb.h>
#endif
/*
* Streams
*/
#define JANET_STREAM_CLOSED 1
#define JANET_STREAM_READABLE 2
#define JANET_STREAM_WRITABLE 4
typedef struct {
#ifdef JANET_WINDOWS
SOCKET socket;
#else
int fd;
#endif
int flags;
} JanetStream;
static int janet_stream_close(void *p, size_t s);
static int janet_stream_getter(void *p, Janet key, Janet *out);
static const JanetAbstractType StreamAT = {
"core/stream",
janet_stream_close,
NULL,
janet_stream_getter,
JANET_ATEND_GET
};
static int janet_stream_close(void *p, size_t s) {
(void) s;
JanetStream *stream = p;
if (!(stream->flags & JANET_STREAM_CLOSED)) {
stream->flags |= JANET_STREAM_CLOSED;
#ifdef JANET_WINDOWS
closesocket(stream->socket);
#else
close(stream->fd);
#endif
}
return 0;
}
#ifdef JANET_WINDOWS
static JanetStream *make_stream(SOCKET socket, int flags) {
u_long iMode = 0;
JanetStream *stream = janet_abstract(&StreamAT, sizeof(JanetStream));
ioctlsocket(socket, FIONBIO, &iMode);
stream->socket = socket;
stream->flags = flags;
return stream;
}
#else
static JanetStream *make_stream(int fd, int flags) {
JanetStream *stream = janet_abstract(&StreamAT, sizeof(JanetStream));
fcntl(fd, F_SETFL, fcntl(fd, F_GETFL, 0) | O_NONBLOCK);
stream->fd = fd;
stream->flags = flags;
return stream;
}
#endif
/*
* Event loop
*/
/* This large struct describes a waiting file descriptor, as well
* as what to do when we get an event for it. It is a variant type, where
* each variant implements a simple state machine. */
typedef struct {
/* File descriptor to listen for events on. */
JanetStream *stream;
/* Fiber to resume when event finishes. Can be NULL, in which case,
* no fiber is resumed when event completes. */
JanetFiber *fiber;
/* What kind of event we are listening for.
* As more IO functionality get's added, we can
* expand this. */
enum {
JLE_READ_CHUNK,
JLE_READ_SOME,
JLE_READ_ACCEPT,
JLE_CONNECT,
JLE_WRITE_FROM_BUFFER,
JLE_WRITE_FROM_STRINGLIKE
} event_type;
/* Each variant can have a different payload. */
union {
/* JLE_READ_CHUNK/JLE_READ_SOME */
struct {
int32_t bytes_left;
JanetBuffer *buf;
} read_chunk;
/* JLE_READ_ACCEPT */
struct {
JanetFunction *handler;
} read_accept;
/* JLE_WRITE_FROM_BUFFER */
struct {
JanetBuffer *buf;
int32_t start;
} write_from_buffer;
/* JLE_WRITE_FROM_STRINGLIKE */
struct {
const uint8_t *str;
int32_t start;
} write_from_stringlike;
} data;
} JanetLoopFD;
#define JANET_LOOPFD_MAX 1024
/* Global loop data */
#ifdef JANET_WINDOWS
JANET_THREAD_LOCAL WSAPOLLFD janet_vm_pollfds[JANET_LOOPFD_MAX];
#else
JANET_THREAD_LOCAL struct pollfd janet_vm_pollfds[JANET_LOOPFD_MAX];
#endif
JANET_THREAD_LOCAL JanetLoopFD janet_vm_loopfds[JANET_LOOPFD_MAX];
JANET_THREAD_LOCAL int janet_vm_loop_count;
/* We could also add/remove gc roots. This is easier for now. */
void janet_net_markloop(void) {
for (int i = 0; i < janet_vm_loop_count; i++) {
JanetLoopFD lfd = janet_vm_loopfds[i];
if (lfd.fiber != NULL) {
janet_mark(janet_wrap_fiber(lfd.fiber));
}
janet_mark(janet_wrap_abstract(lfd.stream));
switch (lfd.event_type) {
default:
break;
case JLE_READ_CHUNK:
case JLE_READ_SOME:
janet_mark(janet_wrap_buffer(lfd.data.read_chunk.buf));
break;
case JLE_READ_ACCEPT:
janet_mark(janet_wrap_function(lfd.data.read_accept.handler));
break;
case JLE_CONNECT:
break;
case JLE_WRITE_FROM_BUFFER:
janet_mark(janet_wrap_buffer(lfd.data.write_from_buffer.buf));
break;
case JLE_WRITE_FROM_STRINGLIKE:
janet_mark(janet_wrap_string(lfd.data.write_from_stringlike.str));
}
}
}
/* Add a loop fd to the global event loop */
static int janet_loop_schedule(JanetLoopFD lfd, short events) {
if (janet_vm_loop_count == JANET_LOOPFD_MAX) {
return -1;
}
int index = janet_vm_loop_count++;
janet_vm_loopfds[index] = lfd;
#ifdef JANET_WINDOWS
janet_vm_pollfds[index].fd = lfd.stream->socket;
#else
janet_vm_pollfds[index].fd = lfd.stream->fd;
#endif
janet_vm_pollfds[index].events = events;
janet_vm_pollfds[index].revents = 0;
return index;
}
/* Remove event from list */
static void janet_loop_rmindex(int index) {
janet_vm_loopfds[index] = janet_vm_loopfds[--janet_vm_loop_count];
janet_vm_pollfds[index] = janet_vm_pollfds[janet_vm_loop_count];
}
/* Return delta in number of loop fds. Abstracted out so
* we can separate out the polling logic */
static size_t janet_loop_event(size_t index) {
JanetLoopFD *jlfd = janet_vm_loopfds + index;
JanetStream *stream = jlfd->stream;
#ifdef JANET_WINDOWS
SOCKET socket = stream->socket;
#else
int fd = stream->fd;
#endif
int ret = 1;
int should_resume = 0;
Janet resumeval = janet_wrap_nil();
if (stream->flags & JANET_STREAM_CLOSED) {
should_resume = 1;
ret = 0;
} else {
switch (jlfd->event_type) {
case JLE_READ_CHUNK:
case JLE_READ_SOME: {
JanetBuffer *buffer = jlfd->data.read_chunk.buf;
int32_t bytes_left = jlfd->data.read_chunk.bytes_left;
janet_buffer_extra(buffer, bytes_left);
if (!(stream->flags & JANET_STREAM_READABLE)) {
should_resume = 1;
ret = 0;
break;
}
#ifdef JANET_WINDOWS
long nread;
do {
nread = recv(socket, buffer->data + buffer->count, bytes_left, 0);
} while (nread == -1 && WSAGetLastError() == WSAEINTR);
if (WSAGetLastError() == WSAEWOULDBLOCK) {
ret = 1;
break;
}
#else
ssize_t nread;
do {
nread = read(fd, buffer->data + buffer->count, bytes_left);
} while (nread == -1 && errno == EINTR);
if (errno == EAGAIN || errno == EWOULDBLOCK) {
ret = 1;
break;
}
#endif
if (nread > 0) {
buffer->count += nread;
bytes_left -= nread;
} else {
bytes_left = 0;
}
if (jlfd->event_type == JLE_READ_SOME || bytes_left == 0) {
should_resume = 1;
if (nread > 0) {
resumeval = janet_wrap_buffer(buffer);
}
ret = 0;
} else {
jlfd->data.read_chunk.bytes_left = bytes_left;
ret = 1;
}
break;
}
case JLE_READ_ACCEPT: {
#ifdef JANET_WINDOWS
SOCKET connfd = accept(socket, NULL, NULL);
if (connfd != INVALID_SOCKET) {
#else
char addr[256] = {0}; /* Just make sure it is large enough for largest address type */
socklen_t len = 0;
int connfd = accept(fd, (void *) &addr, &len);
if (connfd >= 0) {
#endif
/* Made a new connection socket */
JanetStream *stream = make_stream(connfd, JANET_STREAM_READABLE | JANET_STREAM_WRITABLE);
Janet streamv = janet_wrap_abstract(stream);
JanetFunction *handler = jlfd->data.read_accept.handler;
Janet out;
JanetFiber *fiberp = NULL;
/* Launch connection fiber */
JanetSignal sig = janet_pcall(handler, 1, &streamv, &out, &fiberp);
if (sig != JANET_SIGNAL_OK && sig != JANET_SIGNAL_EVENT) {
janet_stacktrace(fiberp, out);
}
}
ret = JANET_LOOPFD_MAX;
break;
}
case JLE_WRITE_FROM_BUFFER:
case JLE_WRITE_FROM_STRINGLIKE: {
int32_t start, len;
const uint8_t *bytes;
if (!(stream->flags & JANET_STREAM_WRITABLE)) {
should_resume = 1;
ret = 0;
break;
}
if (jlfd->event_type == JLE_WRITE_FROM_BUFFER) {
JanetBuffer *buffer = jlfd->data.write_from_buffer.buf;
bytes = buffer->data;
len = buffer->count;
start = jlfd->data.write_from_buffer.start;
} else {
bytes = jlfd->data.write_from_stringlike.str;
len = janet_string_length(bytes);
start = jlfd->data.write_from_stringlike.start;
}
if (start < len) {
int32_t nbytes = len - start;
#ifdef JANET_WINDOWS
long nwrote;
do {
nwrote = send(socket, bytes + start, nbytes, 0);
} while (nwrote == -1 && WSAGetLastError() == WSAEINTR);
#else
ssize_t nwrote;
do {
nwrote = write(fd, bytes + start, nbytes);
} while (nwrote == -1 && errno == EINTR);
#endif
if (nwrote > 0) {
start += nwrote;
} else {
start = len;
}
}
if (start >= len) {
should_resume = 1;
ret = 0;
} else {
if (jlfd->event_type == JLE_WRITE_FROM_BUFFER) {
jlfd->data.write_from_buffer.start = start;
} else {
jlfd->data.write_from_stringlike.start = start;
}
ret = 1;
}
break;
}
case JLE_CONNECT: {
break;
}
}
}
/* Resume a fiber for some events */
if (NULL != jlfd->fiber && should_resume) {
/* Resume the fiber */
Janet out;
JanetSignal sig = janet_continue(jlfd->fiber, resumeval, &out);
if (sig != JANET_SIGNAL_OK && sig != JANET_SIGNAL_EVENT) {
janet_stacktrace(jlfd->fiber, out);
}
}
/* Remove this handler from the handler pool. */
if (should_resume) janet_loop_rmindex((int) index);
return ret;
}
static void janet_loop1(void) {
/* Remove closed file descriptors */
for (int i = 0; i < janet_vm_loop_count;) {
if (janet_vm_loopfds[i].stream->flags & JANET_STREAM_CLOSED) {
janet_loop_rmindex(i);
} else {
i++;
}
}
/* Poll */
if (janet_vm_loop_count == 0) return;
int ready;
#ifdef JANET_WINDOWS
do {
ready = WSAPoll(janet_vm_pollfds, janet_vm_loop_count, -1);
} while (ready == -1 && WSAGetLastError() == WSAEINTR);
if (ready == -1) return;
#else
do {
ready = poll(janet_vm_pollfds, janet_vm_loop_count, -1);
} while (ready == -1 && errno == EAGAIN);
if (ready == -1) return;
#endif
/* Handle events */
for (int i = 0; i < janet_vm_loop_count;) {
int revents = janet_vm_pollfds[i].revents;
janet_vm_pollfds[i].revents = 0;
if ((janet_vm_pollfds[i].events | POLLHUP | POLLERR) & revents) {
size_t delta = janet_loop_event(i);
i += (int) delta;
} else {
i++;
}
}
}
void janet_loop(void) {
while (janet_vm_loop_count) {
janet_loop1();
}
}
/*
* Scheduling Helpers
*/
#define JANET_SCHED_FSOME 1
JANET_NO_RETURN static void janet_sched_read(JanetStream *stream, JanetBuffer *buf, int32_t nbytes, int flags) {
JanetLoopFD lfd = {0};
lfd.stream = stream;
lfd.fiber = janet_root_fiber();
lfd.event_type = (flags & JANET_SCHED_FSOME) ? JLE_READ_SOME : JLE_READ_CHUNK;
lfd.data.read_chunk.buf = buf;
lfd.data.read_chunk.bytes_left = nbytes;
janet_loop_schedule(lfd, POLLIN);
janet_signalv(JANET_SIGNAL_EVENT, janet_wrap_nil());
}
JANET_NO_RETURN static void janet_sched_write_buffer(JanetStream *stream, JanetBuffer *buf) {
JanetLoopFD lfd = {0};
lfd.stream = stream;
lfd.fiber = janet_root_fiber();
lfd.event_type = JLE_WRITE_FROM_BUFFER;
lfd.data.write_from_buffer.buf = buf;
lfd.data.write_from_buffer.start = 0;
janet_loop_schedule(lfd, POLLOUT);
janet_signalv(JANET_SIGNAL_EVENT, janet_wrap_nil());
}
JANET_NO_RETURN static void janet_sched_write_stringlike(JanetStream *stream, const uint8_t *str) {
JanetLoopFD lfd = {0};
lfd.stream = stream;
lfd.fiber = janet_root_fiber();
lfd.event_type = JLE_WRITE_FROM_STRINGLIKE;
lfd.data.write_from_stringlike.str = str;
lfd.data.write_from_stringlike.start = 0;
janet_loop_schedule(lfd, POLLOUT);
janet_signalv(JANET_SIGNAL_EVENT, janet_wrap_nil());
}
/* Needs argc >= offset + 2 */
static struct addrinfo *janet_get_addrinfo(Janet *argv, int32_t offset) {
/* Get host and port */
const char *host = janet_getcstring(argv, offset);
const char *port = janet_getcstring(argv, offset + 1);
/* getaddrinfo */
struct addrinfo *ai = NULL;
struct addrinfo hints = {0};
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
hints.ai_protocol = 0;
hints.ai_flags = AI_PASSIVE;
int status = getaddrinfo(host, port, &hints, &ai);
if (status) {
janet_panicf("could not get address info: %s", gai_strerror(status));
}
return ai;
}
/*
* C Funs
*/
static Janet cfun_net_connect(int32_t argc, Janet *argv) {
janet_fixarity(argc, 2);
struct addrinfo *ai = janet_get_addrinfo(argv, 0);
#ifdef JANET_WINDOWS
/* Create socket */
SOCKET sock = socket(ai->ai_family, ai->ai_socktype, ai->ai_protocol);
if (sock == INVALID_SOCKET) {
freeaddrinfo(ai);
janet_panic("could not create socket");
}
/* Connect to socket */
int status = connect(sock, ai->ai_addr, (int) ai->ai_addrlen);
freeaddrinfo(ai);
if (status == -1) {
closesocket(sock);
janet_panic("could not connect to socket");
}
#else
/* Create socket */
int sock = socket(ai->ai_family, ai->ai_socktype, ai->ai_protocol);
if (sock < 0) {
freeaddrinfo(ai);
janet_panic("could not create socket");
}
/* Connect to socket */
int status = connect(sock, ai->ai_addr, ai->ai_addrlen);
freeaddrinfo(ai);
if (status < 0) {
close(sock);
janet_panic("could not connect to socket");
}
#endif
/* Wrap socket in abstract type JanetStream */
JanetStream *stream = make_stream(sock, JANET_STREAM_READABLE | JANET_STREAM_WRITABLE);
return janet_wrap_abstract(stream);
}
static Janet cfun_net_server(int32_t argc, Janet *argv) {
janet_fixarity(argc, 3);
/* Get host, port, and handler*/
JanetFunction *fun = janet_getfunction(argv, 2);
struct addrinfo *ai = janet_get_addrinfo(argv, 0);
#ifdef JANET_WINDOWS
/* Check all addrinfos in a loop for the first that we can bind to. */
SOCKET sfd = INVALID_SOCKET;
struct addrinfo *rp = NULL;
for (rp = ai; rp != NULL; rp = rp->ai_next) {
sfd = socket(rp->ai_family, rp->ai_socktype, rp->ai_protocol);
if (sfd == INVALID_SOCKET) continue;
/* Set various socket options */
int enable = 1;
if (setsockopt(sfd, SOL_SOCKET, SO_REUSEADDR, (char *) &enable, sizeof(int)) < 0) {
closesocket(sfd);
janet_panic("setsockopt(SO_REUSEADDR) failed");
}
/* Bind */
if (bind(sfd, rp->ai_addr, (int) rp->ai_addrlen) == 0) break;
closesocket(sfd);
}
if (NULL == rp) {
freeaddrinfo(ai);
janet_panic("could not bind to any sockets");
}
/* listen */
int status = listen(sfd, 1024);
freeaddrinfo(ai);
if (status) {
closesocket(sfd);
janet_panic("could not listen on file descriptor");
}
#else
/* Check all addrinfos in a loop for the first that we can bind to. */
int sfd = 0;
struct addrinfo *rp = NULL;
for (rp = ai; rp != NULL; rp = rp->ai_next) {
sfd = socket(rp->ai_family, rp->ai_socktype, rp->ai_protocol);
if (sfd == -1) continue;
/* Set various socket options */
int enable = 1;
if (setsockopt(sfd, SOL_SOCKET, SO_REUSEADDR, &enable, sizeof(int)) < 0) {
close(sfd);
janet_panic("setsockopt(SO_REUSEADDR) failed");
}
#ifdef SO_REUSEPORT
if (setsockopt(sfd, SOL_SOCKET, SO_REUSEPORT, &enable, sizeof(int)) < 0) {
close(sfd);
janet_panic("setsockopt(SO_REUSEPORT) failed");
}
#endif
/* Bind */
if (bind(sfd, rp->ai_addr, rp->ai_addrlen) == 0) break;
close(sfd);
}
if (NULL == rp) {
freeaddrinfo(ai);
janet_panic("could not bind to any sockets");
}
/* listen */
int status = listen(sfd, 1024);
freeaddrinfo(ai);
if (status) {
close(sfd);
janet_panic("could not listen on file descriptor");
}
/* We need to ignore sigpipe when reading and writing to our connection socket.
* Since a connection could be disconnected at any time, any read or write may fail.
* We don't want to blow up the whole application. */
signal(SIGPIPE, SIG_IGN);
#endif
/* Put sfd on our loop */
JanetLoopFD lfd = {0};
lfd.stream = make_stream(sfd, 0);
lfd.event_type = JLE_READ_ACCEPT;
lfd.data.read_accept.handler = fun;
janet_loop_schedule(lfd, POLLIN);
return janet_wrap_abstract(lfd.stream);
}
static Janet cfun_stream_read(int32_t argc, Janet *argv) {
janet_arity(argc, 2, 3);
JanetStream *stream = janet_getabstract(argv, 0, &StreamAT);
int32_t n = janet_getnat(argv, 1);
JanetBuffer *buffer = janet_optbuffer(argv, argc, 2, 10);
janet_sched_read(stream, buffer, n, JANET_SCHED_FSOME);
}
static Janet cfun_stream_chunk(int32_t argc, Janet *argv) {
janet_arity(argc, 2, 3);
JanetStream *stream = janet_getabstract(argv, 0, &StreamAT);
int32_t n = janet_getnat(argv, 1);
JanetBuffer *buffer = janet_optbuffer(argv, argc, 2, 10);
janet_sched_read(stream, buffer, n, 0);
}
static Janet cfun_stream_close(int32_t argc, Janet *argv) {
janet_fixarity(argc, 1);
JanetStream *stream = janet_getabstract(argv, 0, &StreamAT);
janet_stream_close(stream, 0);
return janet_wrap_nil();
}
static Janet cfun_stream_write(int32_t argc, Janet *argv) {
janet_fixarity(argc, 2);
JanetStream *stream = janet_getabstract(argv, 0, &StreamAT);
if (janet_checktype(argv[1], JANET_BUFFER)) {
janet_sched_write_buffer(stream, janet_getbuffer(argv, 1));
} else {
JanetByteView bytes = janet_getbytes(argv, 1);
janet_sched_write_stringlike(stream, bytes.bytes);
}
}
static const JanetMethod stream_methods[] = {
{"chunk", cfun_stream_chunk},
{"close", cfun_stream_close},
{"read", cfun_stream_read},
{"write", cfun_stream_write},
{NULL, NULL}
};
static int janet_stream_getter(void *p, Janet key, Janet *out) {
(void) p;
if (!janet_checktype(key, JANET_KEYWORD)) return 0;
return janet_getmethod(janet_unwrap_keyword(key), stream_methods, out);
}
static const JanetReg net_cfuns[] = {
{
"net/server", cfun_net_server,
JDOC("(net/server host port)\n\nStart a TCP server.")
},
{"net/read", cfun_stream_read, NULL},
{"net/chunk", cfun_stream_chunk, NULL},
{"net/write", cfun_stream_write, NULL},
{"net/close", cfun_stream_close, NULL},
{"net/connect", cfun_net_connect, NULL},
{NULL, NULL, NULL}
};
void janet_lib_net(JanetTable *env) {
janet_vm_loop_count = 0;
#ifdef JANET_WINDOWS
WSADATA wsaData;
janet_assert(!WSAStartup(MAKEWORD(2, 2), &wsaData), "could not start winsock");
#endif
janet_core_cfuns(env, NULL, net_cfuns);
}
void janet_net_deinit(void) {
#ifdef JANET_WINDOWS
WSACleanup();
#endif
}

View File

@@ -731,7 +731,10 @@ static timeint_t entry_getint(Janet env_entry, char *field) {
static Janet os_mktime(int32_t argc, Janet *argv) {
janet_arity(argc, 1, 2);
time_t t;
struct tm t_info = { 0 };
struct tm t_info;
/* Use memset instead of = {0} to silence paranoid warning in macos */
memset(&t_info, 0, sizeof(t_info));
if (!janet_checktype(argv[0], JANET_TABLE) &&
!janet_checktype(argv[0], JANET_STRUCT))
@@ -880,40 +883,23 @@ static Janet os_readlink(int32_t argc, Janet *argv) {
}
#ifdef JANET_WINDOWS
static const uint8_t *janet_decode_permissions(unsigned short m) {
uint8_t flags[9] = {0};
flags[0] = flags[3] = flags[6] = (m & S_IREAD) ? 'r' : '-';
flags[1] = flags[4] = flags[7] = (m & S_IWRITE) ? 'w' : '-';
flags[2] = flags[5] = flags[8] = (m & S_IEXEC) ? 'x' : '-';
return janet_string(flags, sizeof(flags));
typedef struct _stat jstat_t;
typedef unsigned short jmode_t;
static int32_t janet_perm_to_unix(unsigned short m) {
int32_t ret = 0;
if (m & S_IEXEC) ret |= 0111;
if (m & S_IWRITE) ret |= 0222;
if (m & S_IREAD) ret |= 0444;
return ret;
}
static unsigned short janet_encode_permissions(Janet *argv, int32_t n) {
if (janet_checkint(argv[n])) {
int32_t x = janet_unwrap_integer(argv[n]);
if (x < 0 || x > 0777) {
janet_panicf("expected integer in range [0, 8r777], got %v", argv[n]);
}
unsigned short m = 0;
if (x & 1 || x & 010 || x & 0100) m |= S_IEXEC;
if (x & 2 || x & 020 || x & 0200) m |= S_IWRITE;
if (x & 4 || x & 040 || x & 0400) m |= S_IREAD;
return m;
}
JanetString perm = janet_getstring(argv, n);
if (janet_string_length(perm) != 9) {
janet_panicf("expected string of length 9, got %S", perm);
}
static unsigned short janet_perm_from_unix(int32_t x) {
unsigned short m = 0;
if (perm[0] == 'r') m |= S_IREAD;
if (perm[1] == 'w') m |= S_IWRITE;
if (perm[2] == 'x') m |= S_IEXEC;
if (perm[3] == 'r') m |= S_IREAD;
if (perm[4] == 'w') m |= S_IWRITE;
if (perm[5] == 'x') m |= S_IEXEC;
if (perm[6] == 'r') m |= S_IREAD;
if (perm[7] == 'w') m |= S_IWRITE;
if (perm[8] == 'x') m |= S_IEXEC;
if (x & 111) m |= S_IEXEC;
if (x & 222) m |= S_IWRITE;
if (x & 444) m |= S_IREAD;
return m;
}
@@ -924,44 +910,22 @@ static const uint8_t *janet_decode_mode(unsigned short m) {
else if (m & _S_IFCHR) str = "character";
return janet_ckeyword(str);
}
#else
static const uint8_t *janet_decode_permissions(mode_t m) {
uint8_t flags[9] = {0};
flags[0] = (m & S_IRUSR) ? 'r' : '-';
flags[1] = (m & S_IWUSR) ? 'w' : '-';
flags[2] = (m & S_IXUSR) ? 'x' : '-';
flags[3] = (m & S_IRGRP) ? 'r' : '-';
flags[4] = (m & S_IWGRP) ? 'w' : '-';
flags[5] = (m & S_IXGRP) ? 'x' : '-';
flags[6] = (m & S_IROTH) ? 'r' : '-';
flags[7] = (m & S_IWOTH) ? 'w' : '-';
flags[8] = (m & S_IXOTH) ? 'x' : '-';
return janet_string(flags, sizeof(flags));
static int32_t janet_decode_permissions(jmode_t mode) {
return (int32_t)(mode & (S_IEXEC | S_IWRITE | S_IREAD));
}
static mode_t janet_encode_permissions(Janet *argv, int32_t n) {
if (janet_checkint(argv[n])) {
int32_t x = janet_unwrap_integer(argv[n]);
if (x < 0 || x > 0777) {
janet_panicf("expected integer in range [0, 8r777], got %v", argv[n]);
}
return (mode_t) x;
}
JanetString perm = janet_getstring(argv, n);
if (janet_string_length(perm) != 9) {
janet_panicf("expected string of length 9, got %S", perm);
}
mode_t m = 0;
if (perm[0] == 'r') m |= S_IRUSR;
if (perm[1] == 'w') m |= S_IWUSR;
if (perm[2] == 'x') m |= S_IXUSR;
if (perm[3] == 'r') m |= S_IRGRP;
if (perm[4] == 'w') m |= S_IWGRP;
if (perm[5] == 'x') m |= S_IXGRP;
if (perm[6] == 'r') m |= S_IROTH;
if (perm[7] == 'w') m |= S_IWOTH;
if (perm[8] == 'x') m |= S_IXOTH;
return m;
#else
typedef struct stat jstat_t;
typedef mode_t jmode_t;
static int32_t janet_perm_to_unix(mode_t m) {
return (int32_t) m;
}
static mode_t janet_perm_from_unix(int32_t x) {
return (mode_t) x;
}
static const uint8_t *janet_decode_mode(mode_t m) {
@@ -975,13 +939,64 @@ static const uint8_t *janet_decode_mode(mode_t m) {
else if (S_ISCHR(m)) str = "character";
return janet_ckeyword(str);
}
static int32_t janet_decode_permissions(jmode_t mode) {
return (int32_t)(mode & 0777);
}
#endif
#ifdef JANET_WINDOWS
typedef struct _stat jstat_t;
#else
typedef struct stat jstat_t;
#endif
static int32_t os_parse_permstring(const uint8_t *perm) {
int32_t m = 0;
if (perm[0] == 'r') m |= 0400;
if (perm[1] == 'w') m |= 0200;
if (perm[2] == 'x') m |= 0100;
if (perm[3] == 'r') m |= 0040;
if (perm[4] == 'w') m |= 0020;
if (perm[5] == 'x') m |= 0010;
if (perm[6] == 'r') m |= 0004;
if (perm[7] == 'w') m |= 0002;
if (perm[8] == 'x') m |= 0001;
return m;
}
static Janet os_make_permstring(int32_t permissions) {
uint8_t bytes[9] = {0};
bytes[0] = (permissions & 0400) ? 'r' : '-';
bytes[1] = (permissions & 0200) ? 'w' : '-';
bytes[2] = (permissions & 0100) ? 'x' : '-';
bytes[3] = (permissions & 0040) ? 'r' : '-';
bytes[4] = (permissions & 0020) ? 'w' : '-';
bytes[5] = (permissions & 0010) ? 'x' : '-';
bytes[6] = (permissions & 0004) ? 'r' : '-';
bytes[7] = (permissions & 0002) ? 'w' : '-';
bytes[8] = (permissions & 0001) ? 'x' : '-';
return janet_stringv(bytes, sizeof(bytes));
}
static int32_t os_get_unix_mode(const Janet *argv, int32_t n) {
int32_t unix_mode;
if (janet_checkint(argv[n])) {
/* Integer mode */
int32_t x = janet_unwrap_integer(argv[n]);
if (x < 0 || x > 0777) {
janet_panicf("bad slot #%d, expected integer in range [0, 8r777], got %v", n, argv[n]);
}
unix_mode = x;
} else {
/* Bytes mode */
JanetByteView bytes = janet_getbytes(argv, n);
if (bytes.len != 9) {
janet_panicf("bad slot #%d: expected byte sequence of length 9, got %v", n, argv[n]);
}
unix_mode = os_parse_permstring(bytes.bytes);
}
return unix_mode;
}
static jmode_t os_getmode(const Janet *argv, int32_t n) {
return janet_perm_from_unix(os_get_unix_mode(argv, n));
}
/* Getters */
static Janet os_stat_dev(jstat_t *st) {
@@ -993,8 +1008,11 @@ static Janet os_stat_inode(jstat_t *st) {
static Janet os_stat_mode(jstat_t *st) {
return janet_wrap_keyword(janet_decode_mode(st->st_mode));
}
static Janet os_stat_int_permissions(jstat_t *st) {
return janet_wrap_integer(janet_perm_to_unix(janet_decode_permissions(st->st_mode)));
}
static Janet os_stat_permissions(jstat_t *st) {
return janet_wrap_string(janet_decode_permissions(st->st_mode));
return os_make_permstring(janet_perm_to_unix(janet_decode_permissions(st->st_mode)));
}
static Janet os_stat_uid(jstat_t *st) {
return janet_wrap_number(st->st_uid);
@@ -1045,6 +1063,7 @@ static const struct OsStatGetter os_stat_getters[] = {
{"dev", os_stat_dev},
{"inode", os_stat_inode},
{"mode", os_stat_mode},
{"int-permissions", os_stat_int_permissions},
{"permissions", os_stat_permissions},
{"uid", os_stat_uid},
{"gid", os_stat_gid},
@@ -1122,14 +1141,25 @@ static Janet os_chmod(int32_t argc, Janet *argv) {
janet_fixarity(argc, 2);
const char *path = janet_getcstring(argv, 0);
#ifdef JANET_WINDOWS
int res = _chmod(path, janet_encode_permissions(argv, 1));
int res = _chmod(path, os_getmode(argv, 1));
#else
int res = chmod(path, janet_encode_permissions(argv, 1));
int res = chmod(path, os_getmode(argv, 1));
#endif
if (-1 == res) janet_panicf("%s: %s", strerror(errno), path);
return janet_wrap_nil();
}
static Janet os_umask(int32_t argc, Janet *argv) {
janet_fixarity(argc, 1);
int mask = (int) os_getmode(argv, 0);
#ifdef JANET_WINDOWS
int res = _umask(mask);
#else
int res = umask(mask);
#endif
return janet_wrap_integer(janet_perm_to_unix(res));
}
static Janet os_dir(int32_t argc, Janet *argv) {
janet_arity(argc, 1, 2);
const char *dir = janet_getcstring(argv, 0);
@@ -1191,6 +1221,16 @@ static Janet os_realpath(int32_t argc, Janet *argv) {
#endif
}
static Janet os_permission_string(int32_t argc, Janet *argv) {
janet_fixarity(argc, 1);
return os_make_permstring(os_get_unix_mode(argv, 0));
}
static Janet os_permission_int(int32_t argc, Janet *argv) {
janet_fixarity(argc, 1);
return janet_wrap_integer(os_get_unix_mode(argv, 0));
}
#endif /* JANET_REDUCED_OS */
static const JanetReg os_cfuns[] = {
@@ -1249,7 +1289,8 @@ static const JanetReg os_cfuns[] = {
" only that information from stat. If the file or directory does not exist, returns nil. The keys are\n\n"
"\t:dev - the device that the file is on\n"
"\t:mode - the type of file, one of :file, :directory, :block, :character, :fifo, :socket, :link, or :other\n"
"\t:permissions - A unix permission string like \"rwx--x--x\". On windows, a string like \"rwx\".\n"
"\t:int-permissions - A Unix permission integer like 8r744\n"
"\t:permissions - A Unix permission string like \"rwxr--r--\"\n"
"\t:uid - File uid\n"
"\t:gid - File gid\n"
"\t:nlink - number of links to file\n"
@@ -1270,9 +1311,9 @@ static const JanetReg os_cfuns[] = {
"os/chmod", os_chmod,
JDOC("(os/chmod path mode)\n\n"
"Change file permissions, where mode is a permission string as returned by "
"os/stat, or an integer. "
"When mode is an integer, it is interpreted as a unix permission value, best specified in octal, like "
"8r666 or 8r400. Windows will not differentiate between user, group, and other permissions. Returns nil.")
"os/perm-string, or an integer as returned by os/perm-int. "
"When mode is an integer, it is interpreted as a Unix permission value, best specified in octal, like "
"8r666 or 8r400. Windows will not differentiate between user, group, and other permissions, and thus will combine all of these permissions. Returns nil.")
},
{
"os/touch", os_touch,
@@ -1285,11 +1326,16 @@ static const JanetReg os_cfuns[] = {
JDOC("(os/cd path)\n\n"
"Change current directory to path. Returns nil on success, errors on failure.")
},
{
"os/umask", os_umask,
JDOC("(os/umask mask)\n\n"
"Set a new umask, returns the old umask.")
},
{
"os/mkdir", os_mkdir,
JDOC("(os/mkdir path)\n\n"
"Create a new directory. The path will be relative to the current directory if relative, otherwise "
"it will be an absolute path. Returns true if the directory was create, false if the directoyr already exists, and "
"it will be an absolute path. Returns true if the directory was created, false if the directory already exists, and "
"errors otherwise.")
},
{
@@ -1305,8 +1351,10 @@ static const JanetReg os_cfuns[] = {
{
"os/link", os_link,
JDOC("(os/link oldpath newpath &opt symlink)\n\n"
"Create a symlink from oldpath to newpath, returning nil. The 3rd optional paramater "
"enables a symlink iff truthy, hard link otherwise or if not provided. Does not work on Windows.")
"Create a link at newpath that points to oldpath and returns nil. "
"Iff symlink is truthy, creates a symlink. "
"Iff symlink is falsey or not provided, "
"creates a hard link. Does not work on Windows.")
},
{
"os/symlink", os_symlink,
@@ -1376,14 +1424,14 @@ static const JanetReg os_cfuns[] = {
{
"os/cryptorand", os_cryptorand,
JDOC("(os/cryptorand n &opt buf)\n\n"
"Get or append n bytes of good quality random data provided by the os. Returns a new buffer or buf.")
"Get or append n bytes of good quality random data provided by the OS. Returns a new buffer or buf.")
},
{
"os/date", os_date,
JDOC("(os/date &opt time local)\n\n"
"Returns the given time as a date struct, or the current time if no time is given. "
"Returns a struct with following key values. Note that all numbers are 0-indexed. "
"Date is given in UTC unless local is truthy, in which case the date is formated for "
"Date is given in UTC unless local is truthy, in which case the date is formatted for "
"the local timezone.\n\n"
"\t:seconds - number of seconds [0-61]\n"
"\t:minutes - number of minutes [0-59]\n"
@@ -1406,6 +1454,19 @@ static const JanetReg os_cfuns[] = {
"Get the absolute path for a given path, following ../, ./, and symlinks. "
"Returns an absolute path as a string. Will raise an error on Windows.")
},
{
"os/perm-string", os_permission_string,
JDOC("(os/perm-string int)\n\n"
"Convert a Unix octal permission value from a permission integer as returned by os/stat "
"to a human readable string, that follows the formatting "
"of unix tools like ls. Returns the string as a 9 character string of r, w, x and - characters. Does not "
"include the file/directory/symlink character as rendered by `ls`.")
},
{
"os/perm-int", os_permission_int,
JDOC("(os/perm-int bytes)\n\n"
"Parse a 9 character permission string and return an integer that can be used by chmod.")
},
#endif
{NULL, NULL, NULL}
};

View File

@@ -26,6 +26,9 @@
#include "util.h"
#endif
#define JANET_PARSER_DEAD 0x1
#define JANET_PARSER_GENERATED_ERROR 0x2
/* Check if a character is whitespace */
static int is_whitespace(uint8_t c) {
return c == ' '
@@ -201,6 +204,8 @@ static int checkescape(uint8_t c) {
default:
return -1;
case 'x':
case 'u':
case 'U':
return 1;
case 'n':
return '\n';
@@ -228,6 +233,24 @@ static int checkescape(uint8_t c) {
/* Forward declare */
static int stringchar(JanetParser *p, JanetParseState *state, uint8_t c);
static void write_codepoint(JanetParser *p, int32_t codepoint) {
if (codepoint <= 0x7F) {
push_buf(p, (uint8_t) codepoint);
} else if (codepoint <= 0x7FF) {
push_buf(p, (uint8_t)((codepoint >> 6) & 0x1F) | 0xC0);
push_buf(p, (uint8_t)((codepoint >> 0) & 0x3F) | 0x80);
} else if (codepoint <= 0xFFFF) {
push_buf(p, (uint8_t)((codepoint >> 12) & 0x0F) | 0xE0);
push_buf(p, (uint8_t)((codepoint >> 6) & 0x3F) | 0x80);
push_buf(p, (uint8_t)((codepoint >> 0) & 0x3F) | 0x80);
} else {
push_buf(p, (uint8_t)((codepoint >> 18) & 0x07) | 0xF0);
push_buf(p, (uint8_t)((codepoint >> 12) & 0x3F) | 0x80);
push_buf(p, (uint8_t)((codepoint >> 6) & 0x3F) | 0x80);
push_buf(p, (uint8_t)((codepoint >> 0) & 0x3F) | 0x80);
}
}
static int escapeh(JanetParser *p, JanetParseState *state, uint8_t c) {
int digit = to_hex(c);
if (digit < 0) {
@@ -237,7 +260,27 @@ static int escapeh(JanetParser *p, JanetParseState *state, uint8_t c) {
state->argn = (state->argn << 4) + digit;
state->counter--;
if (!state->counter) {
push_buf(p, (state->argn & 0xFF));
push_buf(p, (uint8_t)(state->argn & 0xFF));
state->argn = 0;
state->consumer = stringchar;
}
return 1;
}
static int escapeu(JanetParser *p, JanetParseState *state, uint8_t c) {
int digit = to_hex(c);
if (digit < 0) {
p->error = "invalid hex digit in unicode escape";
return 1;
}
state->argn = (state->argn << 4) + digit;
state->counter--;
if (!state->counter) {
if (state->argn > 0x10FFFF) {
p->error = "invalid unicode codepoint";
return 1;
}
write_codepoint(p, state->argn);
state->argn = 0;
state->consumer = stringchar;
}
@@ -254,6 +297,10 @@ static int escape1(JanetParser *p, JanetParseState *state, uint8_t c) {
state->counter = 2;
state->argn = 0;
state->consumer = escapeh;
} else if (c == 'u' || c == 'U') {
state->counter = c == 'u' ? 4 : 6;
state->argn = 0;
state->consumer = escapeu;
} else {
push_buf(p, (uint8_t) e);
state->consumer = stringchar;
@@ -393,21 +440,23 @@ static Janet close_array(JanetParser *p, JanetParseState *state) {
static Janet close_struct(JanetParser *p, JanetParseState *state) {
JanetKV *st = janet_struct_begin(state->argn >> 1);
for (int32_t i = state->argn; i > 0; i -= 2) {
Janet value = p->args[--p->argcount];
Janet key = p->args[--p->argcount];
for (size_t i = p->argcount - state->argn; i < p->argcount; i += 2) {
Janet key = p->args[i];
Janet value = p->args[i + 1];
janet_struct_put(st, key, value);
}
p->argcount -= state->argn;
return janet_wrap_struct(janet_struct_end(st));
}
static Janet close_table(JanetParser *p, JanetParseState *state) {
JanetTable *table = janet_table(state->argn >> 1);
for (int32_t i = state->argn; i > 0; i -= 2) {
Janet value = p->args[--p->argcount];
Janet key = p->args[--p->argcount];
for (size_t i = p->argcount - state->argn; i < p->argcount; i += 2) {
Janet key = p->args[i];
Janet value = p->args[i + 1];
janet_table_put(table, key, value);
}
p->argcount -= state->argn;
return janet_wrap_table(table);
}
@@ -591,11 +640,30 @@ void janet_parser_eof(JanetParser *parser) {
size_t oldline = parser->line;
janet_parser_consume(parser, '\n');
if (parser->statecount > 1) {
parser->error = "unexpected end of source";
JanetParseState *s = parser->states + (parser->statecount - 1);
JanetBuffer *buffer = janet_buffer(40);
janet_buffer_push_cstring(buffer, "unexpected end of source, ");
if (s->flags & PFLAG_PARENS) {
janet_buffer_push_u8(buffer, '(');
} else if (s->flags & PFLAG_SQRBRACKETS) {
janet_buffer_push_u8(buffer, '[');
} else if (s->flags & PFLAG_CURLYBRACKETS) {
janet_buffer_push_u8(buffer, '{');
} else if (s->flags & PFLAG_STRING) {
janet_buffer_push_u8(buffer, '"');
} else if (s->flags & PFLAG_LONGSTRING) {
int32_t i;
for (i = 0; i < s->argn; i++) {
janet_buffer_push_u8(buffer, '`');
}
}
janet_formatb(buffer, " opened at line %d, column %d", s->line, s->column);
parser->error = (const char *) janet_string(buffer->data, buffer->count);
parser->flag |= JANET_PARSER_GENERATED_ERROR;
}
parser->line = oldline;
parser->column = oldcolumn;
parser->flag = 1;
parser->flag |= JANET_PARSER_DEAD;
}
enum JanetParserStatus janet_parser_status(JanetParser *parser) {
@@ -617,6 +685,7 @@ const char *janet_parser_error(JanetParser *parser) {
if (status == JANET_PARSE_ERROR) {
const char *e = parser->error;
parser->error = NULL;
parser->flag &= ~JANET_PARSER_GENERATED_ERROR;
janet_parser_flush(parser);
return e;
}
@@ -720,6 +789,9 @@ static int parsermark(void *p, size_t size) {
for (i = 0; i < parser->argcount; i++) {
janet_mark(parser->args[i]);
}
if (parser->flag & JANET_PARSER_GENERATED_ERROR) {
janet_mark(janet_wrap_string(parser->error));
}
return 0;
}
@@ -854,7 +926,11 @@ static Janet cfun_parse_error(int32_t argc, Janet *argv) {
janet_fixarity(argc, 1);
JanetParser *p = janet_getabstract(argv, 0, &janet_parser_type);
const char *err = janet_parser_error(p);
if (err) return janet_cstringv(err);
if (err) {
return (p->flag & JANET_PARSER_GENERATED_ERROR)
? janet_wrap_string(err)
: janet_cstringv(err);
}
return janet_wrap_nil();
}
@@ -953,31 +1029,30 @@ struct ParserStateGetter {
};
static Janet parser_state_delimiters(const JanetParser *_p) {
JanetParser *clone = janet_abstract(&janet_parser_type, sizeof(JanetParser));
janet_parser_clone(_p, clone);
JanetParser *p = (JanetParser *)_p;
size_t i;
const uint8_t *str;
size_t oldcount;
oldcount = clone->bufcount;
for (i = 0; i < clone->statecount; i++) {
JanetParseState *s = clone->states + i;
oldcount = p->bufcount;
for (i = 0; i < p->statecount; i++) {
JanetParseState *s = p->states + i;
if (s->flags & PFLAG_PARENS) {
push_buf(clone, '(');
push_buf(p, '(');
} else if (s->flags & PFLAG_SQRBRACKETS) {
push_buf(clone, '[');
push_buf(p, '[');
} else if (s->flags & PFLAG_CURLYBRACKETS) {
push_buf(clone, '{');
push_buf(p, '{');
} else if (s->flags & PFLAG_STRING) {
push_buf(clone, '"');
push_buf(p, '"');
} else if (s->flags & PFLAG_LONGSTRING) {
int32_t i;
for (i = 0; i < s->argn; i++) {
push_buf(clone, '`');
push_buf(p, '`');
}
}
}
str = janet_string(clone->buf + oldcount, (int32_t)(clone->bufcount - oldcount));
clone->bufcount = oldcount;
str = janet_string(p->buf + oldcount, (int32_t)(p->bufcount - oldcount));
p->bufcount = oldcount;
return janet_wrap_string(str);
}

View File

@@ -156,7 +156,7 @@ static void janet_escape_string_impl(JanetBuffer *buffer, const uint8_t *str, in
janet_buffer_push_bytes(buffer, (const uint8_t *)"\\\\", 2);
break;
default:
if (c < 32 || c > 127) {
if (c < 32 || c > 126) {
uint8_t buf[4];
buf[0] = '\\';
buf[1] = 'x';
@@ -459,8 +459,8 @@ static const char *janet_pretty_colors[] = {
#define JANET_PRETTY_DICT_ONELINE 4
#define JANET_PRETTY_IND_ONELINE 10
#define JANET_PRETTY_DICT_LIMIT 16
#define JANET_PRETTY_ARRAY_LIMIT 16
#define JANET_PRETTY_DICT_LIMIT 30
#define JANET_PRETTY_ARRAY_LIMIT 160
/* Helper for pretty printing */
static void janet_pretty_one(struct pretty *S, Janet x, int is_dict_value) {
@@ -591,6 +591,11 @@ static void janet_pretty_one(struct pretty *S, Janet x, int is_dict_value) {
if (is_dict_value && len >= JANET_PRETTY_DICT_ONELINE) print_newline(S, 0);
for (i = 0; i < cap; i++) {
if (!janet_checktype(kvs[i].key, JANET_NIL)) {
if (counter == JANET_PRETTY_DICT_LIMIT) {
print_newline(S, 0);
janet_buffer_push_cstring(S->buffer, "...");
break;
}
if (first_kv_pair) {
first_kv_pair = 0;
} else {
@@ -600,11 +605,6 @@ static void janet_pretty_one(struct pretty *S, Janet x, int is_dict_value) {
janet_buffer_push_u8(S->buffer, ' ');
janet_pretty_one(S, kvs[i].value, 1);
counter++;
if (counter == 10) {
print_newline(S, 0);
janet_buffer_push_cstring(S->buffer, "...");
break;
}
}
}
}
@@ -728,7 +728,7 @@ static const char *scanformat(
return p;
}
void janet_formatb(JanetBuffer *b, const char *format, va_list args) {
void janet_formatbv(JanetBuffer *b, const char *format, va_list args) {
const char *format_end = format + strlen(format);
const char *c = format;
int32_t startlen = b->count;
@@ -853,7 +853,7 @@ const uint8_t *janet_formatc(const char *format, ...) {
va_start(args, format);
/* Run format */
janet_formatb(&buffer, format, args);
janet_formatbv(&buffer, format, args);
/* Iterate length */
va_end(args);
@@ -863,6 +863,14 @@ const uint8_t *janet_formatc(const char *format, ...) {
return ret;
}
JanetBuffer *janet_formatb(JanetBuffer *buffer, const char *format, ...) {
va_list args;
va_start(args, format);
janet_formatbv(buffer, format, args);
va_end(args);
return buffer;
}
/* Shared implementation between string/format and
* buffer/format */
void janet_buffer_format(

View File

@@ -50,7 +50,7 @@ int janet_dobytes(JanetTable *env, const uint8_t *bytes, int32_t len, const char
JanetFiber *fiber = janet_fiber(f, 64, 0, NULL);
fiber->env = env;
JanetSignal status = janet_continue(fiber, janet_wrap_nil(), &ret);
if (status != JANET_SIGNAL_OK) {
if (status != JANET_SIGNAL_OK && status != JANET_SIGNAL_EVENT) {
janet_stacktrace(fiber, ret);
errflags |= 0x01;
done = 1;

View File

@@ -43,6 +43,7 @@ extern JANET_THREAD_LOCAL int janet_vm_stackn;
/* The current running fiber on the current thread.
* Set and unset by janet_run. */
extern JANET_THREAD_LOCAL JanetFiber *janet_vm_fiber;
extern JANET_THREAD_LOCAL JanetFiber *janet_vm_root_fiber;
/* The current pointer to the inner most jmp_buf. The current
* return point for panics. */

View File

@@ -123,7 +123,8 @@ void janet_struct_put(JanetKV *st, Janet key, Janet value) {
dist = otherdist;
hash = otherhash;
} else if (status == 0) {
/* A key was added to the struct more than once */
/* A key was added to the struct more than once - replace old value */
kv->value = value;
return;
}
}

View File

@@ -465,11 +465,15 @@ static int thread_worker(JanetMailboxPair *pair) {
Janet argv[1] = { parentv };
fiber = janet_fiber(func, 64, 1, argv);
JanetSignal sig = janet_continue(fiber, janet_wrap_nil(), &out);
if (sig != JANET_SIGNAL_OK) {
if (sig != JANET_SIGNAL_OK && sig < JANET_SIGNAL_USER0) {
janet_eprintf("in thread %v: ", janet_wrap_abstract(janet_make_thread(pair->newbox, encode)));
janet_stacktrace(fiber, out);
}
#ifdef JANET_NET
janet_loop();
#endif
/* Normal exit */
destroy_mailbox_pair(pair);
janet_deinit();

View File

@@ -380,7 +380,7 @@ void janet_var(JanetTable *env, const char *name, Janet val, const char *doc) {
}
/* Load many cfunctions at once */
void janet_cfuns(JanetTable *env, const char *regprefix, const JanetReg *cfuns) {
static void _janet_cfuns_prefix(JanetTable *env, const char *regprefix, const JanetReg *cfuns, int defprefix) {
uint8_t *longname_buffer = NULL;
size_t prefixlen = 0;
size_t bufsize = 0;
@@ -414,13 +414,29 @@ void janet_cfuns(JanetTable *env, const char *regprefix, const JanetReg *cfuns)
name = janet_csymbolv(cfuns->name);
}
Janet fun = janet_wrap_cfunction(cfuns->cfun);
janet_def(env, cfuns->name, fun, cfuns->documentation);
if (defprefix) {
JanetTable *subt = janet_table(2);
janet_table_put(subt, janet_ckeywordv("value"), fun);
if (cfuns->documentation)
janet_table_put(subt, janet_ckeywordv("doc"), janet_cstringv(cfuns->documentation));
janet_table_put(env, name, janet_wrap_table(subt));
} else {
janet_def(env, cfuns->name, fun, cfuns->documentation);
}
janet_table_put(janet_vm_registry, fun, name);
cfuns++;
}
free(longname_buffer);
}
void janet_cfuns_prefix(JanetTable *env, const char *regprefix, const JanetReg *cfuns) {
_janet_cfuns_prefix(env, regprefix, cfuns, 1);
}
void janet_cfuns(JanetTable *env, const char *regprefix, const JanetReg *cfuns) {
_janet_cfuns_prefix(env, regprefix, cfuns, 0);
}
/* Abstract type introspection */
void janet_register_abstract_type(const JanetAbstractType *at) {

View File

@@ -35,7 +35,7 @@
#ifndef janet_exit
#include <stdio.h>
#define janet_exit(m) do { \
printf("C runtime error at line %d in file %s: %s\n",\
fprintf(stderr, "C runtime error at line %d in file %s: %s\n",\
__LINE__,\
__FILE__,\
(m));\
@@ -50,7 +50,7 @@
/* What to do when out of memory */
#ifndef JANET_OUT_OF_MEMORY
#include <stdio.h>
#define JANET_OUT_OF_MEMORY do { printf("janet out of memory\n"); exit(1); } while (0)
#define JANET_OUT_OF_MEMORY do { fprintf(stderr, "janet out of memory\n"); exit(1); } while (0)
#endif
/* Omit docstrings in some builds */
@@ -126,5 +126,10 @@ void janet_lib_inttypes(JanetTable *env);
#ifdef JANET_THREADS
void janet_lib_thread(JanetTable *env);
#endif
#ifdef JANET_NET
void janet_lib_net(JanetTable *env);
void janet_net_deinit(void);
void janet_net_markloop(void);
#endif
#endif

View File

@@ -38,6 +38,7 @@ JANET_THREAD_LOCAL JanetTable *janet_vm_registry;
JANET_THREAD_LOCAL JanetTable *janet_vm_abstract_registry;
JANET_THREAD_LOCAL int janet_vm_stackn = 0;
JANET_THREAD_LOCAL JanetFiber *janet_vm_fiber = NULL;
JANET_THREAD_LOCAL JanetFiber *janet_vm_root_fiber = NULL;
JANET_THREAD_LOCAL Janet *janet_vm_return_reg = NULL;
JANET_THREAD_LOCAL jmp_buf *janet_vm_jmp_buf = NULL;
@@ -89,8 +90,8 @@ JANET_THREAD_LOCAL jmp_buf *janet_vm_jmp_buf = NULL;
func = janet_stack_frame(stack)->func; \
} while (0)
#define vm_return(sig, val) do { \
vm_commit(); \
janet_vm_return_reg[0] = (val); \
vm_commit(); \
return (sig); \
} while (0)
@@ -107,13 +108,13 @@ JANET_THREAD_LOCAL jmp_buf *janet_vm_jmp_buf = NULL;
#define vm_assert_type(X, T) do { \
if (!(janet_checktype((X), (T)))) { \
vm_commit(); \
janet_panicf("expected %T, got %t", (1 << (T)), (X)); \
janet_panicf("expected %T, got %v", (1 << (T)), (X)); \
} \
} while (0)
#define vm_assert_types(X, TS) do { \
if (!(janet_checktypes((X), (TS)))) { \
vm_commit(); \
janet_panicf("expected %T, got %t", (TS), (X)); \
janet_panicf("expected %T, got %v", (TS), (X)); \
} \
} while (0)
@@ -290,6 +291,10 @@ static Janet janet_binop_call(const char *lmethod, const char *rmethod, Janet lh
}
}
/* Forward declaration */
static JanetSignal janet_check_can_resume(JanetFiber *fiber, Janet *out);
static JanetSignal janet_continue_no_check(JanetFiber *fiber, Janet in, Janet *out);
/* Interpreter main loop */
static JanetSignal run_vm(JanetFiber *fiber, Janet in) {
@@ -824,7 +829,8 @@ static JanetSignal run_vm(JanetFiber *fiber, Janet in) {
vm_assert(func->def->environments_length > eindex, "invalid upvalue environment");
env = func->envs[eindex];
vm_assert(env->length > vindex, "invalid upvalue index");
if (env->offset) {
vm_assert(janet_env_valid(env), "invalid upvalue environment");
if (env->offset > 0) {
/* On stack */
stack[A] = env->as.fiber->data[env->offset + vindex];
} else {
@@ -841,7 +847,8 @@ static JanetSignal run_vm(JanetFiber *fiber, Janet in) {
vm_assert(func->def->environments_length > eindex, "invalid upvalue environment");
env = func->envs[eindex];
vm_assert(env->length > vindex, "invalid upvalue index");
if (env->offset) {
vm_assert(janet_env_valid(env), "invalid upvalue environment");
if (env->offset > 0) {
env->as.fiber->data[env->offset + vindex] = stack[A];
} else {
env->as.values[vindex] = stack[A];
@@ -904,7 +911,7 @@ static JanetSignal run_vm(JanetFiber *fiber, Janet in) {
if (janet_indexed_view(stack[D], &vals, &len)) {
janet_fiber_pushn(fiber, vals, len);
} else {
janet_panicf("expected %T, got %t", JANET_TFLAG_INDEXED, stack[D]);
janet_panicf("expected %T, got %v", JANET_TFLAG_INDEXED, stack[D]);
}
}
stack = fiber->data + fiber->frame;
@@ -997,8 +1004,12 @@ static JanetSignal run_vm(JanetFiber *fiber, Janet in) {
Janet retreg;
vm_assert_type(stack[B], JANET_FIBER);
JanetFiber *child = janet_unwrap_fiber(stack[B]);
if (janet_check_can_resume(child, &retreg)) {
vm_commit();
janet_panicv(retreg);
}
fiber->child = child;
JanetSignal sig = janet_continue(child, stack[C], &retreg);
JanetSignal sig = janet_continue_no_check(child, stack[C], &retreg);
if (sig != JANET_SIGNAL_OK && !(child->flags & (1 << sig))) {
vm_return(sig, retreg);
}
@@ -1234,15 +1245,14 @@ Janet janet_call(JanetFunction *fun, int32_t argc, const Janet *argv) {
janet_vm_stackn = oldn;
janet_gcunlock(handle);
if (signal != JANET_SIGNAL_OK) janet_panicv(*janet_vm_return_reg);
if (signal != JANET_SIGNAL_OK) {
janet_panicv(*janet_vm_return_reg);
}
return *janet_vm_return_reg;
}
/* Enter the main vm loop */
JanetSignal janet_continue(JanetFiber *fiber, Janet in, Janet *out) {
jmp_buf buf;
static JanetSignal janet_check_can_resume(JanetFiber *fiber, Janet *out) {
/* Check conditions */
JanetFiberStatus old_status = janet_fiber_status(fiber);
if (janet_vm_stackn >= JANET_RECURSION_GUARD) {
@@ -1259,13 +1269,22 @@ JanetSignal janet_continue(JanetFiber *fiber, Janet in, Janet *out) {
*out = janet_wrap_string(str);
return JANET_SIGNAL_ERROR;
}
return JANET_SIGNAL_OK;
}
static JanetSignal janet_continue_no_check(JanetFiber *fiber, Janet in, Janet *out) {
jmp_buf buf;
JanetFiberStatus old_status = janet_fiber_status(fiber);
/* Continue child fiber if it exists */
if (fiber->child) {
if (janet_vm_root_fiber == NULL) janet_vm_root_fiber = fiber;
JanetFiber *child = fiber->child;
janet_vm_stackn++;
JanetSignal sig = janet_continue(child, in, &in);
janet_vm_stackn--;
if (janet_vm_root_fiber == fiber) janet_vm_root_fiber = NULL;
if (sig != JANET_SIGNAL_OK && !(child->flags & (1 << sig))) {
*out = in;
return sig;
@@ -1294,6 +1313,7 @@ JanetSignal janet_continue(JanetFiber *fiber, Janet in, Janet *out) {
Janet *old_vm_return_reg = janet_vm_return_reg;
/* Setup fiber */
if (janet_vm_root_fiber == NULL) janet_vm_root_fiber = fiber;
janet_vm_fiber = fiber;
janet_gcroot(janet_wrap_fiber(fiber));
janet_fiber_set_status(fiber, JANET_STATUS_ALIVE);
@@ -1319,6 +1339,7 @@ JanetSignal janet_continue(JanetFiber *fiber, Janet in, Janet *out) {
janet_gcunroot(janet_wrap_fiber(fiber));
/* Restore global state */
if (janet_vm_root_fiber == fiber) janet_vm_root_fiber = NULL;
janet_vm_gc_suspend = handle;
janet_vm_fiber = old_vm_fiber;
janet_vm_stackn = oldn;
@@ -1328,6 +1349,14 @@ JanetSignal janet_continue(JanetFiber *fiber, Janet in, Janet *out) {
return signal;
}
/* Enter the main vm loop */
JanetSignal janet_continue(JanetFiber *fiber, Janet in, Janet *out) {
/* Check conditions */
JanetSignal tmp_signal = janet_check_can_resume(fiber, out);
if (tmp_signal) return tmp_signal;
return janet_continue_no_check(fiber, in, out);
}
JanetSignal janet_pcall(
JanetFunction *fun,
int32_t argc,
@@ -1388,6 +1417,10 @@ int janet_init(void) {
janet_vm_core_env = NULL;
/* Seed RNG */
janet_rng_seed(janet_default_rng(), 0);
/* Fibers */
janet_vm_fiber = NULL;
janet_vm_root_fiber = NULL;
janet_vm_stackn = 0;
/* Threads */
#ifdef JANET_THREADS
janet_threads_init();
@@ -1406,7 +1439,12 @@ void janet_deinit(void) {
janet_vm_registry = NULL;
janet_vm_abstract_registry = NULL;
janet_vm_core_env = NULL;
janet_vm_fiber = NULL;
janet_vm_root_fiber = NULL;
#ifdef JANET_THREADS
janet_threads_deinit();
#endif
#ifdef JANET_NET
janet_net_deinit();
#endif
}

View File

@@ -159,6 +159,11 @@ extern "C" {
#define JANET_TYPED_ARRAY
#endif
/* Enable or disable networking */
#ifndef JANET_NO_NET
#define JANET_NET
#endif
/* Enable or disable large int types (for now 64 bit, maybe 128 / 256 bit integer types) */
#ifndef JANET_NO_INT_TYPES
#define JANET_INT_TYPES
@@ -289,6 +294,8 @@ typedef enum {
JANET_SIGNAL_USER9
} JanetSignal;
#define JANET_SIGNAL_EVENT JANET_SIGNAL_USER9
/* Fiber statuses - mostly corresponds to signals. */
typedef enum {
JANET_STATUS_DEAD,
@@ -733,8 +740,9 @@ struct JanetStackFrame {
int32_t flags;
};
/* Number of Janets a frame takes up in the stack */
#define JANET_FRAME_SIZE ((sizeof(JanetStackFrame) + sizeof(Janet) - 1) / sizeof(Janet))
/* Number of Janets a frame takes up in the stack
* Should be constant across architectures */
#define JANET_FRAME_SIZE 4
/* A dynamic array type. */
struct JanetArray {
@@ -1110,6 +1118,11 @@ extern enum JanetInstructionType janet_instructions[JOP_INSTRUCTION_COUNT];
/***** START SECTION MAIN *****/
/* Event Loop */
#ifdef JANET_NET
JANET_API void janet_loop(void);
#endif
/* Parsing */
extern JANET_API const JanetAbstractType janet_parser_type;
JANET_API void janet_parser_init(JanetParser *parser);
@@ -1237,7 +1250,8 @@ JANET_API void janet_description_b(JanetBuffer *buffer, Janet x);
#define janet_cstringv(cstr) janet_wrap_string(janet_cstring(cstr))
#define janet_stringv(str, len) janet_wrap_string(janet_string((str), (len)))
JANET_API JanetString janet_formatc(const char *format, ...);
JANET_API void janet_formatb(JanetBuffer *bufp, const char *format, va_list args);
JANET_API JanetBuffer *janet_formatb(JanetBuffer *bufp, const char *format, ...);
JANET_API void janet_formatbv(JanetBuffer *bufp, const char *format, va_list args);
/* Symbol functions */
JANET_API JanetSymbol janet_symbol(const uint8_t *str, int32_t len);
@@ -1286,6 +1300,7 @@ JANET_API JanetFiber *janet_fiber(JanetFunction *callee, int32_t capacity, int32
JANET_API JanetFiber *janet_fiber_reset(JanetFiber *fiber, JanetFunction *callee, int32_t argc, const Janet *argv);
JANET_API JanetFiberStatus janet_fiber_status(JanetFiber *fiber);
JANET_API JanetFiber *janet_current_fiber(void);
JANET_API JanetFiber *janet_root_fiber(void);
/* Treat similar types through uniform interfaces for iteration */
JANET_API int janet_indexed_view(Janet seq, const Janet **data, int32_t *len);

View File

@@ -515,6 +515,147 @@ static void check_specials(JanetByteView src) {
check_cmatch(src, "while");
}
static void resolve_format(JanetTable *entry) {
int is_macro = janet_truthy(janet_table_get(entry, janet_ckeywordv("macro")));
Janet refv = janet_table_get(entry, janet_ckeywordv("ref"));
int is_ref = janet_checktype(refv, JANET_ARRAY);
Janet value = janet_wrap_nil();
if (is_ref) {
JanetArray *a = janet_unwrap_array(refv);
if (a->count) value = a->data[0];
} else {
value = janet_table_get(entry, janet_ckeywordv("value"));
}
if (is_macro) {
fprintf(stderr, " macro\n");
gbl_lines_below++;
} else if (is_ref) {
janet_eprintf(" var (%t)\n", value);
gbl_lines_below++;
} else {
janet_eprintf(" %t\n", value);
gbl_lines_below++;
}
Janet sm = janet_table_get(entry, janet_ckeywordv("source-map"));
Janet path = janet_get(sm, janet_wrap_integer(0));
Janet line = janet_get(sm, janet_wrap_integer(1));
Janet col = janet_get(sm, janet_wrap_integer(2));
if (janet_checktype(path, JANET_STRING) && janet_truthy(line) && janet_truthy(col)) {
janet_eprintf(" %S on line %v, column %v\n", janet_unwrap_string(path), line, col);
gbl_lines_below++;
}
}
static void doc_format(JanetString doc, int32_t width) {
int32_t maxcol = width - 8;
uint8_t wordbuf[256] = {0};
int32_t wordp = 0;
int32_t current = 0;
if (maxcol > 200) maxcol = 200;
fprintf(stderr, " ");
for (int32_t i = 0; i < janet_string_length(doc); i++) {
uint8_t b = doc[i];
switch (b) {
default: {
if (maxcol <= current + wordp + 1) {
if (!current) {
fwrite(wordbuf, wordp, 1, stderr);
wordp = 0;
}
fprintf(stderr, "\n ");
gbl_lines_below++;
current = 0;
}
wordbuf[wordp++] = b;
break;
}
case '\t': {
if (maxcol <= current + wordp + 2) {
if (!current) {
fwrite(wordbuf, wordp, 1, stderr);
wordp = 0;
}
fprintf(stderr, "\n ");
gbl_lines_below++;
current = 0;
}
wordbuf[wordp++] = ' ';
wordbuf[wordp++] = ' ';
break;
}
case '\n':
case ' ': {
if (wordp) {
int32_t oldcur = current;
int spacer = maxcol > current + wordp + 1;
if (spacer) current++;
else current = 0;
current += wordp;
if (oldcur) fprintf(stderr, spacer ? " " : "\n ");
if (oldcur && !spacer) gbl_lines_below++;
fwrite(wordbuf, wordp, 1, stderr);
wordp = 0;
}
if (b == '\n') {
fprintf(stderr, "\n ");
gbl_lines_below++;
current = 0;
}
}
}
}
if (wordp) {
int32_t oldcur = current;
int spacer = maxcol > current + wordp + 1;
if (spacer) current++;
else current = 0;
current += wordp + 1;
if (oldcur) fprintf(stderr, spacer ? " " : "\n ");
if (oldcur && !spacer) gbl_lines_below++;
fwrite(wordbuf, wordp, 1, stderr);
wordp = 0;
}
}
static void find_matches(JanetByteView prefix) {
JanetTable *env = gbl_complete_env;
gbl_match_count = 0;
while (NULL != env) {
JanetKV *kvend = env->data + env->capacity;
for (JanetKV *kv = env->data; kv < kvend; kv++) {
if (!janet_checktype(kv->key, JANET_SYMBOL)) continue;
const uint8_t *sym = janet_unwrap_symbol(kv->key);
check_match(prefix, sym, janet_string_length(sym));
}
env = env->proto;
}
}
static void kshowdoc(void) {
if (!gbl_complete_env) return;
while (is_symbol_char_gen(gbl_buf[gbl_pos])) gbl_pos++;
JanetByteView prefix = get_symprefix();
Janet symbol = janet_symbolv(prefix.bytes, prefix.len);
Janet entry = janet_table_get(gbl_complete_env, symbol);
if (!janet_checktype(entry, JANET_TABLE)) return;
Janet doc = janet_table_get(janet_unwrap_table(entry), janet_ckeywordv("doc"));
if (!janet_checktype(doc, JANET_STRING)) return;
JanetString docs = janet_unwrap_string(doc);
int num_cols = getcols();
clearlines();
fprintf(stderr, "\n\n\n");
gbl_lines_below += 3;
resolve_format(janet_unwrap_table(entry));
fprintf(stderr, "\n");
gbl_lines_below += 1;
doc_format(docs, num_cols);
fprintf(stderr, "\n\n");
gbl_lines_below += 2;
/* Go up to original line (zsh-like autocompletion) */
fprintf(stderr, "\x1B[%dA", gbl_lines_below);
fflush(stderr);
}
static void kshowcomp(void) {
JanetTable *env = gbl_complete_env;
if (env == NULL) {
@@ -528,19 +669,9 @@ static void kshowcomp(void) {
gbl_pos++;
JanetByteView prefix = get_symprefix();
if (prefix.len == 0) return;
if (prefix.len == 0) return;
/* Find all matches */
gbl_match_count = 0;
while (NULL != env) {
JanetKV *kvend = env->data + env->capacity;
for (JanetKV *kv = env->data; kv < kvend; kv++) {
if (!janet_checktype(kv->key, JANET_SYMBOL)) continue;
const uint8_t *sym = janet_unwrap_symbol(kv->key);
check_match(prefix, sym, janet_string_length(sym));
}
env = env->proto;
}
find_matches(prefix);
check_specials(prefix);
@@ -633,6 +764,10 @@ static int line() {
case 6: /* ctrl-f */
kright();
break;
case 7: /* ctrl-g */
kshowdoc();
refresh();
break;
case 127: /* backspace */
case 8: /* ctrl-h */
kbackspace(1);
@@ -876,10 +1011,15 @@ int main(int argc, char **argv) {
JanetFiber *fiber = janet_fiber(janet_unwrap_function(mainfun), 64, 1, mainargs);
fiber->env = env;
status = janet_continue(fiber, janet_wrap_nil(), &out);
if (status != JANET_SIGNAL_OK) {
if (status != JANET_SIGNAL_OK && status != JANET_SIGNAL_EVENT) {
janet_stacktrace(fiber, out);
}
#ifdef JANET_NET
status = JANET_SIGNAL_OK;
janet_loop();
#endif
/* Deinitialize vm */
janet_deinit();
janet_line_deinit();

View File

@@ -0,0 +1,22 @@
#include <stdint.h>
#include <string.h>
#include <janet.h>
int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
char *new_str = (char *)malloc(size + 1);
if (new_str == NULL) {
return 0;
}
memcpy(new_str, data, size);
new_str[size] = '\0';
/* janet logic */
janet_init();
JanetTable *env = janet_core_env(NULL);
janet_dostring(env, new_str, "main", NULL);
janet_deinit();
free(new_str);
return 0;
}

View File

@@ -8,7 +8,8 @@
(defn assert
"Override's the default assert with some nice error handling."
[x e]
[x &opt e]
(default e "assert error")
(++ num-tests-run)
(when x (++ num-tests-passed))
(if x

View File

@@ -206,6 +206,10 @@
(def 🐮 :cow)
(assert (= (string "🐼" 🦊 🐮) "🐼foxcow") "emojis 🙉 :)")
(assert (not= 🦊 "🦊") "utf8 strings are not symbols and vice versa")
(assert (= "\U01F637" "😷") "unicode escape 1")
(assert (= "\u2623" "\U002623" "") "unicode escape 2")
(assert (= "\u24c2" "\U0024c2" "") "unicode escape 3")
(assert (= "\u0061" "a") "unicode escape 4")
# Symbols with @ character
@@ -250,6 +254,11 @@
(assert (apply <= (merge @[1 3 5] @[2 4 6 6 6 9])) "merge sort merge 3")
(assert (apply <= (merge '(1 3 5) @[2 4 6 6 6 9])) "merge sort merge 4")
(assert (deep= @[1 2 3 4 5] (sort @[5 3 4 1 2])) "sort 1")
(assert (deep= @[{:a 1} {:a 4} {:a 7}] (sort-by |($ :a) @[{:a 4} {:a 7} {:a 1}])) "sort 2")
(assert (deep= @[1 2 3 4 5] (sorted [5 3 4 1 2])) "sort 3")
(assert (deep= @[{:a 1} {:a 4} {:a 7}] (sorted-by |($ :a) [{:a 4} {:a 7} {:a 1}])) "sort 4")
# Gensym tests
(assert (not= (gensym) (gensym)) "two gensyms not equal")
@@ -319,5 +328,11 @@
(assert (= true ;(map truthy? [0 "" true @{} {} [] '()])) "truthy values")
(assert (= false ;(map truthy? [nil false])) "non-truthy values")
# Struct and Table duplicate elements
(assert (= {:a 3 :b 2} {:a 1 :b 2 :a 3}) "struct literal duplicate keys")
(assert (= {:a 3 :b 2} (struct :a 1 :b 2 :a 3)) "struct constructor duplicate keys")
(assert (deep= @{:a 3 :b 2} @{:a 1 :b 2 :a 3}) "table literal duplicate keys")
(assert (deep= @{:a 3 :b 2} (table :a 1 :b 2 :a 3)) "table constructor duplicate keys")
(end-suite)

View File

@@ -194,4 +194,51 @@
(assert (deep= @[] (accumulate2 + [])) "accumulate2 2")
(assert (deep= @[] (accumulate 0 + [])) "accumulate 2")
# Perm strings
(assert (= (os/perm-int "rwxrwxrwx") 8r777) "perm 1")
(assert (= (os/perm-int "rwxr-xr-x") 8r755) "perm 2")
(assert (= (os/perm-int "rw-r--r--") 8r644) "perm 3")
(assert (= (band (os/perm-int "rwxrwxrwx") 8r077) 8r077) "perm 4")
(assert (= (band (os/perm-int "rwxr-xr-x") 8r077) 8r055) "perm 5")
(assert (= (band (os/perm-int "rw-r--r--") 8r077) 8r044) "perm 6")
(assert (= (os/perm-string 8r777) "rwxrwxrwx") "perm 7")
(assert (= (os/perm-string 8r755) "rwxr-xr-x") "perm 8")
(assert (= (os/perm-string 8r644) "rw-r--r--") "perm 9")
# Issue #336 cases - don't segfault
(assert-error "unmarshal errors 1" (unmarshal @"\xd6\xb9\xb9"))
(assert-error "unmarshal errors 2" (unmarshal @"\xd7bc"))
(assert-error "unmarshal errors 3" (unmarshal "\xd3\x01\xd9\x01\x62\xcf\x03\x78\x79\x7a" load-image-dict))
(assert-error "unmarshal errors 4"
(unmarshal
@"\xD7\xCD\0e/p\x98\0\0\x03\x01\x01\x01\x02\0\0\x04\0\xCEe/p../tools
\0\0\0/afl\0\0\x01\0erate\xDE\xDE\xDE\xDE\xDE\xDE\xDE\xDE\xDE\xDE
\xA8\xDE\xDE\xDE\xDE\xDE\xDE\0\0\0\xDE\xDE_unmarshal_testcase3.ja
neldb\0\0\0\xD8\x05printG\x01\0\xDE\xDE\xDE'\x03\0marshal_tes/\x02
\0\0\0\0\0*\xFE\x01\04\x02\0\0'\x03\0\r\0\r\0\r\0\r" load-image-dict))
# No segfault, valgrind clean.
(def x @"\xCC\xCD.nd\x80\0\r\x1C\xCDg!\0\x07\xCC\xCD\r\x1Ce\x10\0\r;\xCDb\x04\xFF9\xFF\x80\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04uu\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\0\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04}\x04\x04\x04\x04\x04\x04\x04\x04#\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\0\x01\0\0\x03\x04\x04\x04\xE2\x03\x04\x04\x04\x04\x04\x04\x04\x04\x04\x14\x1A\x04\x04\x04\x04\x04\x18\x04\x04!\x04\xE2\x03\x04\x04\x04\x04\x04\x04$\x04\x04\x04\x04\x04\x04\x04\x04\x04\x80\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04\x04A\0\0\0\x03\0\0!\xBF\xFF")
(unmarshal x load-image-dict)
(gccollect)
(marshal x make-image-dict)
(def b @"\xCC\xCD\0\x03\0\x08\x04\rm\xCD\x7F\xFF\xFF\xFF\x02\0\x02\xD7\xCD\0\x98\0\0\x05\x01\x01\x01\x01\x08\xCE\x01f\xCE../tools/afl/generate_unmarshal_testcases.janet\xCE\x012,\x01\0\0&\x03\0\06\x02\x03\x03)\x03\x01\0*\x04\0\00\x03\x04\0>\x03\0\0\x03\x03\0\0*\x05\0\x11\0\x11\0\x05\0\x05\0\x05\0\x05\0\x05\xC9\xDA\x04\xC9\xC9\xC9")
(unmarshal b load-image-dict)
(gccollect)
(def v (unmarshal
@"\xD7\xCD0\xD4000000\0\x03\x01\xCE\00\0\x01\0\0000\x03\0\0\0000000000\xCC0\0000"
load-image-dict))
(gccollect)
# in vs get regression
(assert (nil? (first @"")) "in vs get 1")
(assert (nil? (last @"")) "in vs get 1")
(end-suite)

51
test/suite9.janet Normal file
View File

@@ -0,0 +1,51 @@
# Copyright (c) 2020 Calvin Rose & contributors
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to
# deal in the Software without restriction, including without limitation the
# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
# sell copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
# IN THE SOFTWARE.
(import ./helper :prefix "" :exit true)
(start-suite 9)
# Net testing
(defn handler
"Simple handler for connections."
[stream]
(defer (:close stream)
(def id (gensym))
(def b @"")
(:read stream 1024 b)
(:write stream b)
(buffer/clear b)))
(def s (net/server "127.0.0.1" "8000" handler))
(assert s "made server 1")
(defn test-echo [msg]
(with [conn (net/connect "127.0.0.1" "8000")]
(:write conn msg)
(def res (:read conn 1024))
(assert (= (string res) msg) (string "echo " msg))))
(test-echo "hello")
(test-echo "world")
(test-echo (string/repeat "abcd" 200))
(:close s)
(end-suite)

View File

@@ -3,12 +3,26 @@
To use these, you need to install afl (of course), and xterm. A tiling window manager helps manage
many concurrent fuzzer instances.
Note, afl sometimes requires system configuration, if you find AFL quitting prematurely, try manually
launching it and addressing any error messages.
## Fuzz the parser
```
$ sh ./tools/afl/prepare_to_fuzz.sh
export NFUZZ=1
$ export NFUZZ=1
$ sh ./tools/afl/fuzz.sh parser
Ctrl+C when done to close all fuzzer terminals.
$ sh ./tools/afl/aggregate_cases.sh parser
$ ls ./fuzz_out/parser_aggregated/
```
```
## Fuzz the unmarshaller
```
$ janet ./tools/afl/generate_unmarshal_testcases.janet
$ sh ./tools/afl/prepare_to_fuzz.sh
$ export NFUZZ=1
$ sh ./tools/afl/fuzz.sh unmarshal
Ctrl+C when done to close all fuzzer terminals.
$ sh ./tools/afl/aggregate_cases.sh unmarshal
$ ls ./fuzz_out/unmarshal_aggregated/
```

View File

@@ -0,0 +1,49 @@
(os/mkdir "./tools/afl/unmarshal_testcases/")
(defn spit-case [n v]
(spit
(string "./tools/afl/unmarshal_testcases/" (string n))
(marshal v make-image-dict)))
(def cases [
nil
"abc"
:def
'hij
123
(int/s64 123)
"7"
[1 2 3]
@[1 2 3]
{:a 123}
@{:b 'xyz}
(peg/compile
'{:a (* "a" :b "a")
:b (* "b" (+ :a 0) "b")
:main (* "(" :b ")")})
(fn f [a] (fn [] {:ab a}))
(fn f [a] (print "hello world!"))
(do
(defn f [a] (yield) @[1 "2"])
(def fb (fiber/new f))
(resume fb)
fb)
])
(eachk i cases
(spit-case i (in cases i)))

View File

@@ -0,0 +1,6 @@
# Unmarshal garbage.
(def v (unmarshal (slurp ((dyn :args) 1)) load-image-dict))
# Trigger leaks or use after free.
(gccollect)
# Attempt to use generated value.
(marshal v make-image-dict)

View File

@@ -308,7 +308,7 @@
<array>
<dict>
<key>match</key>
<string>(\\[nevr0zft"\\']|\\x[0-9a-fA-F][0-9a-fA-f])</string>
<string>(\\[nevr0zft"\\']|\\x[0-9a-fA-F]{2}|\\u[0-9a-fA-F]{4}|\\U[0-9a-fA-F]{6})</string>
<key>name</key>
<string>constant.character.escape.janet</string>
</dict>