mirrors/janet
1
0
Fork 0
mirror of https://github.com/janet-lang/janet synced 2024-11-10 10:49:54 +00:00
Code Issues 1 Releases Wiki Activity
1418ada38f
janet/src/core/ev.c

3182 lines
112 KiB
C
Raw Normal View History

Why is the copyright gone...
2021-10-20 23:05:32 +00:00
/*
Update copyright year.
2023-01-07 21:03:35 +00:00
* Copyright (c) 2023 Calvin Rose
Why is the copyright gone...
2021-10-20 23:05:32 +00:00
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to
* deal in the Software without restriction, including without limitation the
* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
* sell copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
Start with ev module.
2020-05-28 15:39:40 +00:00
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*/
#ifndef JANET_AMALG
#include "features.h"
#include <janet.h>
#include "util.h"
#include "gc.h"
#include "state.h"
Add timeouts to net functions. Further debugging of the general timeout system, as well as having a single fiber wait on multiple state machines (select).
2020-07-20 00:41:12 +00:00
#include "fiber.h"
Start with ev module.
2020-05-28 15:39:40 +00:00
#endif
#ifdef JANET_EV
Move math.h include out of windows ifdef
2020-11-15 16:21:37 +00:00
#include <math.h>
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
#ifdef JANET_WINDOWS
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
#include <winsock2.h>
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
#include <windows.h>
#else
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
#include <pthread.h>
More updates to the ev library.
2020-05-28 21:51:11 +00:00
#include <limits.h>
Start with ev module.
2020-05-28 15:39:40 +00:00
#include <errno.h>
#include <unistd.h>
#include <signal.h>
#include <sys/ioctl.h>
#include <sys/types.h>
#include <fcntl.h>
Add header on BSDs.
2020-09-07 20:22:18 +00:00
#include <netinet/in.h>
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
#include <netinet/tcp.h>
#include <netdb.h>
#include <sys/socket.h>
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
#include <sys/wait.h>
Conditionally include epoll headers.
2020-09-07 18:13:28 +00:00
#ifdef JANET_EV_EPOLL
#include <sys/epoll.h>
Start with ev module.
2020-05-28 15:39:40 +00:00
#include <sys/timerfd.h>
Conditionally include epoll headers.
2020-09-07 18:13:28 +00:00
#endif
Add kqueue support to Janet Note that this is a work in progress and simply a first attempt at getting some code into place before being able to test it. This code follows of sorts both the poll and epoll sections of the codebase hoping to achieve the exact same.
2021-09-03 19:29:13 +00:00
#ifdef JANET_EV_KQUEUE
#include <sys/event.h>
#endif
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
#ifdef JANET_EV_POLL
#include <poll.h>
#endif
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
#endif
Add initial implementation for supervisor channels. Supervisor channels are a simple concept to more efficiently enable dynamic, structure concurrency. When a top-level fiber completes (or errors), it will push itself to it's supervisor channel if it has one (instead of printing a stacktrace). This let's another fiber poll a channel and "supervise" a set of fibers.
2021-01-07 05:19:22 +00:00
typedef struct {
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
JanetVM *thread;
Add initial implementation for supervisor channels. Supervisor channels are a simple concept to more efficiently enable dynamic, structure concurrency. When a top-level fiber completes (or errors), it will push itself to it's supervisor channel if it has one (instead of printing a stacktrace). This let's another fiber poll a channel and "supervise" a set of fibers.
2021-01-07 05:19:22 +00:00
JanetFiber *fiber;
uint32_t sched_id;
enum {
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
JANET_CP_MODE_READ,
JANET_CP_MODE_WRITE,
Add initial implementation for supervisor channels. Supervisor channels are a simple concept to more efficiently enable dynamic, structure concurrency. When a top-level fiber completes (or errors), it will push itself to it's supervisor channel if it has one (instead of printing a stacktrace). This let's another fiber poll a channel and "supervise" a set of fibers.
2021-01-07 05:19:22 +00:00
JANET_CP_MODE_CHOICE_READ,
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
JANET_CP_MODE_CHOICE_WRITE,
JANET_CP_MODE_CLOSE
Add initial implementation for supervisor channels. Supervisor channels are a simple concept to more efficiently enable dynamic, structure concurrency. When a top-level fiber completes (or errors), it will push itself to it's supervisor channel if it has one (instead of printing a stacktrace). This let's another fiber poll a channel and "supervise" a set of fibers.
2021-01-07 05:19:22 +00:00
} mode;
} JanetChannelPending;
typedef struct {
JanetQueue items;
JanetQueue read_pending;
JanetQueue write_pending;
int32_t limit;
Add semantics for closing channels. This makes certain algorithms simpler as channels now have an explicit lifetime - multiple readers can coordinate closing without needing to ensure the same number of reads as writes.
2021-07-31 00:26:42 +00:00
int closed;
Add basic runtime support for threaded abstracts. A threaded abstract is an abstract type that can be freely shared between threads. While no synchronization is provided, refcounting and transport between threads is. This will let implementers more easily exploit OS-level parallelism in C library code. The caveat with these types is that they need to be careful in how they interact with objects on other heaps.
2021-08-20 02:16:20 +00:00
int is_threaded;
Remove pthread.h from janet.h Should make janet a bit easier to use. Also changes the header to not expose the size of native mutexes and rwlocks, except with janet_os_mutex_size and janet_os_rwlock_size.
2022-06-17 22:13:58 +00:00
#ifdef JANET_WINDOWS
CRITICAL_SECTION lock;
#else
pthread_mutex_t lock;
#endif
Add initial implementation for supervisor channels. Supervisor channels are a simple concept to more efficiently enable dynamic, structure concurrency. When a top-level fiber completes (or errors), it will push itself to it's supervisor channel if it has one (instead of printing a stacktrace). This let's another fiber poll a channel and "supervise" a set of fibers.
2021-01-07 05:19:22 +00:00
} JanetChannel;
Move some defines.
2021-07-20 23:24:56 +00:00
typedef struct {
JanetFiber *fiber;
Janet value;
JanetSignal sig;
Address #820 - ev/cancel to work on already scheduled fibers.
2021-09-28 20:41:41 +00:00
uint32_t expected_sched_id; /* If the fiber has been rescheduled this loop, don't run first scheduling. */
Move some defines.
2021-07-20 23:24:56 +00:00
} JanetTask;
/* Wrap return value by pairing it with the callback used to handle it
* in the main thread */
typedef struct {
JanetEVGenericMessage msg;
JanetThreadedCallback cb;
} JanetSelfPipeEvent;
/* Structure used to initialize threads in the thread pool
* (same head structure as self pipe event)*/
typedef struct {
JanetEVGenericMessage msg;
JanetThreadedCallback cb;
JanetThreadedSubroutine subr;
JanetHandle write_pipe;
} JanetEVThreadInit;
Update timestamp function to work on windows.
2020-10-04 20:18:31 +00:00
#define JANET_MAX_Q_CAPACITY 0x7FFFFFF
Use a common queue implementation. Queues occur in three places, so we use a single implementation rather than three separate ones. This also has the result that janet_vm_spawn will not overflow in the case of channel-heavy, IO-light operation.
2020-08-01 19:20:58 +00:00
static void janet_q_init(JanetQueue *q) {
q->data = NULL;
q->head = 0;
q->tail = 0;
q->capacity = 0;
}
static void janet_q_deinit(JanetQueue *q) {
Add config support for custom allocators.
2021-03-23 10:00:48 +00:00
janet_free(q->data);
Use a common queue implementation. Queues occur in three places, so we use a single implementation rather than three separate ones. This also has the result that janet_vm_spawn will not overflow in the case of channel-heavy, IO-light operation.
2020-08-01 19:20:58 +00:00
}
static int32_t janet_q_count(JanetQueue *q) {
return (q->head > q->tail)
? (q->tail + q->capacity - q->head)
: (q->tail - q->head);
}
Allow multiple simultaneous interrupts cleanly for #1262 Instead of setting a flag, each interrupt increments an atomic counter. When the interrupt is finally handled, either by scheduling code to run on the event loop or executing some out of band code, the user must now decrement the interrupt counter with janet_interpreter_interrupt_handled. While this counter is non-zero, the event loop will not enter the interpreter. This changes the API a bit but makes it possible and easy to handle signals without race conditions or scheduler hacks, as the runtime can ensure that high priority code is run before re-entering possibly blocking interpreter code again. Also included is a new function janet_schedule_soon, which prepends to the task queue instead of appending, allowing interrupt handler to skip ahead of all other scheduled fibers. Lastly, also update meson default options to include the interpreter_interrupt code and raise a runtime error if os/sigaction is used with interpreter interrupt but that build option is not enabled.
2023-08-24 01:14:38 +00:00
static int janet_q_maybe_resize(JanetQueue *q, size_t itemsize) {
Use a common queue implementation. Queues occur in three places, so we use a single implementation rather than three separate ones. This also has the result that janet_vm_spawn will not overflow in the case of channel-heavy, IO-light operation.
2020-08-01 19:20:58 +00:00
int32_t count = janet_q_count(q);
/* Resize if needed */
if (count + 1 >= q->capacity) {
if (count + 1 >= JANET_MAX_Q_CAPACITY) return 1;
int32_t newcap = (count + 2) * 2;
if (newcap > JANET_MAX_Q_CAPACITY) newcap = JANET_MAX_Q_CAPACITY;
Add config support for custom allocators.
2021-03-23 10:00:48 +00:00
q->data = janet_realloc(q->data, itemsize * newcap);
Use a common queue implementation. Queues occur in three places, so we use a single implementation rather than three separate ones. This also has the result that janet_vm_spawn will not overflow in the case of channel-heavy, IO-light operation.
2020-08-01 19:20:58 +00:00
if (NULL == q->data) {
JANET_OUT_OF_MEMORY;
}
if (q->head > q->tail) {
/* Two segments, fix 2nd seg. */
int32_t newhead = q->head + (newcap - q->capacity);
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
size_t seg1 = (size_t)(q->capacity - q->head);
if (seg1 > 0) {
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
memmove((char *) q->data + (newhead * itemsize),
(char *) q->data + (q->head * itemsize),
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
seg1 * itemsize);
}
Use a common queue implementation. Queues occur in three places, so we use a single implementation rather than three separate ones. This also has the result that janet_vm_spawn will not overflow in the case of channel-heavy, IO-light operation.
2020-08-01 19:20:58 +00:00
q->head = newhead;
}
q->capacity = newcap;
}
Allow multiple simultaneous interrupts cleanly for #1262 Instead of setting a flag, each interrupt increments an atomic counter. When the interrupt is finally handled, either by scheduling code to run on the event loop or executing some out of band code, the user must now decrement the interrupt counter with janet_interpreter_interrupt_handled. While this counter is non-zero, the event loop will not enter the interpreter. This changes the API a bit but makes it possible and easy to handle signals without race conditions or scheduler hacks, as the runtime can ensure that high priority code is run before re-entering possibly blocking interpreter code again. Also included is a new function janet_schedule_soon, which prepends to the task queue instead of appending, allowing interrupt handler to skip ahead of all other scheduled fibers. Lastly, also update meson default options to include the interpreter_interrupt code and raise a runtime error if os/sigaction is used with interpreter interrupt but that build option is not enabled.
2023-08-24 01:14:38 +00:00
return 0;
}
static int janet_q_push(JanetQueue *q, void *item, size_t itemsize) {
if (janet_q_maybe_resize(q, itemsize)) return 1;
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
memcpy((char *) q->data + itemsize * q->tail, item, itemsize);
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
q->tail = q->tail + 1 < q->capacity ? q->tail + 1 : 0;
Use a common queue implementation. Queues occur in three places, so we use a single implementation rather than three separate ones. This also has the result that janet_vm_spawn will not overflow in the case of channel-heavy, IO-light operation.
2020-08-01 19:20:58 +00:00
return 0;
}
Allow multiple simultaneous interrupts cleanly for #1262 Instead of setting a flag, each interrupt increments an atomic counter. When the interrupt is finally handled, either by scheduling code to run on the event loop or executing some out of band code, the user must now decrement the interrupt counter with janet_interpreter_interrupt_handled. While this counter is non-zero, the event loop will not enter the interpreter. This changes the API a bit but makes it possible and easy to handle signals without race conditions or scheduler hacks, as the runtime can ensure that high priority code is run before re-entering possibly blocking interpreter code again. Also included is a new function janet_schedule_soon, which prepends to the task queue instead of appending, allowing interrupt handler to skip ahead of all other scheduled fibers. Lastly, also update meson default options to include the interpreter_interrupt code and raise a runtime error if os/sigaction is used with interpreter interrupt but that build option is not enabled.
2023-08-24 01:14:38 +00:00
static int janet_q_push_head(JanetQueue *q, void *item, size_t itemsize) {
if (janet_q_maybe_resize(q, itemsize)) return 1;
int32_t newhead = q->head - 1;
if (newhead < 0) {
newhead += q->capacity;
}
memcpy((char *) q->data + itemsize * newhead, item, itemsize);
q->head = newhead;
return 0;
}
Use a common queue implementation. Queues occur in three places, so we use a single implementation rather than three separate ones. This also has the result that janet_vm_spawn will not overflow in the case of channel-heavy, IO-light operation.
2020-08-01 19:20:58 +00:00
static int janet_q_pop(JanetQueue *q, void *out, size_t itemsize) {
if (q->head == q->tail) return 1;
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
memcpy(out, (char *) q->data + itemsize * q->head, itemsize);
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
q->head = q->head + 1 < q->capacity ? q->head + 1 : 0;
Use a common queue implementation. Queues occur in three places, so we use a single implementation rather than three separate ones. This also has the result that janet_vm_spawn will not overflow in the case of channel-heavy, IO-light operation.
2020-08-01 19:20:58 +00:00
return 0;
}
More updates to the ev library.
2020-05-28 21:51:11 +00:00
/* Get current timestamp (millisecond precision) */
static JanetTimestamp ts_now(void);
/* Get current timestamp + an interval (millisecond precision) */
static JanetTimestamp ts_delta(JanetTimestamp ts, double delta) {
Allow infinite wait to work without issues.
2023-02-05 15:29:39 +00:00
if (isinf(delta)) {
return delta < 0 ? ts : INT64_MAX;
}
More updates to the ev library.
2020-05-28 21:51:11 +00:00
ts += (int64_t)round(delta * 1000);
return ts;
Start with ev module.
2020-05-28 15:39:40 +00:00
}
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
/* Look at the next timeout value without removing it. */
More work on timeouts and racing listeners. When two listeners are racing to resume the same fiber, the first should cancel out the other.
2020-07-05 22:26:17 +00:00
static int peek_timeout(JanetTimeout *out) {
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
if (janet_vm.tq_count == 0) return 0;
*out = janet_vm.tq[0];
More work on timeouts and racing listeners. When two listeners are racing to resume the same fiber, the first should cancel out the other.
2020-07-05 22:26:17 +00:00
return 1;
}
/* Remove the next timeout from the priority queue */
static void pop_timeout(size_t index) {
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
if (janet_vm.tq_count <= index) return;
janet_vm.tq[index] = janet_vm.tq[--janet_vm.tq_count];
More work on timeouts and racing listeners. When two listeners are racing to resume the same fiber, the first should cancel out the other.
2020-07-05 22:26:17 +00:00
for (;;) {
size_t left = (index << 1) + 1;
size_t right = left + 1;
size_t smallest = index;
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
if (left < janet_vm.tq_count &&
(janet_vm.tq[left].when < janet_vm.tq[smallest].when))
More work on timeouts and racing listeners. When two listeners are racing to resume the same fiber, the first should cancel out the other.
2020-07-05 22:26:17 +00:00
smallest = left;
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
if (right < janet_vm.tq_count &&
(janet_vm.tq[right].when < janet_vm.tq[smallest].when))
More work on timeouts and racing listeners. When two listeners are racing to resume the same fiber, the first should cancel out the other.
2020-07-05 22:26:17 +00:00
smallest = right;
if (smallest == index) return;
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
JanetTimeout temp = janet_vm.tq[index];
janet_vm.tq[index] = janet_vm.tq[smallest];
janet_vm.tq[smallest] = temp;
More work on timeouts and racing listeners. When two listeners are racing to resume the same fiber, the first should cancel out the other.
2020-07-05 22:26:17 +00:00
index = smallest;
}
}
Start with ev module.
2020-05-28 15:39:40 +00:00
/* Add a timeout to the timeout min heap */
More updates to the ev library.
2020-05-28 21:51:11 +00:00
static void add_timeout(JanetTimeout to) {
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
size_t oldcount = janet_vm.tq_count;
Start with ev module.
2020-05-28 15:39:40 +00:00
size_t newcount = oldcount + 1;
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
if (newcount > janet_vm.tq_capacity) {
Start with ev module.
2020-05-28 15:39:40 +00:00
size_t newcap = 2 * newcount;
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
JanetTimeout *tq = janet_realloc(janet_vm.tq, newcap * sizeof(JanetTimeout));
Start with ev module.
2020-05-28 15:39:40 +00:00
if (NULL == tq) {
JANET_OUT_OF_MEMORY;
}
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
janet_vm.tq = tq;
janet_vm.tq_capacity = newcap;
Start with ev module.
2020-05-28 15:39:40 +00:00
}
/* Append */
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
janet_vm.tq_count = (int32_t) newcount;
janet_vm.tq[oldcount] = to;
Start with ev module.
2020-05-28 15:39:40 +00:00
/* Heapify */
size_t index = oldcount;
while (index > 0) {
size_t parent = (index - 1) >> 1;
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
if (janet_vm.tq[parent].when <= janet_vm.tq[index].when) break;
Start with ev module.
2020-05-28 15:39:40 +00:00
/* Swap */
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
JanetTimeout tmp = janet_vm.tq[index];
janet_vm.tq[index] = janet_vm.tq[parent];
janet_vm.tq[parent] = tmp;
Start with ev module.
2020-05-28 15:39:40 +00:00
/* Next */
index = parent;
}
}
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
void janet_async_end(JanetFiber *fiber) {
if (fiber->ev_callback) {
janet_gcunroot(janet_wrap_abstract(fiber->ev_stream));
fiber->ev_callback = NULL;
if (fiber->ev_state) {
Remove some extra fiber state and use a flag.
2023-10-07 19:07:05 +00:00
if (!(fiber->flags & JANET_FIBER_EV_FLAG_IN_FLIGHT)) {
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
janet_free(fiber->ev_state);
janet_ev_dec_refcount();
}
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
fiber->ev_state = NULL;
Redo state management for Janet listeners. Make more use of the built in GC code for abstracts to be sure things are more correct. Issue before was streams could be freed before IOCP events arrived.
2023-09-25 07:43:36 +00:00
}
}
}
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
void *janet_async_start(JanetFiber *fiber, JanetStream *stream, JanetAsyncMode mode, JanetEVCallback callback, size_t data_size) {
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
janet_assert(!fiber->ev_callback, "double async on fiber");
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
if (mode & JANET_ASYNC_LISTEN_READ) stream->read_fiber = fiber;
if (mode & JANET_ASYNC_LISTEN_WRITE) stream->write_fiber = fiber;
fiber->ev_callback = callback;
fiber->ev_stream = stream;
janet_ev_inc_refcount();
janet_gcroot(janet_wrap_abstract(stream));
if (data_size) {
void *data = janet_malloc(data_size);
fiber->ev_state = data;
return data;
} else {
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
fiber->ev_state = NULL;
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
return NULL;
Redo state management for Janet listeners. Make more use of the built in GC code for abstracts to be sure things are more correct. Issue before was streams could be freed before IOCP events arrived.
2023-09-25 07:43:36 +00:00
}
}
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
void janet_fiber_did_resume(JanetFiber *fiber) {
janet_async_end(fiber);
Start with ev module.
2020-05-28 15:39:40 +00:00
}
Add TOCLOSE back.
2023-09-24 19:28:35 +00:00
static void janet_stream_checktoclose(JanetStream *stream) {
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
if ((stream->flags & JANET_STREAM_TOCLOSE) && !stream->read_fiber && !stream->write_fiber) {
Add TOCLOSE back.
2023-09-24 19:28:35 +00:00
janet_stream_close(stream);
}
}
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
/* Forward declaration */
static void janet_register_stream(JanetStream *stream);
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
static const JanetMethod ev_default_stream_methods[] = {
{"close", janet_cfun_stream_close},
{"read", janet_cfun_stream_read},
{"chunk", janet_cfun_stream_chunk},
{"write", janet_cfun_stream_write},
{NULL, NULL}
};
/* Create a stream*/
JanetStream *janet_stream(JanetHandle handle, uint32_t flags, const JanetMethod *methods) {
JanetStream *stream = janet_abstract(&janet_stream_type, sizeof(JanetStream));
stream->handle = handle;
stream->flags = flags;
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
stream->read_fiber = NULL;
stream->write_fiber = NULL;
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
if (methods == NULL) methods = ev_default_stream_methods;
stream->methods = methods;
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
stream->index = 0;
janet_register_stream(stream);
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
return stream;
}
Redo state management for Janet listeners. Make more use of the built in GC code for abstracts to be sure things are more correct. Issue before was streams could be freed before IOCP events arrived.
2023-09-25 07:43:36 +00:00
static void janet_stream_close_impl(JanetStream *stream) {
Redo async connect code to be moved out of ev.c. Async connect is different than write.
2023-09-24 17:08:40 +00:00
stream->flags |= JANET_STREAM_CLOSED;
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
#ifdef JANET_WINDOWS
Work on debugging issue with server spawning.
2023-09-24 22:32:34 +00:00
if (stream->handle != INVALID_HANDLE_VALUE) {
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
#ifdef JANET_NET
Work on debugging issue with server spawning.
2023-09-24 22:32:34 +00:00
if (stream->flags & JANET_STREAM_SOCKET) {
closesocket((SOCKET) stream->handle);
} else
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
#endif
Work on debugging issue with server spawning.
2023-09-24 22:32:34 +00:00
{
CloseHandle(stream->handle);
}
stream->handle = INVALID_HANDLE_VALUE;
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
}
#else
Work on debugging issue with server spawning.
2023-09-24 22:32:34 +00:00
if (stream->handle != -1) {
close(stream->handle);
stream->handle = -1;
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
#ifdef JANET_EV_POLL
uint32_t i = stream->index;
size_t j = janet_vm.stream_count - 1;
JanetStream *last = janet_vm.streams[j];
struct pollfd lastfd = janet_vm.fds[j + 1];
janet_vm.fds[i + 1] = lastfd;
janet_vm.streams[i] = last;
last->index = stream->index;
janet_vm.stream_count--;
#endif
Work on debugging issue with server spawning.
2023-09-24 22:32:34 +00:00
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
#endif
}
Redo state management for Janet listeners. Make more use of the built in GC code for abstracts to be sure things are more correct. Issue before was streams could be freed before IOCP events arrived.
2023-09-25 07:43:36 +00:00
void janet_stream_close(JanetStream *stream) {
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
JanetFiber *rf = stream->read_fiber;
JanetFiber *wf = stream->write_fiber;
if (rf && rf->ev_callback) {
rf->ev_callback(rf, JANET_ASYNC_EVENT_CLOSE);
stream->read_fiber = NULL;
Redo state management for Janet listeners. Make more use of the built in GC code for abstracts to be sure things are more correct. Issue before was streams could be freed before IOCP events arrived.
2023-09-25 07:43:36 +00:00
}
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
if (wf && wf->ev_callback) {
wf->ev_callback(wf, JANET_ASYNC_EVENT_CLOSE);
stream->write_fiber = NULL;
Redo state management for Janet listeners. Make more use of the built in GC code for abstracts to be sure things are more correct. Issue before was streams could be freed before IOCP events arrived.
2023-09-25 07:43:36 +00:00
}
janet_stream_close_impl(stream);
}
Prevent possible bad garbage collection when finalizing streams. The GC finalizer for streams would sometimes try and use other objects after they had already been freed.
2021-07-21 00:42:35 +00:00
/* Called to clean up a stream */
static int janet_stream_gc(void *p, size_t s) {
(void) s;
JanetStream *stream = (JanetStream *)p;
Redo state management for Janet listeners. Make more use of the built in GC code for abstracts to be sure things are more correct. Issue before was streams could be freed before IOCP events arrived.
2023-09-25 07:43:36 +00:00
janet_stream_close_impl(stream);
Prevent possible bad garbage collection when finalizing streams. The GC finalizer for streams would sometimes try and use other objects after they had already been freed.
2021-07-21 00:42:35 +00:00
return 0;
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
/* Mark a stream for GC */
static int janet_stream_mark(void *p, size_t s) {
(void) s;
JanetStream *stream = (JanetStream *) p;
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
JanetFiber *rf = stream->read_fiber;
JanetFiber *wf = stream->write_fiber;
if (rf) {
janet_mark(janet_wrap_fiber(rf));
Redo state management for Janet listeners. Make more use of the built in GC code for abstracts to be sure things are more correct. Issue before was streams could be freed before IOCP events arrived.
2023-09-25 07:43:36 +00:00
}
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
if (wf) {
janet_mark(janet_wrap_fiber(wf));
Start with ev module.
2020-05-28 15:39:40 +00:00
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
return 0;
Start with ev module.
2020-05-28 15:39:40 +00:00
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
static int janet_stream_getter(void *p, Janet key, Janet *out) {
JanetStream *stream = (JanetStream *)p;
if (!janet_checktype(key, JANET_KEYWORD)) return 0;
const JanetMethod *stream_methods = stream->methods;
return janet_getmethod(janet_unwrap_keyword(key), stream_methods, out);
Start with ev module.
2020-05-28 15:39:40 +00:00
}
Fix buffer with NULL data pointer issue. Simply prevent buffers from ever having a NULL data pointer.
2020-12-13 15:33:05 +00:00
static void janet_stream_marshal(void *p, JanetMarshalContext *ctx) {
JanetStream *s = p;
if (!(ctx->flags & JANET_MARSHAL_UNSAFE)) {
janet_panic("can only marshal stream with unsafe flag");
}
janet_marshal_abstract(ctx, p);
janet_marshal_int(ctx, (int32_t) s->flags);
Squashed commit of the following: commit fbb0711ae1bb8bf1cc3738c46682b96938c50f78 Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 12:07:55 2023 -0500 Distinguish between subprocess when testing. commit 676b233566fa8fdb90af9ff801c29d7b4703c255 Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 11:59:17 2023 -0500 Hack for qemu based testing (also should work with valgrind) commit d7431c7cdbf0509ebe3e42888189dfe3cf6c7910 Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 11:54:04 2023 -0500 Revert "Test removing 32bit ptr marshalling." This reverts commit 566b45ea443d1d1c9b0bc6c345c4c33b3e07ed0e. commit 566b45ea443d1d1c9b0bc6c345c4c33b3e07ed0e Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 11:52:22 2023 -0500 Test removing 32bit ptr marshalling. commit ff2f71d2bca868206bee1923dcc8cd3ae5ec066e Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 11:42:10 2023 -0500 Conditionally compile marshal_ptr code. commit bd420aeb0e51b4905fb7976fc379943cb55dc777 Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 11:38:34 2023 -0500 Add range checking to bit-shift code to prevent undefined behavior. commit b738319f8d4037dba639da1a310b52a441e4ba34 Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 11:17:30 2023 -0500 Remove range check on 32 bit arch since it will always pass. commit 72486262357aef3a5eaa4652e6288328c381ea7f Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 10:56:45 2023 -0500 Quiet some build warnings. commit 141c1de946ff8376de6ecff3534e875fff047928 Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 10:50:13 2023 -0500 Add marshal utilities for pointers. commit c2d77d67207b1d4e71cab47a3b12ac27f801e72c Merge: 677b8a6f ff90b81e Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 10:40:35 2023 -0500 Merge branch 'master' into armtest commit 677b8a6f320e9170ea047fea9af74602881c4659 Author: Ico Doornekamp <ico@zevv.nl> Date: Mon Jun 12 21:01:26 2023 +0200 Added ARM32 test
2023-06-24 17:13:51 +00:00
janet_marshal_ptr(ctx, s->methods);
Fix buffer with NULL data pointer issue. Simply prevent buffers from ever having a NULL data pointer.
2020-12-13 15:33:05 +00:00
#ifdef JANET_WINDOWS
/* TODO - ref counting to avoid situation where a handle is closed or GCed
* while in transit, and it's value gets reused. DuplicateHandle does not work
More complete fix with some debugging tools.
2023-07-02 18:04:42 +00:00
* for network sockets, and in general for winsock it is better to not duplicate
Fix buffer with NULL data pointer issue. Simply prevent buffers from ever having a NULL data pointer.
2020-12-13 15:33:05 +00:00
* unless there is a need to. */
Update for sending streams to new threads.
2021-01-24 22:48:46 +00:00
HANDLE duph = INVALID_HANDLE_VALUE;
if (s->flags & JANET_STREAM_SOCKET) {
duph = s->handle;
} else {
DuplicateHandle(
Simpler overflow check.
2021-01-30 02:11:38 +00:00
GetCurrentProcess(),
s->handle,
GetCurrentProcess(),
&duph,
0,
FALSE,
DUPLICATE_SAME_ACCESS);
Update for sending streams to new threads.
2021-01-24 22:48:46 +00:00
}
janet_marshal_int64(ctx, (int64_t)(duph));
Fix buffer with NULL data pointer issue. Simply prevent buffers from ever having a NULL data pointer.
2020-12-13 15:33:05 +00:00
#else
/* Marshal after dup becuse it is easier than maintaining our own ref counting. */
int duph = dup(s->handle);
if (duph < 0) janet_panicf("failed to duplicate stream handle: %V", janet_ev_lasterr());
janet_marshal_int(ctx, (int32_t)(duph));
#endif
}
static void *janet_stream_unmarshal(JanetMarshalContext *ctx) {
if (!(ctx->flags & JANET_MARSHAL_UNSAFE)) {
janet_panic("can only unmarshal stream with unsafe flag");
}
JanetStream *p = janet_unmarshal_abstract(ctx, sizeof(JanetStream));
/* Can't share listening state and such across threads */
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
p->read_fiber = NULL;
p->write_fiber = NULL;
Fix buffer with NULL data pointer issue. Simply prevent buffers from ever having a NULL data pointer.
2020-12-13 15:33:05 +00:00
p->flags = (uint32_t) janet_unmarshal_int(ctx);
Squashed commit of the following: commit fbb0711ae1bb8bf1cc3738c46682b96938c50f78 Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 12:07:55 2023 -0500 Distinguish between subprocess when testing. commit 676b233566fa8fdb90af9ff801c29d7b4703c255 Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 11:59:17 2023 -0500 Hack for qemu based testing (also should work with valgrind) commit d7431c7cdbf0509ebe3e42888189dfe3cf6c7910 Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 11:54:04 2023 -0500 Revert "Test removing 32bit ptr marshalling." This reverts commit 566b45ea443d1d1c9b0bc6c345c4c33b3e07ed0e. commit 566b45ea443d1d1c9b0bc6c345c4c33b3e07ed0e Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 11:52:22 2023 -0500 Test removing 32bit ptr marshalling. commit ff2f71d2bca868206bee1923dcc8cd3ae5ec066e Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 11:42:10 2023 -0500 Conditionally compile marshal_ptr code. commit bd420aeb0e51b4905fb7976fc379943cb55dc777 Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 11:38:34 2023 -0500 Add range checking to bit-shift code to prevent undefined behavior. commit b738319f8d4037dba639da1a310b52a441e4ba34 Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 11:17:30 2023 -0500 Remove range check on 32 bit arch since it will always pass. commit 72486262357aef3a5eaa4652e6288328c381ea7f Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 10:56:45 2023 -0500 Quiet some build warnings. commit 141c1de946ff8376de6ecff3534e875fff047928 Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 10:50:13 2023 -0500 Add marshal utilities for pointers. commit c2d77d67207b1d4e71cab47a3b12ac27f801e72c Merge: 677b8a6f ff90b81e Author: Calvin Rose <calsrose@gmail.com> Date: Sat Jun 24 10:40:35 2023 -0500 Merge branch 'master' into armtest commit 677b8a6f320e9170ea047fea9af74602881c4659 Author: Ico Doornekamp <ico@zevv.nl> Date: Mon Jun 12 21:01:26 2023 +0200 Added ARM32 test
2023-06-24 17:13:51 +00:00
p->methods = janet_unmarshal_ptr(ctx);
Fix buffer with NULL data pointer issue. Simply prevent buffers from ever having a NULL data pointer.
2020-12-13 15:33:05 +00:00
#ifdef JANET_WINDOWS
p->handle = (JanetHandle) janet_unmarshal_int64(ctx);
#else
p->handle = (JanetHandle) janet_unmarshal_int(ctx);
#endif
return p;
}
Allow iterating through the properties of core abstract types.
2021-01-12 05:14:07 +00:00
static Janet janet_stream_next(void *p, Janet key) {
JanetStream *stream = (JanetStream *)p;
return janet_nextmethod(stream->methods, key);
}
Fix buffer with NULL data pointer issue. Simply prevent buffers from ever having a NULL data pointer.
2020-12-13 15:33:05 +00:00
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
const JanetAbstractType janet_stream_type = {
"core/stream",
janet_stream_gc,
janet_stream_mark,
janet_stream_getter,
Fix buffer with NULL data pointer issue. Simply prevent buffers from ever having a NULL data pointer.
2020-12-13 15:33:05 +00:00
NULL,
janet_stream_marshal,
janet_stream_unmarshal,
Allow iterating through the properties of core abstract types.
2021-01-12 05:14:07 +00:00
NULL,
NULL,
NULL,
janet_stream_next,
JANET_ATEND_NEXT
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
};
Start with ev module.
2020-05-28 15:39:40 +00:00
/* Register a fiber to resume with value */
Allow multiple simultaneous interrupts cleanly for #1262 Instead of setting a flag, each interrupt increments an atomic counter. When the interrupt is finally handled, either by scheduling code to run on the event loop or executing some out of band code, the user must now decrement the interrupt counter with janet_interpreter_interrupt_handled. While this counter is non-zero, the event loop will not enter the interpreter. This changes the API a bit but makes it possible and easy to handle signals without race conditions or scheduler hacks, as the runtime can ensure that high priority code is run before re-entering possibly blocking interpreter code again. Also included is a new function janet_schedule_soon, which prepends to the task queue instead of appending, allowing interrupt handler to skip ahead of all other scheduled fibers. Lastly, also update meson default options to include the interpreter_interrupt code and raise a runtime error if os/sigaction is used with interpreter interrupt but that build option is not enabled.
2023-08-24 01:14:38 +00:00
static void janet_schedule_general(JanetFiber *fiber, Janet value, JanetSignal sig, int soon) {
Fix #892 - Remove racy ref counts for channels Rather than manual reference counting for suspended fibers, we automate the process by incrementing "extra_listeners" every time we suspend a fiber in the event loop, and decrement when that fiber is resumed. In this manner, we keep track of the number of suspending fibers in a simpler, more correct way.
2021-12-10 00:44:55 +00:00
if (fiber->gc.flags & JANET_FIBER_EV_FLAG_CANCELED) return;
Add `ev/all-tasks` to get running and pending root fibers.
2023-04-01 23:57:13 +00:00
if (!(fiber->gc.flags & JANET_FIBER_FLAG_ROOT)) {
Janet task_element = janet_wrap_fiber(fiber);
janet_table_put(&janet_vm.active_tasks, task_element, janet_wrap_true());
}
Address #820 - ev/cancel to work on already scheduled fibers.
2021-09-28 20:41:41 +00:00
JanetTask t = { fiber, value, sig, ++fiber->sched_id };
Add `ev/all-tasks` to get running and pending root fibers.
2023-04-01 23:57:13 +00:00
fiber->gc.flags |= JANET_FIBER_FLAG_ROOT;
Fix #892 - Remove racy ref counts for channels Rather than manual reference counting for suspended fibers, we automate the process by incrementing "extra_listeners" every time we suspend a fiber in the event loop, and decrement when that fiber is resumed. In this manner, we keep track of the number of suspending fibers in a simpler, more correct way.
2021-12-10 00:44:55 +00:00
if (sig == JANET_SIGNAL_ERROR) fiber->gc.flags |= JANET_FIBER_EV_FLAG_CANCELED;
Allow multiple simultaneous interrupts cleanly for #1262 Instead of setting a flag, each interrupt increments an atomic counter. When the interrupt is finally handled, either by scheduling code to run on the event loop or executing some out of band code, the user must now decrement the interrupt counter with janet_interpreter_interrupt_handled. While this counter is non-zero, the event loop will not enter the interpreter. This changes the API a bit but makes it possible and easy to handle signals without race conditions or scheduler hacks, as the runtime can ensure that high priority code is run before re-entering possibly blocking interpreter code again. Also included is a new function janet_schedule_soon, which prepends to the task queue instead of appending, allowing interrupt handler to skip ahead of all other scheduled fibers. Lastly, also update meson default options to include the interpreter_interrupt code and raise a runtime error if os/sigaction is used with interpreter interrupt but that build option is not enabled.
2023-08-24 01:14:38 +00:00
if (soon) {
janet_q_push_head(&janet_vm.spawn, &t, sizeof(t));
} else {
janet_q_push(&janet_vm.spawn, &t, sizeof(t));
}
}
void janet_schedule_signal(JanetFiber *fiber, Janet value, JanetSignal sig) {
janet_schedule_general(fiber, value, sig, 0);
}
void janet_schedule_soon(JanetFiber *fiber, Janet value, JanetSignal sig) {
janet_schedule_general(fiber, value, sig, 1);
Start with ev module.
2020-05-28 15:39:40 +00:00
}
Merge branch 'master' into ev
2020-08-16 22:29:57 +00:00
void janet_cancel(JanetFiber *fiber, Janet value) {
janet_schedule_signal(fiber, value, JANET_SIGNAL_ERROR);
}
void janet_schedule(JanetFiber *fiber, Janet value) {
janet_schedule_signal(fiber, value, JANET_SIGNAL_OK);
}
Start with ev module.
2020-05-28 15:39:40 +00:00
/* Mark all pending tasks */
void janet_ev_mark(void) {
Don't use gcroot/unroot for tracking IO operations. This could have bad effects in higher load situations, and duplicates code. It is better to keep a dedicated list of scheduled IO operations which can be efficiently added and removed from. It also provides and easy way to enumerate scheduled IO operations.
2020-11-16 15:30:04 +00:00
/* Pending tasks */
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
JanetTask *tasks = janet_vm.spawn.data;
if (janet_vm.spawn.head <= janet_vm.spawn.tail) {
for (int32_t i = janet_vm.spawn.head; i < janet_vm.spawn.tail; i++) {
Use a common queue implementation. Queues occur in three places, so we use a single implementation rather than three separate ones. This also has the result that janet_vm_spawn will not overflow in the case of channel-heavy, IO-light operation.
2020-08-01 19:20:58 +00:00
janet_mark(janet_wrap_fiber(tasks[i].fiber));
janet_mark(tasks[i].value);
}
} else {
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
for (int32_t i = janet_vm.spawn.head; i < janet_vm.spawn.capacity; i++) {
Use a common queue implementation. Queues occur in three places, so we use a single implementation rather than three separate ones. This also has the result that janet_vm_spawn will not overflow in the case of channel-heavy, IO-light operation.
2020-08-01 19:20:58 +00:00
janet_mark(janet_wrap_fiber(tasks[i].fiber));
janet_mark(tasks[i].value);
}
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
for (int32_t i = 0; i < janet_vm.spawn.tail; i++) {
Use a common queue implementation. Queues occur in three places, so we use a single implementation rather than three separate ones. This also has the result that janet_vm_spawn will not overflow in the case of channel-heavy, IO-light operation.
2020-08-01 19:20:58 +00:00
janet_mark(janet_wrap_fiber(tasks[i].fiber));
janet_mark(tasks[i].value);
}
Start with ev module.
2020-05-28 15:39:40 +00:00
}
Don't use gcroot/unroot for tracking IO operations. This could have bad effects in higher load situations, and duplicates code. It is better to keep a dedicated list of scheduled IO operations which can be efficiently added and removed from. It also provides and easy way to enumerate scheduled IO operations.
2020-11-16 15:30:04 +00:00
/* Pending timeouts */
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
for (size_t i = 0; i < janet_vm.tq_count; i++) {
janet_mark(janet_wrap_fiber(janet_vm.tq[i].fiber));
if (janet_vm.tq[i].curr_fiber != NULL) {
janet_mark(janet_wrap_fiber(janet_vm.tq[i].curr_fiber));
Add ev/deadline and ev/with-deadline. This should be more useful than timeouts in real-world use cases. The deadline system is based on fibers and is target to much more coarse-grained (and therfor reliable) timeouts than things like ev/sleep and timeout arguments.
2020-12-05 16:32:34 +00:00
}
Start with ev module.
2020-05-28 15:39:40 +00:00
}
}
Add a "new_channel" for root fibers. When new fibers are scheduled on the event loop, this new_channel receives the newly created fibers. This lets a fiber track which fibers have been added and let's a user implement a supervisor. Fix formatting.
2021-01-10 00:33:40 +00:00
static int janet_channel_push(JanetChannel *channel, Janet x, int mode);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
static int janet_channel_pop(JanetChannel *channel, Janet *item, int is_choice);
Add initial implementation for supervisor channels. Supervisor channels are a simple concept to more efficiently enable dynamic, structure concurrency. When a top-level fiber completes (or errors), it will push itself to it's supervisor channel if it has one (instead of printing a stacktrace). This let's another fiber poll a channel and "supervise" a set of fibers.
2021-01-07 05:19:22 +00:00
Allow passing function to ev/thread. Convenient when there is no need to create an entire fiber.
2021-08-31 03:04:15 +00:00
static Janet make_supervisor_event(const char *name, JanetFiber *fiber, int threaded) {
Add support for a dyn :task-id Adds extra information to default information from supervisor channels. For threaded channels as supervisors, we don't get the source fiber so identifying the source of messages was not possible. This change allows better multithreading with supervisors.
2022-06-25 23:15:58 +00:00
Janet tup[3];
Go back to a single supervisor channel per fiber. We now also use the fiber mask to figure out which flags to wait for.
2021-01-13 03:35:28 +00:00
tup[0] = janet_ckeywordv(name);
Threaded supervisors return fiber->last_value instead of the fiber itself.
2021-08-31 19:50:27 +00:00
tup[1] = threaded ? fiber->last_value : janet_wrap_fiber(fiber) ;
Add support for a dyn :task-id Adds extra information to default information from supervisor channels. For threaded channels as supervisors, we don't get the source fiber so identifying the source of messages was not possible. This change allows better multithreading with supervisors.
2022-06-25 23:15:58 +00:00
if (fiber->env != NULL) {
tup[2] = janet_table_get(fiber->env, janet_ckeywordv("task-id"));
} else {
tup[2] = janet_wrap_nil();
}
return janet_wrap_tuple(janet_tuple_n(tup, 3));
Go back to a single supervisor channel per fiber. We now also use the fiber mask to figure out which flags to wait for.
2021-01-13 03:35:28 +00:00
}
Start with ev module.
2020-05-28 15:39:40 +00:00
/* Common init code */
void janet_ev_init_common(void) {
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
janet_q_init(&janet_vm.spawn);
janet_vm.tq = NULL;
janet_vm.tq_count = 0;
janet_vm.tq_capacity = 0;
Initial threaded abstract types.
2021-08-20 01:56:48 +00:00
janet_table_init_raw(&janet_vm.threaded_abstracts, 0);
Add `ev/all-tasks` to get running and pending root fibers.
2023-04-01 23:57:13 +00:00
janet_table_init_raw(&janet_vm.active_tasks, 0);
Add os/sigaction for signal handling. Also improve interrupts to work better with busy loops and signals.
2023-08-19 18:13:22 +00:00
janet_table_init_raw(&janet_vm.signal_handlers, 0);
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
janet_rng_seed(&janet_vm.ev_rng, 0);
Create pthread threads with detached attribute. Rather than calling pthread_detach on a default thread.
2022-11-11 17:01:59 +00:00
#ifndef JANET_WINDOWS
pthread_attr_init(&janet_vm.new_thread_attr);
pthread_attr_setdetachstate(&janet_vm.new_thread_attr, PTHREAD_CREATE_DETACHED);
#endif
Start with ev module.
2020-05-28 15:39:40 +00:00
}
/* Common deinit code */
void janet_ev_deinit_common(void) {
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
janet_q_deinit(&janet_vm.spawn);
janet_free(janet_vm.tq);
Initial threaded abstract types.
2021-08-20 01:56:48 +00:00
janet_table_deinit(&janet_vm.threaded_abstracts);
Add `ev/all-tasks` to get running and pending root fibers.
2023-04-01 23:57:13 +00:00
janet_table_deinit(&janet_vm.active_tasks);
Add os/sigaction for signal handling. Also improve interrupts to work better with busy loops and signals.
2023-08-19 18:13:22 +00:00
janet_table_deinit(&janet_vm.signal_handlers);
Create pthread threads with detached attribute. Rather than calling pthread_detach on a default thread.
2022-11-11 17:01:59 +00:00
#ifndef JANET_WINDOWS
pthread_attr_destroy(&janet_vm.new_thread_attr);
#endif
Start with ev module.
2020-05-28 15:39:40 +00:00
}
Add `ev/all-tasks` to get running and pending root fibers.
2023-04-01 23:57:13 +00:00
/* Shorthand to yield to event loop */
Start with ev module.
2020-05-28 15:39:40 +00:00
void janet_await(void) {
Add `ev/all-tasks` to get running and pending root fibers.
2023-04-01 23:57:13 +00:00
/* Store the fiber in a gobal table */
Start with ev module.
2020-05-28 15:39:40 +00:00
janet_signalv(JANET_SIGNAL_EVENT, janet_wrap_nil());
}
More work on timeouts and racing listeners. When two listeners are racing to resume the same fiber, the first should cancel out the other.
2020-07-05 22:26:17 +00:00
/* Set timeout for the current root fiber */
void janet_addtimeout(double sec) {
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
JanetFiber *fiber = janet_vm.root_fiber;
More work on timeouts and racing listeners. When two listeners are racing to resume the same fiber, the first should cancel out the other.
2020-07-05 22:26:17 +00:00
JanetTimeout to;
to.when = ts_delta(ts_now(), sec);
to.fiber = fiber;
Add ev/deadline and ev/with-deadline. This should be more useful than timeouts in real-world use cases. The deadline system is based on fibers and is target to much more coarse-grained (and therfor reliable) timeouts than things like ev/sleep and timeout arguments.
2020-12-05 16:32:34 +00:00
to.curr_fiber = NULL;
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
to.sched_id = fiber->sched_id;
Merge branch 'master' into ev
2020-08-16 22:29:57 +00:00
to.is_error = 1;
More work on timeouts and racing listeners. When two listeners are racing to resume the same fiber, the first should cancel out the other.
2020-07-05 22:26:17 +00:00
add_timeout(to);
}
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
void janet_ev_inc_refcount(void) {
Get rid of req for 64 bit atomics
2023-10-01 15:27:51 +00:00
janet_atomic_inc(&janet_vm.listener_count);
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
}
void janet_ev_dec_refcount(void) {
Get rid of req for 64 bit atomics
2023-10-01 15:27:51 +00:00
janet_atomic_dec(&janet_vm.listener_count);
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
}
Begin work on channels.
2020-07-27 04:45:25 +00:00
/* Channels */
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
#define JANET_MAX_CHANNEL_CAPACITY 0xFFFFFF
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
static inline int janet_chan_is_threaded(JanetChannel *chan) {
Add basic runtime support for threaded abstracts. A threaded abstract is an abstract type that can be freely shared between threads. While no synchronization is provided, refcounting and transport between threads is. This will let implementers more easily exploit OS-level parallelism in C library code. The caveat with these types is that they need to be careful in how they interact with objects on other heaps.
2021-08-20 02:16:20 +00:00
return chan->is_threaded;
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
}
static int janet_chan_pack(JanetChannel *chan, Janet *x) {
Add basic runtime support for threaded abstracts. A threaded abstract is an abstract type that can be freely shared between threads. While no synchronization is provided, refcounting and transport between threads is. This will let implementers more easily exploit OS-level parallelism in C library code. The caveat with these types is that they need to be careful in how they interact with objects on other heaps.
2021-08-20 02:16:20 +00:00
if (!janet_chan_is_threaded(chan)) return 0;
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
switch (janet_type(*x)) {
Add marshal/unmarshal to items pushed to threaded channel.
2021-08-15 21:12:40 +00:00
default: {
JanetBuffer *buf = janet_malloc(sizeof(JanetBuffer));
if (NULL == buf) {
JANET_OUT_OF_MEMORY;
}
janet_buffer_init(buf, 10);
janet_marshal(buf, *x, NULL, JANET_MARSHAL_UNSAFE);
*x = janet_wrap_buffer(buf);
return 0;
}
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
case JANET_NIL:
case JANET_NUMBER:
case JANET_POINTER:
case JANET_BOOLEAN:
case JANET_CFUNCTION:
return 0;
}
}
Decrement thread channel pointer during cleanup without adding to heap.
2021-08-20 21:41:19 +00:00
static int janet_chan_unpack(JanetChannel *chan, Janet *x, int is_cleanup) {
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
if (!janet_chan_is_threaded(chan)) return 0;
switch (janet_type(*x)) {
default:
return 1;
Add marshal/unmarshal to items pushed to threaded channel.
2021-08-15 21:12:40 +00:00
case JANET_BUFFER: {
JanetBuffer *buf = janet_unwrap_buffer(*x);
Fix channel packing bug.
2021-12-04 16:28:00 +00:00
int flags = is_cleanup ? (JANET_MARSHAL_UNSAFE | JANET_MARSHAL_DECREF) : JANET_MARSHAL_UNSAFE;
Decrement thread channel pointer during cleanup without adding to heap.
2021-08-20 21:41:19 +00:00
*x = janet_unmarshal(buf->data, buf->count, flags, NULL, NULL);
Add marshal/unmarshal to items pushed to threaded channel.
2021-08-15 21:12:40 +00:00
janet_buffer_deinit(buf);
janet_free(buf);
return 0;
}
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
case JANET_NIL:
case JANET_NUMBER:
case JANET_POINTER:
case JANET_BOOLEAN:
case JANET_CFUNCTION:
return 0;
}
}
static void janet_chan_init(JanetChannel *chan, int32_t limit, int threaded) {
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
chan->limit = limit;
Add semantics for closing channels. This makes certain algorithms simpler as channels now have an explicit lifetime - multiple readers can coordinate closing without needing to ensure the same number of reads as writes.
2021-07-31 00:26:42 +00:00
chan->closed = 0;
Add basic runtime support for threaded abstracts. A threaded abstract is an abstract type that can be freely shared between threads. While no synchronization is provided, refcounting and transport between threads is. This will let implementers more easily exploit OS-level parallelism in C library code. The caveat with these types is that they need to be careful in how they interact with objects on other heaps.
2021-08-20 02:16:20 +00:00
chan->is_threaded = threaded;
Use a common queue implementation. Queues occur in three places, so we use a single implementation rather than three separate ones. This also has the result that janet_vm_spawn will not overflow in the case of channel-heavy, IO-light operation.
2020-08-01 19:20:58 +00:00
janet_q_init(&chan->items);
janet_q_init(&chan->read_pending);
janet_q_init(&chan->write_pending);
Remove pthread.h from janet.h Should make janet a bit easier to use. Also changes the header to not expose the size of native mutexes and rwlocks, except with janet_os_mutex_size and janet_os_rwlock_size.
2022-06-17 22:13:58 +00:00
janet_os_mutex_init((JanetOSMutex *) &chan->lock);
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
}
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
static void janet_chan_lock(JanetChannel *chan) {
if (!janet_chan_is_threaded(chan)) return;
Remove pthread.h from janet.h Should make janet a bit easier to use. Also changes the header to not expose the size of native mutexes and rwlocks, except with janet_os_mutex_size and janet_os_rwlock_size.
2022-06-17 22:13:58 +00:00
janet_os_mutex_lock((JanetOSMutex *) &chan->lock);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
}
static void janet_chan_unlock(JanetChannel *chan) {
if (!janet_chan_is_threaded(chan)) return;
Remove pthread.h from janet.h Should make janet a bit easier to use. Also changes the header to not expose the size of native mutexes and rwlocks, except with janet_os_mutex_size and janet_os_rwlock_size.
2022-06-17 22:13:58 +00:00
janet_os_mutex_unlock((JanetOSMutex *) &chan->lock);
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
}
Address #1125 - fix ev/select to only take and release locks once. By take and releasing locks twice per channel in the case where nothing is reading, there was an opportunity for ev/select to hang in the multithreaded case. Also silence valgrind/helgrind errors.
2023-05-07 16:52:11 +00:00
static void janet_chan_deinit(JanetChannel *chan) {
if (janet_chan_is_threaded(chan)) {
Janet item;
janet_chan_lock(chan);
janet_q_deinit(&chan->read_pending);
janet_q_deinit(&chan->write_pending);
while (!janet_q_pop(&chan->items, &item, sizeof(item))) {
janet_chan_unpack(chan, &item, 1);
}
janet_q_deinit(&chan->items);
janet_chan_unlock(chan);
} else {
janet_q_deinit(&chan->read_pending);
janet_q_deinit(&chan->write_pending);
janet_q_deinit(&chan->items);
}
janet_os_mutex_deinit((JanetOSMutex *) &chan->lock);
}
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
/*
* Janet Channel abstract type
*/
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
static Janet janet_wrap_channel(JanetChannel *channel) {
Add basic runtime support for threaded abstracts. A threaded abstract is an abstract type that can be freely shared between threads. While no synchronization is provided, refcounting and transport between threads is. This will let implementers more easily exploit OS-level parallelism in C library code. The caveat with these types is that they need to be careful in how they interact with objects on other heaps.
2021-08-20 02:16:20 +00:00
return janet_wrap_abstract(channel);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
}
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
static int janet_chanat_gc(void *p, size_t s) {
Add basic runtime support for threaded abstracts. A threaded abstract is an abstract type that can be freely shared between threads. While no synchronization is provided, refcounting and transport between threads is. This will let implementers more easily exploit OS-level parallelism in C library code. The caveat with these types is that they need to be careful in how they interact with objects on other heaps.
2021-08-20 02:16:20 +00:00
(void) s;
JanetChannel *channel = p;
janet_chan_deinit(channel);
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
return 0;
}
Use a common queue implementation. Queues occur in three places, so we use a single implementation rather than three separate ones. This also has the result that janet_vm_spawn will not overflow in the case of channel-heavy, IO-light operation.
2020-08-01 19:20:58 +00:00
static void janet_chanat_mark_fq(JanetQueue *fq) {
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
JanetChannelPending *pending = fq->data;
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
if (fq->head <= fq->tail) {
for (int32_t i = fq->head; i < fq->tail; i++)
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
janet_mark(janet_wrap_fiber(pending[i].fiber));
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
} else {
for (int32_t i = fq->head; i < fq->capacity; i++)
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
janet_mark(janet_wrap_fiber(pending[i].fiber));
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
for (int32_t i = 0; i < fq->tail; i++)
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
janet_mark(janet_wrap_fiber(pending[i].fiber));
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
}
}
static int janet_chanat_mark(void *p, size_t s) {
Add basic runtime support for threaded abstracts. A threaded abstract is an abstract type that can be freely shared between threads. While no synchronization is provided, refcounting and transport between threads is. This will let implementers more easily exploit OS-level parallelism in C library code. The caveat with these types is that they need to be careful in how they interact with objects on other heaps.
2021-08-20 02:16:20 +00:00
(void) s;
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
JanetChannel *chan = p;
janet_chanat_mark_fq(&chan->read_pending);
janet_chanat_mark_fq(&chan->write_pending);
Use a common queue implementation. Queues occur in three places, so we use a single implementation rather than three separate ones. This also has the result that janet_vm_spawn will not overflow in the case of channel-heavy, IO-light operation.
2020-08-01 19:20:58 +00:00
JanetQueue *items = &chan->items;
Janet *data = chan->items.data;
if (items->head <= items->tail) {
for (int32_t i = items->head; i < items->tail; i++)
janet_mark(data[i]);
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
} else {
Use a common queue implementation. Queues occur in three places, so we use a single implementation rather than three separate ones. This also has the result that janet_vm_spawn will not overflow in the case of channel-heavy, IO-light operation.
2020-08-01 19:20:58 +00:00
for (int32_t i = items->head; i < items->capacity; i++)
janet_mark(data[i]);
for (int32_t i = 0; i < items->tail; i++)
janet_mark(data[i]);
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
}
return 0;
}
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
static Janet make_write_result(JanetChannel *channel) {
Janet *tup = janet_tuple_begin(2);
tup[0] = janet_ckeywordv("give");
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
tup[1] = janet_wrap_channel(channel);
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
return janet_wrap_tuple(janet_tuple_end(tup));
}
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
static Janet make_read_result(JanetChannel *channel, Janet x) {
Janet *tup = janet_tuple_begin(3);
tup[0] = janet_ckeywordv("take");
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
tup[1] = janet_wrap_channel(channel);
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
tup[2] = x;
return janet_wrap_tuple(janet_tuple_end(tup));
}
Add semantics for closing channels. This makes certain algorithms simpler as channels now have an explicit lifetime - multiple readers can coordinate closing without needing to ensure the same number of reads as writes.
2021-07-31 00:26:42 +00:00
static Janet make_close_result(JanetChannel *channel) {
Janet *tup = janet_tuple_begin(2);
tup[0] = janet_ckeywordv("close");
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
tup[1] = janet_wrap_channel(channel);
Add semantics for closing channels. This makes certain algorithms simpler as channels now have an explicit lifetime - multiple readers can coordinate closing without needing to ensure the same number of reads as writes.
2021-07-31 00:26:42 +00:00
return janet_wrap_tuple(janet_tuple_end(tup));
}
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
/* Callback to use for scheduling a fiber from another thread. */
static void janet_thread_chan_cb(JanetEVGenericMessage msg) {
uint32_t sched_id = (uint32_t) msg.argi;
JanetFiber *fiber = msg.fiber;
int mode = msg.tag;
JanetChannel *channel = (JanetChannel *) msg.argp;
Janet x = msg.argj;
Address #1125 - fix ev/select to only take and release locks once. By take and releasing locks twice per channel in the case where nothing is reading, there was an opportunity for ev/select to hang in the multithreaded case. Also silence valgrind/helgrind errors.
2023-05-07 16:52:11 +00:00
janet_chan_lock(channel);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
if (fiber->sched_id == sched_id) {
if (mode == JANET_CP_MODE_CHOICE_READ) {
Decrement thread channel pointer during cleanup without adding to heap.
2021-08-20 21:41:19 +00:00
janet_assert(!janet_chan_unpack(channel, &x, 0), "packing error");
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
janet_schedule(fiber, make_read_result(channel, x));
} else if (mode == JANET_CP_MODE_CHOICE_WRITE) {
janet_schedule(fiber, make_write_result(channel));
} else if (mode == JANET_CP_MODE_READ) {
Decrement thread channel pointer during cleanup without adding to heap.
2021-08-20 21:41:19 +00:00
janet_assert(!janet_chan_unpack(channel, &x, 0), "packing error");
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
janet_schedule(fiber, x);
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
} else if (mode == JANET_CP_MODE_WRITE) {
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
janet_schedule(fiber, janet_wrap_channel(channel));
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
} else { /* (mode == JANET_CP_MODE_CLOSE) */
janet_schedule(fiber, janet_wrap_nil());
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
}
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
} else if (mode != JANET_CP_MODE_CLOSE) {
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
/* Fiber has already been cancelled or resumed. */
/* Resend event to another waiting thread, depending on mode */
int is_read = (mode == JANET_CP_MODE_CHOICE_READ) || (mode == JANET_CP_MODE_READ);
if (is_read) {
JanetChannelPending reader;
if (!janet_q_pop(&channel->read_pending, &reader, sizeof(reader))) {
JanetVM *vm = reader.thread;
JanetEVGenericMessage msg;
msg.tag = reader.mode;
msg.fiber = reader.fiber;
msg.argi = (int32_t) reader.sched_id;
msg.argp = channel;
msg.argj = x;
janet_ev_post_event(vm, janet_thread_chan_cb, msg);
}
} else {
JanetChannelPending writer;
if (!janet_q_pop(&channel->write_pending, &writer, sizeof(writer))) {
JanetVM *vm = writer.thread;
JanetEVGenericMessage msg;
msg.tag = writer.mode;
msg.fiber = writer.fiber;
msg.argi = (int32_t) writer.sched_id;
msg.argp = channel;
msg.argj = janet_wrap_nil();
janet_ev_post_event(vm, janet_thread_chan_cb, msg);
}
}
}
Address #1125 - fix ev/select to only take and release locks once. By take and releasing locks twice per channel in the case where nothing is reading, there was an opportunity for ev/select to hang in the multithreaded case. Also silence valgrind/helgrind errors.
2023-05-07 16:52:11 +00:00
janet_chan_unlock(channel);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
}
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
/* Push a value to a channel, and return 1 if channel should block, zero otherwise.
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
* If the push would block, will add to the write_pending queue in the channel.
* Handles both threaded and unthreaded channels. */
Address #1125 - fix ev/select to only take and release locks once. By take and releasing locks twice per channel in the case where nothing is reading, there was an opportunity for ev/select to hang in the multithreaded case. Also silence valgrind/helgrind errors.
2023-05-07 16:52:11 +00:00
static int janet_channel_push_with_lock(JanetChannel *channel, Janet x, int mode) {
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
JanetChannelPending reader;
int is_empty;
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
if (janet_chan_pack(channel, &x)) {
Address #1125 - fix ev/select to only take and release locks once. By take and releasing locks twice per channel in the case where nothing is reading, there was an opportunity for ev/select to hang in the multithreaded case. Also silence valgrind/helgrind errors.
2023-05-07 16:52:11 +00:00
janet_chan_unlock(channel);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
janet_panicf("failed to pack value for channel: %v", x);
}
if (channel->closed) {
janet_chan_unlock(channel);
janet_panic("cannot write to closed channel");
}
int is_threaded = janet_chan_is_threaded(channel);
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
if (is_threaded) {
/* don't dereference fiber from another thread */
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
is_empty = janet_q_pop(&channel->read_pending, &reader, sizeof(reader));
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
} else {
do {
is_empty = janet_q_pop(&channel->read_pending, &reader, sizeof(reader));
} while (!is_empty && (reader.sched_id != reader.fiber->sched_id));
}
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
if (is_empty) {
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
/* No pending reader */
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
if (janet_q_push(&channel->items, &x, sizeof(Janet))) {
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
janet_chan_unlock(channel);
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
janet_panicf("channel overflow: %v", x);
Use a common queue implementation. Queues occur in three places, so we use a single implementation rather than three separate ones. This also has the result that janet_vm_spawn will not overflow in the case of channel-heavy, IO-light operation.
2020-08-01 19:20:58 +00:00
} else if (janet_q_count(&channel->items) > channel->limit) {
Add initial implementation for supervisor channels. Supervisor channels are a simple concept to more efficiently enable dynamic, structure concurrency. When a top-level fiber completes (or errors), it will push itself to it's supervisor channel if it has one (instead of printing a stacktrace). This let's another fiber poll a channel and "supervise" a set of fibers.
2021-01-07 05:19:22 +00:00
/* No root fiber, we are in completion on a root fiber. Don't block. */
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
if (mode == 2) {
janet_chan_unlock(channel);
return 0;
}
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
/* Pushed successfully, but should block. */
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
JanetChannelPending pending;
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
pending.thread = &janet_vm;
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
pending.fiber = janet_vm.root_fiber,
pending.sched_id = janet_vm.root_fiber->sched_id,
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
pending.mode = mode ? JANET_CP_MODE_CHOICE_WRITE : JANET_CP_MODE_WRITE;
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
janet_q_push(&channel->write_pending, &pending, sizeof(pending));
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
janet_chan_unlock(channel);
if (is_threaded) {
janet_gcroot(janet_wrap_fiber(pending.fiber));
}
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
return 1;
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
}
} else {
/* Pending reader */
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
if (is_threaded) {
JanetVM *vm = reader.thread;
JanetEVGenericMessage msg;
msg.tag = reader.mode;
msg.fiber = reader.fiber;
msg.argi = (int32_t) reader.sched_id;
msg.argp = channel;
msg.argj = x;
janet_ev_post_event(vm, janet_thread_chan_cb, msg);
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
} else {
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
if (reader.mode == JANET_CP_MODE_CHOICE_READ) {
janet_schedule(reader.fiber, make_read_result(channel, x));
} else {
janet_schedule(reader.fiber, x);
}
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
}
}
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
janet_chan_unlock(channel);
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
return 0;
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
}
Address #1125 - fix ev/select to only take and release locks once. By take and releasing locks twice per channel in the case where nothing is reading, there was an opportunity for ev/select to hang in the multithreaded case. Also silence valgrind/helgrind errors.
2023-05-07 16:52:11 +00:00
static int janet_channel_push(JanetChannel *channel, Janet x, int mode) {
janet_chan_lock(channel);
return janet_channel_push_with_lock(channel, x, mode);
}
Add option to build janet without thread library.
2021-07-17 20:13:28 +00:00
/* Pop from a channel - returns 1 if item was obtained, 0 otherwise. The item
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
* is returned by reference. If the pop would block, will add to the read_pending
* queue in the channel. */
Address #1125 - fix ev/select to only take and release locks once. By take and releasing locks twice per channel in the case where nothing is reading, there was an opportunity for ev/select to hang in the multithreaded case. Also silence valgrind/helgrind errors.
2023-05-07 16:52:11 +00:00
static int janet_channel_pop_with_lock(JanetChannel *channel, Janet *item, int is_choice) {
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
JanetChannelPending writer;
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
if (channel->closed) {
janet_chan_unlock(channel);
*item = janet_wrap_nil();
return 1;
}
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
int is_threaded = janet_chan_is_threaded(channel);
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
if (janet_q_pop(&channel->items, item, sizeof(Janet))) {
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
/* Queue empty */
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
JanetChannelPending pending;
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
pending.thread = &janet_vm;
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
pending.fiber = janet_vm.root_fiber,
pending.sched_id = janet_vm.root_fiber->sched_id;
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
pending.mode = is_choice ? JANET_CP_MODE_CHOICE_READ : JANET_CP_MODE_READ;
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
janet_q_push(&channel->read_pending, &pending, sizeof(pending));
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
janet_chan_unlock(channel);
if (is_threaded) {
janet_gcroot(janet_wrap_fiber(pending.fiber));
}
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
return 0;
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
}
Decrement thread channel pointer during cleanup without adding to heap.
2021-08-20 21:41:19 +00:00
janet_assert(!janet_chan_unpack(channel, item, 0), "bad channel packing");
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
if (!janet_q_pop(&channel->write_pending, &writer, sizeof(writer))) {
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
/* Pending writer */
if (is_threaded) {
JanetVM *vm = writer.thread;
JanetEVGenericMessage msg;
msg.tag = writer.mode;
msg.fiber = writer.fiber;
msg.argi = (int32_t) writer.sched_id;
msg.argp = channel;
msg.argj = janet_wrap_nil();
janet_ev_post_event(vm, janet_thread_chan_cb, msg);
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
} else {
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
if (writer.mode == JANET_CP_MODE_CHOICE_WRITE) {
janet_schedule(writer.fiber, make_write_result(channel));
} else {
janet_schedule(writer.fiber, janet_wrap_abstract(channel));
}
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
}
}
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
janet_chan_unlock(channel);
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
return 1;
}
Address #1125 - fix ev/select to only take and release locks once. By take and releasing locks twice per channel in the case where nothing is reading, there was an opportunity for ev/select to hang in the multithreaded case. Also silence valgrind/helgrind errors.
2023-05-07 16:52:11 +00:00
static int janet_channel_pop(JanetChannel *channel, Janet *item, int is_choice) {
janet_chan_lock(channel);
return janet_channel_pop_with_lock(channel, item, is_choice);
}
Fix threaded supervisor channels - Fix #766 Some pointer casting with abstract types was incorrect, resulting in strange behavior when trying to use supervisor channels that were threaded. This fix also adds the ability to supply a supervisor channel directly when creating a thread.
2021-08-17 02:14:06 +00:00
JanetChannel *janet_channel_unwrap(void *abstract) {
Add basic runtime support for threaded abstracts. A threaded abstract is an abstract type that can be freely shared between threads. While no synchronization is provided, refcounting and transport between threads is. This will let implementers more easily exploit OS-level parallelism in C library code. The caveat with these types is that they need to be careful in how they interact with objects on other heaps.
2021-08-20 02:16:20 +00:00
return abstract;
Fix threaded supervisor channels - Fix #766 Some pointer casting with abstract types was incorrect, resulting in strange behavior when trying to use supervisor channels that were threaded. This fix also adds the ability to supply a supervisor channel directly when creating a thread.
2021-08-17 02:14:06 +00:00
}
JanetChannel *janet_getchannel(const Janet *argv, int32_t n) {
return janet_channel_unwrap(janet_getabstract(argv, n, &janet_channel_type));
}
JanetChannel *janet_optchannel(const Janet *argv, int32_t argc, int32_t n, JanetChannel *dflt) {
if (argc > n && !janet_checktype(argv[n], JANET_NIL)) {
return janet_getchannel(argv, n);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
} else {
Fix threaded supervisor channels - Fix #766 Some pointer casting with abstract types was incorrect, resulting in strange behavior when trying to use supervisor channels that were threaded. This fix also adds the ability to supply a supervisor channel directly when creating a thread.
2021-08-17 02:14:06 +00:00
return dflt;
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
}
}
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
/* Channel Methods */
Update ev.c to use the new binding style that provides source information
2021-07-26 05:25:38 +00:00
JANET_CORE_FN(cfun_channel_push,
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
"(ev/give channel value)",
Add semantics for closing channels. This makes certain algorithms simpler as channels now have an explicit lifetime - multiple readers can coordinate closing without needing to ensure the same number of reads as writes.
2021-07-31 00:26:42 +00:00
"Write a value to a channel, suspending the current fiber if the channel is full. "
"Returns the channel if the write succeeded, nil otherwise.") {
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
janet_fixarity(argc, 2);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
JanetChannel *channel = janet_getchannel(argv, 0);
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
if (janet_channel_push(channel, argv[1], 0)) {
janet_await();
}
return argv[0];
}
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
JANET_CORE_FN(cfun_channel_pop,
"(ev/take channel)",
"Read from a channel, suspending the current fiber if no value is available.") {
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
janet_fixarity(argc, 1);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
JanetChannel *channel = janet_getchannel(argv, 0);
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
Janet item;
if (janet_channel_pop(channel, &item, 0)) {
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
janet_schedule(janet_vm.root_fiber, item);
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
}
janet_await();
}
Address #1125 - fix ev/select to only take and release locks once. By take and releasing locks twice per channel in the case where nothing is reading, there was an opportunity for ev/select to hang in the multithreaded case. Also silence valgrind/helgrind errors.
2023-05-07 16:52:11 +00:00
static void chan_unlock_args(const Janet *argv, int32_t n) {
for (int32_t i = 0; i < n; i++) {
int32_t len;
const Janet *data;
JanetChannel *chan;
if (janet_indexed_view(argv[i], &data, &len) && len == 2) {
chan = janet_getchannel(data, 0);
} else {
chan = janet_getchannel(argv, i);
}
janet_chan_unlock(chan);
}
}
Update ev.c to use the new binding style that provides source information
2021-07-26 05:25:38 +00:00
JANET_CORE_FN(cfun_channel_choice,
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
"(ev/select & clauses)",
Tweak ev/select docstring
2023-05-07 13:22:33 +00:00
"Block until the first of several channel operations occur. Returns a "
Update CHANGELOG. and format.
2023-05-11 23:03:38 +00:00
"tuple of the form [:give chan], [:take chan x], or [:close chan], "
"where a :give tuple is the result of a write and a :take tuple is the "
"result of a read. Each clause must be either a channel (for a channel "
"take operation) or a tuple [channel x] (for a channel give operation). "
"Operations are tried in order such that earlier clauses take "
"precedence over later clauses. Both give and take operations can "
"return a [:close chan] tuple, which indicates that the specified "
"channel was closed while waiting, or that the channel was already "
"closed.") {
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
janet_arity(argc, 1, -1);
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
int32_t len;
const Janet *data;
/* Check channels for immediate reads and writes */
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
for (int32_t i = 0; i < argc; i++) {
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
if (janet_indexed_view(argv[i], &data, &len) && len == 2) {
/* Write */
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
JanetChannel *chan = janet_getchannel(data, 0);
janet_chan_lock(chan);
if (chan->closed) {
Proper locking on select.
2021-12-10 00:59:59 +00:00
janet_chan_unlock(chan);
Address #1125 - fix ev/select to only take and release locks once. By take and releasing locks twice per channel in the case where nothing is reading, there was an opportunity for ev/select to hang in the multithreaded case. Also silence valgrind/helgrind errors.
2023-05-07 16:52:11 +00:00
chan_unlock_args(argv, i);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
return make_close_result(chan);
}
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
if (janet_q_count(&chan->items) < chan->limit) {
Address #1125 - fix ev/select to only take and release locks once. By take and releasing locks twice per channel in the case where nothing is reading, there was an opportunity for ev/select to hang in the multithreaded case. Also silence valgrind/helgrind errors.
2023-05-07 16:52:11 +00:00
janet_channel_push_with_lock(chan, data[1], 1);
chan_unlock_args(argv, i);
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
return make_write_result(chan);
}
} else {
/* Read */
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
JanetChannel *chan = janet_getchannel(argv, i);
Proper locking on select.
2021-12-10 00:59:59 +00:00
janet_chan_lock(chan);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
if (chan->closed) {
Proper locking on select.
2021-12-10 00:59:59 +00:00
janet_chan_unlock(chan);
Address #1125 - fix ev/select to only take and release locks once. By take and releasing locks twice per channel in the case where nothing is reading, there was an opportunity for ev/select to hang in the multithreaded case. Also silence valgrind/helgrind errors.
2023-05-07 16:52:11 +00:00
chan_unlock_args(argv, i);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
return make_close_result(chan);
}
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
if (chan->items.head != chan->items.tail) {
Janet item;
Address #1125 - fix ev/select to only take and release locks once. By take and releasing locks twice per channel in the case where nothing is reading, there was an opportunity for ev/select to hang in the multithreaded case. Also silence valgrind/helgrind errors.
2023-05-07 16:52:11 +00:00
janet_channel_pop_with_lock(chan, &item, 1);
chan_unlock_args(argv, i);
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
return make_read_result(chan, item);
}
}
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
}
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
/* Wait for all readers or writers */
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
for (int32_t i = 0; i < argc; i++) {
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
if (janet_indexed_view(argv[i], &data, &len) && len == 2) {
/* Write */
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
JanetChannel *chan = janet_getchannel(data, 0);
Address #1125 - fix ev/select to only take and release locks once. By take and releasing locks twice per channel in the case where nothing is reading, there was an opportunity for ev/select to hang in the multithreaded case. Also silence valgrind/helgrind errors.
2023-05-07 16:52:11 +00:00
janet_channel_push_with_lock(chan, data[1], 1);
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
} else {
/* Read */
Janet item;
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
JanetChannel *chan = janet_getchannel(argv, i);
Address #1125 - fix ev/select to only take and release locks once. By take and releasing locks twice per channel in the case where nothing is reading, there was an opportunity for ev/select to hang in the multithreaded case. Also silence valgrind/helgrind errors.
2023-05-07 16:52:11 +00:00
janet_channel_pop_with_lock(chan, &item, 1);
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
}
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
}
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
janet_await();
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
}
Update ev.c to use the new binding style that provides source information
2021-07-26 05:25:38 +00:00
JANET_CORE_FN(cfun_channel_full,
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
"(ev/full channel)",
"Check if a channel is full or not.") {
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
janet_fixarity(argc, 1);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
JanetChannel *channel = janet_getchannel(argv, 0);
janet_chan_lock(channel);
Janet ret = janet_wrap_boolean(janet_q_count(&channel->items) >= channel->limit);
janet_chan_unlock(channel);
return ret;
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
}
Update ev.c to use the new binding style that provides source information
2021-07-26 05:25:38 +00:00
JANET_CORE_FN(cfun_channel_capacity,
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
"(ev/capacity channel)",
"Get the number of items a channel will store before blocking writers.") {
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
janet_fixarity(argc, 1);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
JanetChannel *channel = janet_getchannel(argv, 0);
janet_chan_lock(channel);
Janet ret = janet_wrap_integer(channel->limit);
janet_chan_unlock(channel);
return ret;
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
}
Update ev.c to use the new binding style that provides source information
2021-07-26 05:25:38 +00:00
JANET_CORE_FN(cfun_channel_count,
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
"(ev/count channel)",
"Get the number of items currently waiting in a channel.") {
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
janet_fixarity(argc, 1);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
JanetChannel *channel = janet_getchannel(argv, 0);
janet_chan_lock(channel);
Janet ret = janet_wrap_integer(janet_q_count(&channel->items));
janet_chan_unlock(channel);
return ret;
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
}
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
/* Fisher yates shuffle of arguments to get fairness */
static void fisher_yates_args(int32_t argc, Janet *argv) {
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
for (int32_t i = argc; i > 1; i--) {
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
int32_t swap_index = janet_rng_u32(&janet_vm.ev_rng) % i;
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
Janet temp = argv[swap_index];
argv[swap_index] = argv[i - 1];
argv[i - 1] = temp;
}
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
}
Update ev.c to use the new binding style that provides source information
2021-07-26 05:25:38 +00:00
JANET_CORE_FN(cfun_channel_rchoice,
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
"(ev/rselect & clauses)",
"Similar to ev/select, but will try clauses in a random order for fairness.") {
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
fisher_yates_args(argc, argv);
return cfun_channel_choice(argc, argv);
Add ev/select and ev/rselect initial implementation. Getting closer to a CSP implmententation. Probably useful to move scheduling fields outside of fibers and into an external table.
2020-08-09 05:20:27 +00:00
}
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
JANET_CORE_FN(cfun_channel_new,
"(ev/chan &opt capacity)",
"Create a new channel. capacity is the number of values to queue before "
"blocking writers, defaults to 0 if not provided. Returns a new channel.") {
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
janet_arity(argc, 0, 1);
Set default channel size to 0.
2020-08-03 12:57:02 +00:00
int32_t limit = janet_optnat(argv, argc, 0, 0);
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
JanetChannel *channel = janet_abstract(&janet_channel_type, sizeof(JanetChannel));
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
janet_chan_init(channel, limit, 0);
Add preliminary channel implementation.
2020-08-01 18:13:58 +00:00
return janet_wrap_abstract(channel);
}
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
JANET_CORE_FN(cfun_channel_new_threaded,
"(ev/thread-chan &opt limit)",
"Create a threaded channel. A threaded channel is a channel that can be shared between threads and "
"used to communicate between any number of operating system threads.") {
janet_arity(argc, 0, 1);
int32_t limit = janet_optnat(argv, argc, 0, 0);
Add basic runtime support for threaded abstracts. A threaded abstract is an abstract type that can be freely shared between threads. While no synchronization is provided, refcounting and transport between threads is. This will let implementers more easily exploit OS-level parallelism in C library code. The caveat with these types is that they need to be careful in how they interact with objects on other heaps.
2021-08-20 02:16:20 +00:00
JanetChannel *tchan = janet_abstract_threaded(&janet_channel_type, sizeof(JanetChannel));
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
janet_chan_init(tchan, limit, 1);
Add basic runtime support for threaded abstracts. A threaded abstract is an abstract type that can be freely shared between threads. While no synchronization is provided, refcounting and transport between threads is. This will let implementers more easily exploit OS-level parallelism in C library code. The caveat with these types is that they need to be careful in how they interact with objects on other heaps.
2021-08-20 02:16:20 +00:00
return janet_wrap_abstract(tchan);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
}
Add semantics for closing channels. This makes certain algorithms simpler as channels now have an explicit lifetime - multiple readers can coordinate closing without needing to ensure the same number of reads as writes.
2021-07-31 00:26:42 +00:00
JANET_CORE_FN(cfun_channel_close,
"(ev/chan-close chan)",
"Close a channel. A closed channel will cause all pending reads and writes to return nil. "
"Returns the channel.") {
janet_fixarity(argc, 1);
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
JanetChannel *channel = janet_getchannel(argv, 0);
janet_chan_lock(channel);
Add semantics for closing channels. This makes certain algorithms simpler as channels now have an explicit lifetime - multiple readers can coordinate closing without needing to ensure the same number of reads as writes.
2021-07-31 00:26:42 +00:00
if (!channel->closed) {
channel->closed = 1;
JanetChannelPending writer;
while (!janet_q_pop(&channel->write_pending, &writer, sizeof(writer))) {
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
if (writer.thread != &janet_vm) {
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
JanetVM *vm = writer.thread;
JanetEVGenericMessage msg;
msg.fiber = writer.fiber;
msg.argp = channel;
msg.tag = JANET_CP_MODE_CLOSE;
msg.argi = (int32_t) writer.sched_id;
msg.argj = janet_wrap_nil();
janet_ev_post_event(vm, janet_thread_chan_cb, msg);
Add semantics for closing channels. This makes certain algorithms simpler as channels now have an explicit lifetime - multiple readers can coordinate closing without needing to ensure the same number of reads as writes.
2021-07-31 00:26:42 +00:00
} else {
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
if (writer.mode == JANET_CP_MODE_CHOICE_WRITE) {
janet_schedule(writer.fiber, make_close_result(channel));
Fix resumption values when closing a channel. When suspended in `ev/give` or `ev/take`, closing the channel should cause the result of `ev/give` or `ev/take` to be `nil`. When suspended in `ev/select`, closing the channel should cause the result of `ev/select` to be `[:close ch]`. The results were flipped before.
2023-06-07 18:56:31 +00:00
} else {
janet_schedule(writer.fiber, janet_wrap_nil());
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
}
Add semantics for closing channels. This makes certain algorithms simpler as channels now have an explicit lifetime - multiple readers can coordinate closing without needing to ensure the same number of reads as writes.
2021-07-31 00:26:42 +00:00
}
}
JanetChannelPending reader;
while (!janet_q_pop(&channel->read_pending, &reader, sizeof(reader))) {
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
if (reader.thread != &janet_vm) {
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
JanetVM *vm = reader.thread;
JanetEVGenericMessage msg;
msg.fiber = reader.fiber;
msg.argp = channel;
msg.tag = JANET_CP_MODE_CLOSE;
msg.argi = (int32_t) reader.sched_id;
msg.argj = janet_wrap_nil();
janet_ev_post_event(vm, janet_thread_chan_cb, msg);
Add semantics for closing channels. This makes certain algorithms simpler as channels now have an explicit lifetime - multiple readers can coordinate closing without needing to ensure the same number of reads as writes.
2021-07-31 00:26:42 +00:00
} else {
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
if (reader.mode == JANET_CP_MODE_CHOICE_READ) {
janet_schedule(reader.fiber, make_close_result(channel));
Fix resumption values when closing a channel. When suspended in `ev/give` or `ev/take`, closing the channel should cause the result of `ev/give` or `ev/take` to be `nil`. When suspended in `ev/select`, closing the channel should cause the result of `ev/select` to be `[:close ch]`. The results were flipped before.
2023-06-07 18:56:31 +00:00
} else {
janet_schedule(reader.fiber, janet_wrap_nil());
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
}
Add semantics for closing channels. This makes certain algorithms simpler as channels now have an explicit lifetime - multiple readers can coordinate closing without needing to ensure the same number of reads as writes.
2021-07-31 00:26:42 +00:00
}
}
}
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
janet_chan_unlock(channel);
return argv[0];
Add semantics for closing channels. This makes certain algorithms simpler as channels now have an explicit lifetime - multiple readers can coordinate closing without needing to ensure the same number of reads as writes.
2021-07-31 00:26:42 +00:00
}
Allow iterating through the properties of core abstract types.
2021-01-12 05:14:07 +00:00
static const JanetMethod ev_chanat_methods[] = {
{"select", cfun_channel_choice},
{"rselect", cfun_channel_rchoice},
{"count", cfun_channel_count},
{"take", cfun_channel_pop},
{"give", cfun_channel_push},
{"capacity", cfun_channel_capacity},
{"full", cfun_channel_full},
Add semantics for closing channels. This makes certain algorithms simpler as channels now have an explicit lifetime - multiple readers can coordinate closing without needing to ensure the same number of reads as writes.
2021-07-31 00:26:42 +00:00
{"close", cfun_channel_close},
Allow iterating through the properties of core abstract types.
2021-01-12 05:14:07 +00:00
{NULL, NULL}
};
static int janet_chanat_get(void *p, Janet key, Janet *out) {
(void) p;
if (!janet_checktype(key, JANET_KEYWORD)) return 0;
return janet_getmethod(janet_unwrap_keyword(key), ev_chanat_methods, out);
}
static Janet janet_chanat_next(void *p, Janet key) {
(void) p;
return janet_nextmethod(ev_chanat_methods, key);
}
Add channel marshalling.
2022-11-10 22:32:54 +00:00
static void janet_chanat_marshal(void *p, JanetMarshalContext *ctx) {
JanetChannel *channel = (JanetChannel *)p;
Distinguish between threaded channels and non-threaded when marshalling. Threaded channels _can_ be marshalled, just not for communication between threads. This is a special case since the same abstract type is used for both threaded and non-threaded channels.
2023-08-16 19:26:52 +00:00
janet_marshal_byte(ctx, channel->is_threaded);
Test small fix for marshalling.
2023-07-02 17:58:55 +00:00
janet_marshal_abstract(ctx, channel);
Add channel marshalling.
2022-11-10 22:32:54 +00:00
janet_marshal_byte(ctx, channel->closed);
janet_marshal_int(ctx, channel->limit);
int32_t count = janet_q_count(&channel->items);
janet_marshal_int(ctx, count);
JanetQueue *items = &channel->items;
Janet *data = channel->items.data;
if (items->head <= items->tail) {
for (int32_t i = items->head; i < items->tail; i++)
janet_marshal_janet(ctx, data[i]);
} else {
for (int32_t i = items->head; i < items->capacity; i++)
janet_marshal_janet(ctx, data[i]);
for (int32_t i = 0; i < items->tail; i++)
janet_marshal_janet(ctx, data[i]);
}
}
static void *janet_chanat_unmarshal(JanetMarshalContext *ctx) {
Distinguish between threaded channels and non-threaded when marshalling. Threaded channels _can_ be marshalled, just not for communication between threads. This is a special case since the same abstract type is used for both threaded and non-threaded channels.
2023-08-16 19:26:52 +00:00
uint8_t is_threaded = janet_unmarshal_byte(ctx);
JanetChannel *abst;
if (is_threaded) {
abst = janet_unmarshal_abstract_threaded(ctx, sizeof(JanetChannel));
} else {
abst = janet_unmarshal_abstract(ctx, sizeof(JanetChannel));
}
Add channel marshalling.
2022-11-10 22:32:54 +00:00
uint8_t is_closed = janet_unmarshal_byte(ctx);
int32_t limit = janet_unmarshal_int(ctx);
int32_t count = janet_unmarshal_int(ctx);
if (count < 0) janet_panic("invalid negative channel count");
janet_chan_init(abst, limit, 0);
abst->closed = !!is_closed;
for (int32_t i = 0; i < count; i++) {
Janet item = janet_unmarshal_janet(ctx);
janet_q_push(&abst->items, &item, sizeof(item));
}
return abst;
}
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
const JanetAbstractType janet_channel_type = {
"core/channel",
janet_chanat_gc,
janet_chanat_mark,
janet_chanat_get,
NULL, /* put */
Add channel marshalling.
2022-11-10 22:32:54 +00:00
janet_chanat_marshal,
janet_chanat_unmarshal,
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
NULL, /* tostring */
NULL, /* compare */
NULL, /* hash */
janet_chanat_next,
JANET_ATEND_NEXT
};
More updates to the ev library.
2020-05-28 21:51:11 +00:00
/* Main event loop */
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
void janet_loop1_impl(int has_timeout, JanetTimestamp timeout);
More updates to the ev library.
2020-05-28 21:51:11 +00:00
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
int janet_loop_done(void) {
Redo state management for Janet listeners. Make more use of the built in GC code for abstracts to be sure things are more correct. Issue before was streams could be freed before IOCP events arrived.
2023-09-25 07:43:36 +00:00
return !((janet_vm.spawn.head != janet_vm.spawn.tail) ||
Fix linux issues with epoll on normal files. We use the selfpipe trick if epoll fails with EPERM when trying to register a file descriptor.
2021-07-26 02:47:52 +00:00
janet_vm.tq_count ||
Expose atomic refcounts to be easier to port. This code was duplicate in a few places.
2023-10-01 15:09:23 +00:00
janet_vm.listener_count);
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
}
JanetFiber *janet_loop1(void) {
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
/* Schedule expired timers */
JanetTimeout to;
JanetTimestamp now = ts_now();
while (peek_timeout(&to) && to.when <= now) {
pop_timeout(0);
Add ev/deadline and ev/with-deadline. This should be more useful than timeouts in real-world use cases. The deadline system is based on fibers and is target to much more coarse-grained (and therfor reliable) timeouts than things like ev/sleep and timeout arguments.
2020-12-05 16:32:34 +00:00
if (to.curr_fiber != NULL) {
Add `ev/all-tasks` to get running and pending root fibers.
2023-04-01 23:57:13 +00:00
if (janet_fiber_can_resume(to.curr_fiber)) {
Fix some formatting and undefined behavior.
2020-12-06 20:33:08 +00:00
janet_cancel(to.fiber, janet_cstringv("deadline expired"));
}
Add ev/deadline and ev/with-deadline. This should be more useful than timeouts in real-world use cases. The deadline system is based on fibers and is target to much more coarse-grained (and therfor reliable) timeouts than things like ev/sleep and timeout arguments.
2020-12-05 16:32:34 +00:00
} else {
/* This is a timeout (for a function call, not a whole fiber) */
if (to.fiber->sched_id == to.sched_id) {
if (to.is_error) {
janet_cancel(to.fiber, janet_cstringv("timeout"));
} else {
janet_schedule(to.fiber, janet_wrap_nil());
}
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
}
}
}
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
Allow multiple simultaneous interrupts cleanly for #1262 Instead of setting a flag, each interrupt increments an atomic counter. When the interrupt is finally handled, either by scheduling code to run on the event loop or executing some out of band code, the user must now decrement the interrupt counter with janet_interpreter_interrupt_handled. While this counter is non-zero, the event loop will not enter the interpreter. This changes the API a bit but makes it possible and easy to handle signals without race conditions or scheduler hacks, as the runtime can ensure that high priority code is run before re-entering possibly blocking interpreter code again. Also included is a new function janet_schedule_soon, which prepends to the task queue instead of appending, allowing interrupt handler to skip ahead of all other scheduled fibers. Lastly, also update meson default options to include the interpreter_interrupt code and raise a runtime error if os/sigaction is used with interpreter interrupt but that build option is not enabled.
2023-08-24 01:14:38 +00:00
/* Run scheduled fibers unless interrupts need to be handled. */
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
while (janet_vm.spawn.head != janet_vm.spawn.tail) {
Allow multiple simultaneous interrupts cleanly for #1262 Instead of setting a flag, each interrupt increments an atomic counter. When the interrupt is finally handled, either by scheduling code to run on the event loop or executing some out of band code, the user must now decrement the interrupt counter with janet_interpreter_interrupt_handled. While this counter is non-zero, the event loop will not enter the interpreter. This changes the API a bit but makes it possible and easy to handle signals without race conditions or scheduler hacks, as the runtime can ensure that high priority code is run before re-entering possibly blocking interpreter code again. Also included is a new function janet_schedule_soon, which prepends to the task queue instead of appending, allowing interrupt handler to skip ahead of all other scheduled fibers. Lastly, also update meson default options to include the interpreter_interrupt code and raise a runtime error if os/sigaction is used with interpreter interrupt but that build option is not enabled.
2023-08-24 01:14:38 +00:00
/* Don't run until all interrupts have been marked as handled by calling janet_interpreter_interrupt_handled */
if (janet_vm.auto_suspend) break;
Address #820 - ev/cancel to work on already scheduled fibers.
2021-09-28 20:41:41 +00:00
JanetTask task = {NULL, janet_wrap_nil(), JANET_SIGNAL_OK, 0};
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
janet_q_pop(&janet_vm.spawn, &task, sizeof(task));
Fix #892 - Remove racy ref counts for channels Rather than manual reference counting for suspended fibers, we automate the process by incrementing "extra_listeners" every time we suspend a fiber in the event loop, and decrement when that fiber is resumed. In this manner, we keep track of the number of suspending fibers in a simpler, more correct way.
2021-12-10 00:44:55 +00:00
if (task.fiber->gc.flags & JANET_FIBER_EV_FLAG_SUSPENDED) janet_ev_dec_refcount();
task.fiber->gc.flags &= ~(JANET_FIBER_EV_FLAG_CANCELED | JANET_FIBER_EV_FLAG_SUSPENDED);
Address #820 - ev/cancel to work on already scheduled fibers.
2021-09-28 20:41:41 +00:00
if (task.expected_sched_id != task.fiber->sched_id) continue;
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
Janet res;
JanetSignal sig = janet_continue_signal(task.fiber, task.value, &res, task.sig);
Add `ev/all-tasks` to get running and pending root fibers.
2023-04-01 23:57:13 +00:00
if (!janet_fiber_can_resume(task.fiber)) {
janet_table_remove(&janet_vm.active_tasks, janet_wrap_fiber(task.fiber));
}
Fix threaded supervisor channels - Fix #766 Some pointer casting with abstract types was incorrect, resulting in strange behavior when trying to use supervisor channels that were threaded. This fix also adds the ability to supply a supervisor channel directly when creating a thread.
2021-08-17 02:14:06 +00:00
void *sv = task.fiber->supervisor_channel;
Errored threads always emit stacktrace or supervisor event. That way, it is much harder to swallow errors. Error swallowing behavior would have to be done explicitly by wrapping fibers with `protect` or `try`.
2021-09-02 02:05:05 +00:00
int is_suspended = sig == JANET_SIGNAL_EVENT || sig == JANET_SIGNAL_YIELD || sig == JANET_SIGNAL_INTERRUPT;
Fix #892 - Remove racy ref counts for channels Rather than manual reference counting for suspended fibers, we automate the process by incrementing "extra_listeners" every time we suspend a fiber in the event loop, and decrement when that fiber is resumed. In this manner, we keep track of the number of suspending fibers in a simpler, more correct way.
2021-12-10 00:44:55 +00:00
if (is_suspended) {
task.fiber->gc.flags |= JANET_FIBER_EV_FLAG_SUSPENDED;
janet_ev_inc_refcount();
}
Fix threaded supervisor channels - Fix #766 Some pointer casting with abstract types was incorrect, resulting in strange behavior when trying to use supervisor channels that were threaded. This fix also adds the ability to supply a supervisor channel directly when creating a thread.
2021-08-17 02:14:06 +00:00
if (NULL == sv) {
Errored threads always emit stacktrace or supervisor event. That way, it is much harder to swallow errors. Error swallowing behavior would have to be done explicitly by wrapping fibers with `protect` or `try`.
2021-09-02 02:05:05 +00:00
if (!is_suspended) {
Don't use janet_stacktrace anymore. Behavior of janet_stacktrace_ext is more consistent.
2021-12-06 14:51:40 +00:00
janet_stacktrace_ext(task.fiber, res, "");
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
}
} else if (sig == JANET_SIGNAL_OK || (task.fiber->flags & (1 << sig))) {
Fix threaded supervisor channels - Fix #766 Some pointer casting with abstract types was incorrect, resulting in strange behavior when trying to use supervisor channels that were threaded. This fix also adds the ability to supply a supervisor channel directly when creating a thread.
2021-08-17 02:14:06 +00:00
JanetChannel *chan = janet_channel_unwrap(sv);
Allow passing function to ev/thread. Convenient when there is no need to create an entire fiber.
2021-08-31 03:04:15 +00:00
janet_channel_push(chan, make_supervisor_event(janet_signal_names[sig],
Fix atomics - warnings on windows
2023-08-24 13:00:26 +00:00
task.fiber, chan->is_threaded), 2);
Errored threads always emit stacktrace or supervisor event. That way, it is much harder to swallow errors. Error swallowing behavior would have to be done explicitly by wrapping fibers with `protect` or `try`.
2021-09-02 02:05:05 +00:00
} else if (!is_suspended) {
Don't use janet_stacktrace anymore. Behavior of janet_stacktrace_ext is more consistent.
2021-12-06 14:51:40 +00:00
janet_stacktrace_ext(task.fiber, res, "");
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
}
if (sig == JANET_SIGNAL_INTERRUPT) {
return task.fiber;
}
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
}
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
Allow multiple simultaneous interrupts cleanly for #1262 Instead of setting a flag, each interrupt increments an atomic counter. When the interrupt is finally handled, either by scheduling code to run on the event loop or executing some out of band code, the user must now decrement the interrupt counter with janet_interpreter_interrupt_handled. While this counter is non-zero, the event loop will not enter the interpreter. This changes the API a bit but makes it possible and easy to handle signals without race conditions or scheduler hacks, as the runtime can ensure that high priority code is run before re-entering possibly blocking interpreter code again. Also included is a new function janet_schedule_soon, which prepends to the task queue instead of appending, allowing interrupt handler to skip ahead of all other scheduled fibers. Lastly, also update meson default options to include the interpreter_interrupt code and raise a runtime error if os/sigaction is used with interpreter interrupt but that build option is not enabled.
2023-08-24 01:14:38 +00:00
/* Poll for events */
Expose atomic refcounts to be easier to port. This code was duplicate in a few places.
2023-10-01 15:09:23 +00:00
if (janet_vm.tq_count || janet_vm.listener_count) {
Allow multiple simultaneous interrupts cleanly for #1262 Instead of setting a flag, each interrupt increments an atomic counter. When the interrupt is finally handled, either by scheduling code to run on the event loop or executing some out of band code, the user must now decrement the interrupt counter with janet_interpreter_interrupt_handled. While this counter is non-zero, the event loop will not enter the interpreter. This changes the API a bit but makes it possible and easy to handle signals without race conditions or scheduler hacks, as the runtime can ensure that high priority code is run before re-entering possibly blocking interpreter code again. Also included is a new function janet_schedule_soon, which prepends to the task queue instead of appending, allowing interrupt handler to skip ahead of all other scheduled fibers. Lastly, also update meson default options to include the interpreter_interrupt code and raise a runtime error if os/sigaction is used with interpreter interrupt but that build option is not enabled.
2023-08-24 01:14:38 +00:00
JanetTimeout to;
memset(&to, 0, sizeof(to));
int has_timeout;
/* Drop timeouts that are no longer needed */
while ((has_timeout = peek_timeout(&to))) {
if (to.curr_fiber != NULL) {
if (!janet_fiber_can_resume(to.curr_fiber)) {
janet_table_remove(&janet_vm.active_tasks, janet_wrap_fiber(to.curr_fiber));
pop_timeout(0);
continue;
}
} else if (to.fiber->sched_id != to.sched_id) {
pop_timeout(0);
continue;
}
break;
}
/* Run polling implementation only if pending timeouts or pending events */
Expose atomic refcounts to be easier to port. This code was duplicate in a few places.
2023-10-01 15:09:23 +00:00
if (janet_vm.tq_count || janet_vm.listener_count) {
Allow multiple simultaneous interrupts cleanly for #1262 Instead of setting a flag, each interrupt increments an atomic counter. When the interrupt is finally handled, either by scheduling code to run on the event loop or executing some out of band code, the user must now decrement the interrupt counter with janet_interpreter_interrupt_handled. While this counter is non-zero, the event loop will not enter the interpreter. This changes the API a bit but makes it possible and easy to handle signals without race conditions or scheduler hacks, as the runtime can ensure that high priority code is run before re-entering possibly blocking interpreter code again. Also included is a new function janet_schedule_soon, which prepends to the task queue instead of appending, allowing interrupt handler to skip ahead of all other scheduled fibers. Lastly, also update meson default options to include the interpreter_interrupt code and raise a runtime error if os/sigaction is used with interpreter interrupt but that build option is not enabled.
2023-08-24 01:14:38 +00:00
janet_loop1_impl(has_timeout, to.when);
}
}
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
/* No fiber was interrupted */
return NULL;
}
/* Same as janet_interpreter_interrupt, but will also
* break out of the event loop if waiting for an event
* (say, waiting for ev/sleep to finish). Does this by pushing
* an empty event to the event loop. */
void janet_loop1_interrupt(JanetVM *vm) {
janet_interpreter_interrupt(vm);
JanetEVGenericMessage msg = {0};
JanetCallback cb = NULL;
janet_ev_post_event(vm, cb, msg);
More updates to the ev library.
2020-05-28 21:51:11 +00:00
}
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
void janet_loop(void) {
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
while (!janet_loop_done()) {
JanetFiber *interrupted_fiber = janet_loop1();
if (NULL != interrupted_fiber) {
janet_schedule(interrupted_fiber, janet_wrap_nil());
}
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
}
}
More updates to the ev library.
2020-05-28 21:51:11 +00:00
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
/*
Working implementation of process waiting with threads. Does not require all sorts of signal handling code that is not thread-safe and can "steal processes". However, there is a much simpler way to add this functionality by creating a new stream and thread for each subprocess when it is waited on. This is perhaps _slightly_ less efficient but oh so much simpler, since we can reuse all of our concepts from streams and there is no need to implement a whole system around the selfpipe.
2020-12-31 17:22:18 +00:00
* Self-pipe handling code.
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
*/
#ifdef JANET_WINDOWS
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
/* On windows, use PostQueuedCompletionStatus instead for
* custom events */
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
#else
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
static void janet_ev_setup_selfpipe(void) {
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
if (janet_make_pipe(janet_vm.selfpipe, 0)) {
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
JANET_EXIT("failed to initialize self pipe in event loop");
}
}
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
Working implementation of process waiting with threads. Does not require all sorts of signal handling code that is not thread-safe and can "steal processes". However, there is a much simpler way to add this functionality by creating a new stream and thread for each subprocess when it is waited on. This is perhaps _slightly_ less efficient but oh so much simpler, since we can reuse all of our concepts from streams and there is no need to implement a whole system around the selfpipe.
2020-12-31 17:22:18 +00:00
/* Handle events from the self pipe inside the event loop */
static void janet_ev_handle_selfpipe(void) {
Add ev api for making threaded calls. Easy way to make arbitrary functions in C async.
2020-12-31 22:12:42 +00:00
JanetSelfPipeEvent response;
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
while (read(janet_vm.selfpipe[0], &response, sizeof(response)) > 0) {
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
if (NULL != response.cb) {
response.cb(response.msg);
}
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
}
}
Working implementation of process waiting with threads. Does not require all sorts of signal handling code that is not thread-safe and can "steal processes". However, there is a much simpler way to add this functionality by creating a new stream and thread for each subprocess when it is waited on. This is perhaps _slightly_ less efficient but oh so much simpler, since we can reuse all of our concepts from streams and there is no need to implement a whole system around the selfpipe.
2020-12-31 17:22:18 +00:00
static void janet_ev_cleanup_selfpipe(void) {
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
close(janet_vm.selfpipe[0]);
close(janet_vm.selfpipe[1]);
Working implementation of process waiting with threads. Does not require all sorts of signal handling code that is not thread-safe and can "steal processes". However, there is a much simpler way to add this functionality by creating a new stream and thread for each subprocess when it is waited on. This is perhaps _slightly_ less efficient but oh so much simpler, since we can reuse all of our concepts from streams and there is no need to implement a whole system around the selfpipe.
2020-12-31 17:22:18 +00:00
}
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
#endif
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
#ifdef JANET_WINDOWS
static JanetTimestamp ts_now(void) {
Update timestamp function to work on windows.
2020-10-04 20:18:31 +00:00
return (JanetTimestamp) GetTickCount64();
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
}
void janet_ev_init(void) {
janet_ev_init_common();
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
janet_vm.iocp = CreateIoCompletionPort(INVALID_HANDLE_VALUE, NULL, 0, 0);
if (NULL == janet_vm.iocp) janet_panic("could not create io completion port");
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
}
void janet_ev_deinit(void) {
janet_ev_deinit_common();
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
CloseHandle(janet_vm.iocp);
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
}
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
void janet_register_stream(JanetStream *stream) {
if (NULL == CreateIoCompletionPort(stream->handle, janet_vm.iocp, (ULONG_PTR) stream, 0)) {
janet_panicf("failed to listen for events: %V", janet_ev_lasterr());
Get socket reads and writes working with IOCP.
2020-11-08 16:38:28 +00:00
}
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
}
Update windows ev code.
2020-11-02 14:19:53 +00:00
void janet_loop1_impl(int has_timeout, JanetTimestamp to) {
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
ULONG_PTR completionKey = 0;
DWORD num_bytes_transfered = 0;
Prepare for 1.14.1 release.
2021-01-18 17:43:53 +00:00
LPOVERLAPPED overlapped = NULL;
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
/* Calculate how long to wait before timeout */
uint64_t waittime;
if (has_timeout) {
JanetTimestamp now = ts_now();
Small change to windows ev.
2020-11-02 14:27:45 +00:00
if (now > to) {
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
waittime = 0;
} else {
Small change to windows ev.
2020-11-02 14:27:45 +00:00
waittime = (uint64_t)(to - now);
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
}
} else {
waittime = INFINITE;
}
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
BOOL result = GetQueuedCompletionStatus(janet_vm.iocp, &num_bytes_transfered, &completionKey, &overlapped, (DWORD) waittime);
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
Revert dccb60b to address #1299 This commit was to address issues with a use after free error in the windows event loop, but the erroneous code was later reworked.
2023-10-03 05:14:25 +00:00
if (result || overlapped) {
Fix windows swallowing IOCP events in many cases. This fixes windows hanging on "failed" IO operations.
2021-01-18 02:41:59 +00:00
if (0 == completionKey) {
/* Custom event */
JanetSelfPipeEvent *response = (JanetSelfPipeEvent *)(overlapped);
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
if (NULL != response->cb) {
response->cb(response->msg);
}
Add config support for custom allocators.
2021-03-23 10:00:48 +00:00
janet_free(response);
Fix windows swallowing IOCP events in many cases. This fixes windows hanging on "failed" IO operations.
2021-01-18 02:41:59 +00:00
} else {
/* Normal event */
JanetStream *stream = (JanetStream *) completionKey;
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
JanetFiber *fiber = NULL;
if (stream->read_fiber && stream->read_fiber->ev_state == overlapped) {
fiber = stream->read_fiber;
} else if (stream->write_fiber && stream->write_fiber->ev_state == overlapped) {
fiber = stream->write_fiber;
Redo state management for Janet listeners. Make more use of the built in GC code for abstracts to be sure things are more correct. Issue before was streams could be freed before IOCP events arrived.
2023-09-25 07:43:36 +00:00
}
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
if (fiber != NULL) {
Remove some extra fiber state and use a flag.
2023-10-07 19:07:05 +00:00
fiber->flags &= ~JANET_FIBER_EV_FLAG_IN_FLIGHT;
Reuse overlapped overhead on windows for something useful.
2023-10-07 18:25:20 +00:00
/* System is done with this, we can reused this data */
overlapped->InternalHigh = (ULONG_PTR) num_bytes_transfered;
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
fiber->ev_callback(fiber, result ? JANET_ASYNC_EVENT_COMPLETE : JANET_ASYNC_EVENT_FAILED);
} else {
janet_free((void *) overlapped);
janet_ev_dec_refcount();
Get socket reads and writes working with IOCP.
2020-11-08 16:38:28 +00:00
}
Add TOCLOSE back.
2023-09-24 19:28:35 +00:00
janet_stream_checktoclose(stream);
Get socket reads and writes working with IOCP.
2020-11-08 16:38:28 +00:00
}
Work on getting ev support on windows.
2020-10-04 17:46:15 +00:00
}
}
Fix EPOLL implementation.
2020-11-16 01:40:47 +00:00
#elif defined(JANET_EV_EPOLL)
More updates to the ev library.
2020-05-28 21:51:11 +00:00
static JanetTimestamp ts_now(void) {
struct timespec now;
janet_assert(-1 != clock_gettime(CLOCK_MONOTONIC, &now), "failed to get time");
uint64_t res = 1000 * now.tv_sec;
res += now.tv_nsec / 1000000;
return res;
}
Start with ev module.
2020-05-28 15:39:40 +00:00
/* Wait for the next event */
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
static void janet_register_stream(JanetStream *stream) {
struct epoll_event ev;
ev.events = EPOLLIN | EPOLLOUT | EPOLLET;
ev.data.ptr = stream;
int status;
do {
status = epoll_ctl(janet_vm.epoll, EPOLL_CTL_ADD, stream->handle, &ev);
} while (status == -1 && errno == EINTR);
if (status == -1) {
if (errno == EPERM) {
/* Couldn't add to event loop, so assume that it completes
* synchronously. */
stream->flags |= JANET_STREAM_UNREGISTERED;
} else {
/* Unexpected error */
janet_panicv(janet_ev_lasterr());
Fix linux issues with epoll on normal files. We use the selfpipe trick if epoll fails with EPERM when trying to register a file descriptor.
2021-07-26 02:47:52 +00:00
}
Start with ev module.
2020-05-28 15:39:40 +00:00
}
}
#define JANET_EPOLL_MAX_EVENTS 64
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
void janet_loop1_impl(int has_timeout, JanetTimestamp timeout) {
Fix EPOLL implementation.
2020-11-16 01:40:47 +00:00
struct itimerspec its;
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
if (janet_vm.timer_enabled || has_timeout) {
More updates to the ev library.
2020-05-28 21:51:11 +00:00
memset(&its, 0, sizeof(its));
if (has_timeout) {
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
its.it_value.tv_sec = timeout / 1000;
its.it_value.tv_nsec = (timeout % 1000) * 1000000;
More updates to the ev library.
2020-05-28 21:51:11 +00:00
}
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
timerfd_settime(janet_vm.timerfd, TFD_TIMER_ABSTIME, &its, NULL);
More updates to the ev library.
2020-05-28 21:51:11 +00:00
}
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
janet_vm.timer_enabled = has_timeout;
More updates to the ev library.
2020-05-28 21:51:11 +00:00
Start with ev module.
2020-05-28 15:39:40 +00:00
/* Poll for events */
struct epoll_event events[JANET_EPOLL_MAX_EVENTS];
int ready;
do {
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
ready = epoll_wait(janet_vm.epoll, events, JANET_EPOLL_MAX_EVENTS, -1);
Start with ev module.
2020-05-28 15:39:40 +00:00
} while (ready == -1 && errno == EINTR);
if (ready == -1) {
JANET_EXIT("failed to poll events");
}
More updates to the ev library.
2020-05-28 21:51:11 +00:00
Start with ev module.
2020-05-28 15:39:40 +00:00
/* Step state machines */
for (int i = 0; i < ready; i++) {
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
void *p = events[i].data.ptr;
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
if (&janet_vm.timerfd == p) {
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
/* Timer expired, ignore */;
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
} else if (janet_vm.selfpipe == p) {
Working implementation of process waiting with threads. Does not require all sorts of signal handling code that is not thread-safe and can "steal processes". However, there is a much simpler way to add this functionality by creating a new stream and thread for each subprocess when it is waited on. This is perhaps _slightly_ less efficient but oh so much simpler, since we can reuse all of our concepts from streams and there is no need to implement a whole system around the selfpipe.
2020-12-31 17:22:18 +00:00
/* Self-pipe handling */
janet_ev_handle_selfpipe();
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
} else {
JanetStream *stream = p;
More updates to the ev library.
2020-05-28 21:51:11 +00:00
int mask = events[i].events;
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
int has_err = mask & EPOLLERR;
int has_hup = mask & EPOLLHUP;
JanetFiber *rf = stream->read_fiber;
JanetFiber *wf = stream->write_fiber;
if (rf) {
if (rf->ev_callback && (mask & EPOLLIN)) {
rf->ev_callback(rf, JANET_ASYNC_EVENT_READ);
}
if (rf->ev_callback && has_err) {
rf->ev_callback(rf, JANET_ASYNC_EVENT_ERR);
}
if (rf->ev_callback && has_hup) {
rf->ev_callback(rf, JANET_ASYNC_EVENT_HUP);
}
}
if (wf) {
if (wf->ev_callback && (mask & EPOLLOUT)) {
wf->ev_callback(wf, JANET_ASYNC_EVENT_WRITE);
}
if (wf->ev_callback && has_err) {
wf->ev_callback(wf, JANET_ASYNC_EVENT_ERR);
}
if (wf->ev_callback && has_hup) {
wf->ev_callback(wf, JANET_ASYNC_EVENT_HUP);
Partial work updating epoll reimplentation.
2023-09-26 01:52:15 +00:00
}
More updates to the ev library.
2020-05-28 21:51:11 +00:00
}
Add TOCLOSE back.
2023-09-24 19:28:35 +00:00
janet_stream_checktoclose(stream);
Start with ev module.
2020-05-28 15:39:40 +00:00
}
}
}
void janet_ev_init(void) {
janet_ev_init_common();
Working implementation of process waiting with threads. Does not require all sorts of signal handling code that is not thread-safe and can "steal processes". However, there is a much simpler way to add this functionality by creating a new stream and thread for each subprocess when it is waited on. This is perhaps _slightly_ less efficient but oh so much simpler, since we can reuse all of our concepts from streams and there is no need to implement a whole system around the selfpipe.
2020-12-31 17:22:18 +00:00
janet_ev_setup_selfpipe();
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
janet_vm.epoll = epoll_create1(EPOLL_CLOEXEC);
janet_vm.timerfd = timerfd_create(CLOCK_MONOTONIC, TFD_CLOEXEC | TFD_NONBLOCK);
janet_vm.timer_enabled = 0;
if (janet_vm.epoll == -1 || janet_vm.timerfd == -1) goto error;
More updates to the ev library.
2020-05-28 21:51:11 +00:00
struct epoll_event ev;
ev.events = EPOLLIN | EPOLLET;
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
ev.data.ptr = &janet_vm.timerfd;
if (-1 == epoll_ctl(janet_vm.epoll, EPOLL_CTL_ADD, janet_vm.timerfd, &ev)) goto error;
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
ev.events = EPOLLIN | EPOLLET;
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
ev.data.ptr = janet_vm.selfpipe;
if (-1 == epoll_ctl(janet_vm.epoll, EPOLL_CTL_ADD, janet_vm.selfpipe[0], &ev)) goto error;
More updates to the ev library.
2020-05-28 21:51:11 +00:00
return;
error:
JANET_EXIT("failed to initialize event loop");
Start with ev module.
2020-05-28 15:39:40 +00:00
}
void janet_ev_deinit(void) {
janet_ev_deinit_common();
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
close(janet_vm.epoll);
close(janet_vm.timerfd);
Working implementation of process waiting with threads. Does not require all sorts of signal handling code that is not thread-safe and can "steal processes". However, there is a much simpler way to add this functionality by creating a new stream and thread for each subprocess when it is waited on. This is perhaps _slightly_ less efficient but oh so much simpler, since we can reuse all of our concepts from streams and there is no need to implement a whole system around the selfpipe.
2020-12-31 17:22:18 +00:00
janet_ev_cleanup_selfpipe();
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
janet_vm.epoll = 0;
Start with ev module.
2020-05-28 15:39:40 +00:00
}
/*
* End epoll implementation
*/
Add kqueue support to Janet Note that this is a work in progress and simply a first attempt at getting some code into place before being able to test it. This code follows of sorts both the poll and epoll sections of the codebase hoping to achieve the exact same.
2021-09-03 19:29:13 +00:00
#elif defined(JANET_EV_KQUEUE)
NetBSD support Checking throught NetBSD's man pages, excepting for NetBSD-current, NetBSD uses `intptr_t` as the type for `.udata`. This change allows for `.udata` to match whatever type (by cast) the underlying system uses.
2021-09-05 12:30:22 +00:00
/* Definition from:
* https://github.com/wahern/cqueues/blob/master/src/lib/kpoll.c
* NetBSD uses intptr_t while others use void * for .udata */
#define EV_SETx(ev, a, b, c, d, e, f) EV_SET((ev), (a), (b), (c), (d), (e), ((__typeof__((ev)->udata))(f)))
Fixes for BSD where BSD != FreeBSD NetBSD and OpenBSD lack NOTE_ABSTIME and NOTE_MSECONDS, so we define those and create a macro that we use for all timeout values in EV_TIMER events that will on all BSD excepting FreeBSD change an absolute time into an interval.
2021-09-06 01:48:42 +00:00
#define JANET_KQUEUE_TF (EV_ADD | EV_ENABLE | EV_CLEAR | EV_ONESHOT)
Fixes for NetBSD (again) Minimum interval for a timer must be 1 or more (or we get EINVAL) and Janet fails tests and halts events that the programmer may still be interested in.
2021-09-06 02:46:55 +00:00
#define JANET_KQUEUE_MIN_INTERVAL 0
Fixes for BSD where BSD != FreeBSD NetBSD and OpenBSD lack NOTE_ABSTIME and NOTE_MSECONDS, so we define those and create a macro that we use for all timeout values in EV_TIMER events that will on all BSD excepting FreeBSD change an absolute time into an interval.
2021-09-06 01:48:42 +00:00
/* NOTE:
Fixing kqueue under the BSDs. FreeBSD is the only BSD supporting ABSTIME timers, whereas the rest demand intervals. Janet operates on timestamps, which are absolute times, as per ABSTIME. The idea was to use that under FreeBSD but not the other BSDs. This commit changes that since ABSTIME breaks when the timeout supplied is for a time prior to whatever the time is now (invalid argument). We now utilize the same logic we use on the other BSDs with FreeBSD to effect interval timeouts since intervals are absolutely sometime beyond now, be it now and less than a millisecond, or more than a millisecond. This will hopefully unbreak BSD builds when running the test suite.
2021-09-28 22:32:10 +00:00
* NetBSD and OpenBSD expect things are always intervals, and FreeBSD doesn't
* like an ABSTIME in the past so just use intervals always. Introduces a
* calculation to determine the minimum timeout per timeout requested of
* kqueue. Also note that NetBSD doesn't accept timeout intervals less than 1
* millisecond, so correct all intervals on that platform to be at least 1
* millisecond.*/
Change the way timeouts work in kqueue. Don't use a timer filter, just set the timeout on each call to kevent. Should hopefully work around the 1ms minimum on NetBSD and be possibly more performant.
2021-09-29 20:19:30 +00:00
JanetTimestamp to_interval(const JanetTimestamp ts) {
Fixes for NetBSD (again) Minimum interval for a timer must be 1 or more (or we get EINVAL) and Janet fails tests and halts events that the programmer may still be interested in.
2021-09-06 02:46:55 +00:00
return ts >= JANET_KQUEUE_MIN_INTERVAL ? ts : JANET_KQUEUE_MIN_INTERVAL;
}
Change the way timeouts work in kqueue. Don't use a timer filter, just set the timeout on each call to kevent. Should hopefully work around the 1ms minimum on NetBSD and be possibly more performant.
2021-09-29 20:19:30 +00:00
#define JANET_KQUEUE_INTERVAL(timestamp) (to_interval((timestamp - ts_now())))
Add kqueue support to Janet Note that this is a work in progress and simply a first attempt at getting some code into place before being able to test it. This code follows of sorts both the poll and epoll sections of the codebase hoping to achieve the exact same.
2021-09-03 19:29:13 +00:00
static JanetTimestamp ts_now(void) {
struct timespec now;
janet_assert(-1 != clock_gettime(CLOCK_MONOTONIC, &now), "failed to get time");
More typo and syntax fixes.
2021-09-03 21:29:39 +00:00
uint64_t res = 1000 * now.tv_sec;
Add kqueue support to Janet Note that this is a work in progress and simply a first attempt at getting some code into place before being able to test it. This code follows of sorts both the poll and epoll sections of the codebase hoping to achieve the exact same.
2021-09-03 19:29:13 +00:00
res += now.tv_nsec / 1000000;
return res;
}
Change the way timeouts work in kqueue. Don't use a timer filter, just set the timeout on each call to kevent. Should hopefully work around the 1ms minimum on NetBSD and be possibly more performant.
2021-09-29 20:19:30 +00:00
/* NOTE: Assumes Janet's timestamp precision is in milliseconds. */
static void timestamp2timespec(struct timespec *t, JanetTimestamp ts) {
t->tv_sec = ts == 0 ? 0 : ts / 1000;
t->tv_nsec = ts == 0 ? 0 : (ts % 1000) * 1000000;
}
Quick blind take on getting kqueue similar to poll and poll.
2023-10-06 05:48:05 +00:00
void janet_register_stream(JanetStream *stream) {
struct kevent kev;
Add EV_EOF for kqueue.
2023-10-06 22:39:52 +00:00
EV_SETx(&kev, stream->handle, EVFILT_READ | EVFILT_WRITE, EV_ADD | EV_ENABLE | EV_CLEAR | EV_EOF, 0, 0, stream);
Quick blind take on getting kqueue similar to poll and poll.
2023-10-06 05:48:05 +00:00
int status;
do {
status = kevent(janet_vm.kq, &kev, 1, NULL, 0, NULL);
} while (status == -1 && errno != EINTR);
if (status == -1) {
janet_panicv(janet_ev_lasterr());
Fix incorrect use of EV_SET on pipes. It would seem adding a read and a write event filter on a pipe which is unidirectional might just be a bad idea.
2021-09-04 00:44:27 +00:00
}
Add kqueue support to Janet Note that this is a work in progress and simply a first attempt at getting some code into place before being able to test it. This code follows of sorts both the poll and epoll sections of the codebase hoping to achieve the exact same.
2021-09-03 19:29:13 +00:00
}
#define JANET_KQUEUE_MAX_EVENTS 64
void janet_loop1_impl(int has_timeout, JanetTimestamp timeout) {
Complete kqueue feature From this point things should be bug fixes or code formatting most likely. Updated commentary (removed superfluous comments, and commented out code). Refined commentary where it seemed important and may help whoever comes behind me keep from making bad assumptions similar to the ones I made. All tests ran with `gmake test` now pass. `valgrind` with FreeBSD does not support forking so `gmake valtest` fails once child processes are started. Determined not an issue, can't fix valgrind.
2021-09-04 13:23:03 +00:00
/* Poll for events */
Change the way timeouts work in kqueue. Don't use a timer filter, just set the timeout on each call to kevent. Should hopefully work around the 1ms minimum on NetBSD and be possibly more performant.
2021-09-29 20:19:30 +00:00
/* NOTE:
* We calculate the timeout interval per iteration. When the interval
* drops to 0 or negative, we effect a timeout of 0. Effecting a timeout
* of infinity will not work and could make other fibers with timeouts
* miss their timeouts if we did so.
* JANET_KQUEUE_INTERVAL insures we have a timeout of no less than 0. */
Add kqueue support to Janet Note that this is a work in progress and simply a first attempt at getting some code into place before being able to test it. This code follows of sorts both the poll and epoll sections of the codebase hoping to achieve the exact same.
2021-09-03 19:29:13 +00:00
int status;
Change the way timeouts work in kqueue. Don't use a timer filter, just set the timeout on each call to kevent. Should hopefully work around the 1ms minimum on NetBSD and be possibly more performant.
2021-09-29 20:19:30 +00:00
struct timespec ts;
Add kqueue support to Janet Note that this is a work in progress and simply a first attempt at getting some code into place before being able to test it. This code follows of sorts both the poll and epoll sections of the codebase hoping to achieve the exact same.
2021-09-03 19:29:13 +00:00
struct kevent events[JANET_KQUEUE_MAX_EVENTS];
do {
Change the way timeouts work in kqueue. Don't use a timer filter, just set the timeout on each call to kevent. Should hopefully work around the 1ms minimum on NetBSD and be possibly more performant.
2021-09-29 20:19:30 +00:00
if (janet_vm.timer_enabled || has_timeout) {
timestamp2timespec(&ts, JANET_KQUEUE_INTERVAL(timeout));
status = kevent(janet_vm.kq, NULL, 0, events,
JANET_KQUEUE_MAX_EVENTS, &ts);
} else {
status = kevent(janet_vm.kq, NULL, 0, events,
JANET_KQUEUE_MAX_EVENTS, NULL);
}
Add kqueue support to Janet Note that this is a work in progress and simply a first attempt at getting some code into place before being able to test it. This code follows of sorts both the poll and epoll sections of the codebase hoping to achieve the exact same.
2021-09-03 19:29:13 +00:00
} while (status == -1 && errno == EINTR);
Add EV_EOF for kqueue.
2023-10-06 22:39:52 +00:00
if (status == -1) {
Add kqueue support to Janet Note that this is a work in progress and simply a first attempt at getting some code into place before being able to test it. This code follows of sorts both the poll and epoll sections of the codebase hoping to achieve the exact same.
2021-09-03 19:29:13 +00:00
JANET_EXIT("failed to poll events");
Add EV_EOF for kqueue.
2023-10-06 22:39:52 +00:00
}
Add kqueue support to Janet Note that this is a work in progress and simply a first attempt at getting some code into place before being able to test it. This code follows of sorts both the poll and epoll sections of the codebase hoping to achieve the exact same.
2021-09-03 19:29:13 +00:00
Change the way timeouts work in kqueue. Don't use a timer filter, just set the timeout on each call to kevent. Should hopefully work around the 1ms minimum on NetBSD and be possibly more performant.
2021-09-29 20:19:30 +00:00
/* Make sure timer is set accordingly. */
janet_vm.timer_enabled = has_timeout;
Complete kqueue feature From this point things should be bug fixes or code formatting most likely. Updated commentary (removed superfluous comments, and commented out code). Refined commentary where it seemed important and may help whoever comes behind me keep from making bad assumptions similar to the ones I made. All tests ran with `gmake test` now pass. `valgrind` with FreeBSD does not support forking so `gmake valtest` fails once child processes are started. Determined not an issue, can't fix valgrind.
2021-09-04 13:23:03 +00:00
/* Step state machines */
Code style fixes. Pretty obvious I thought control statements were glued to their opening parenthesis at first and then I realized not and voila, a bundle of mixed style. Hopefully this fixes all of it.
2021-09-04 14:34:47 +00:00
for (int i = 0; i < status; i++) {
Format and update CHANGELOG.md
2021-09-18 18:40:32 +00:00
void *p = (void *) events[i].udata;
Change the way timeouts work in kqueue. Don't use a timer filter, just set the timeout on each call to kevent. Should hopefully work around the 1ms minimum on NetBSD and be possibly more performant.
2021-09-29 20:19:30 +00:00
if (janet_vm.selfpipe == p) {
Add kqueue support to Janet Note that this is a work in progress and simply a first attempt at getting some code into place before being able to test it. This code follows of sorts both the poll and epoll sections of the codebase hoping to achieve the exact same.
2021-09-03 19:29:13 +00:00
/* Self-pipe handling */
janet_ev_handle_selfpipe();
} else {
JanetStream *stream = p;
Quick blind take on getting kqueue similar to poll and poll.
2023-10-06 05:48:05 +00:00
int filt = events[i].filter;
int has_err = events[i].flags & EV_ERROR;
int has_hup = events[i].flags & EV_EOF;
Remove duplicate code in kqueue event handling.
2023-10-07 19:40:43 +00:00
for (int j = 0; j < 2; j++) {
JanetFiber *f = j ? stream->read_fiber : stream->write_fiber;
if (!f) continue;
if (f->ev_callback && has_err) {
f->ev_callback(f, JANET_ASYNC_EVENT_ERR);
Update kqueue
2023-10-06 06:33:51 +00:00
} else {
Remove duplicate code in kqueue event handling.
2023-10-07 19:40:43 +00:00
if (f->ev_callback && (filt == EVFILT_READ) && f == stream->read_fiber) {
f->ev_callback(f, JANET_ASYNC_EVENT_READ);
Update kqueue
2023-10-06 06:33:51 +00:00
}
Remove duplicate code in kqueue event handling.
2023-10-07 19:40:43 +00:00
if (f->ev_callback && (filt == EVFILT_WRITE) && f == stream->write_fiber) {
f->ev_callback(f, JANET_ASYNC_EVENT_WRITE);
Update kqueue
2023-10-06 06:33:51 +00:00
}
Remove duplicate code in kqueue event handling.
2023-10-07 19:40:43 +00:00
if (f->ev_callback && has_hup) {
f->ev_callback(f, JANET_ASYNC_EVENT_HUP);
Update kqueue
2023-10-06 06:33:51 +00:00
}
Work on debugging issue with server spawning.
2023-09-24 22:32:34 +00:00
}
Handle null state, don't read/write on error Need to guard against errors when reading/writing probably, if there is an error, forgo those events. Guard against null state (and the byproduct, a segfault), check if the state is null before utilizing it.
2021-09-04 04:22:07 +00:00
}
Add TOCLOSE back.
2023-09-24 19:28:35 +00:00
janet_stream_checktoclose(stream);
Add kqueue support to Janet Note that this is a work in progress and simply a first attempt at getting some code into place before being able to test it. This code follows of sorts both the poll and epoll sections of the codebase hoping to achieve the exact same.
2021-09-03 19:29:13 +00:00
}
}
}
void janet_ev_init(void) {
janet_ev_init_common();
janet_ev_setup_selfpipe();
janet_vm.kq = kqueue();
janet_vm.timer_enabled = 0;
if (janet_vm.kq == -1) goto error;
Change the way timeouts work in kqueue. Don't use a timer filter, just set the timeout on each call to kevent. Should hopefully work around the 1ms minimum on NetBSD and be possibly more performant.
2021-09-29 20:19:30 +00:00
struct kevent event;
EV_SETx(&event, janet_vm.selfpipe[0], EVFILT_READ, EV_ADD | EV_ENABLE, 0, 0, janet_vm.selfpipe);
Attempt 2 to fix bsd compilation errors.
2023-10-05 04:27:56 +00:00
int status;
Address some bsd issues.
2023-10-05 04:24:40 +00:00
do {
status = kevent(janet_vm.kq, &event, 1, NULL, 0, NULL);
Missing &&
2023-10-05 04:29:54 +00:00
} while (status == -1 && errno != EINTR);
Address some bsd issues.
2023-10-05 04:24:40 +00:00
if (status == -1) goto error;
Add kqueue support to Janet Note that this is a work in progress and simply a first attempt at getting some code into place before being able to test it. This code follows of sorts both the poll and epoll sections of the codebase hoping to achieve the exact same.
2021-09-03 19:29:13 +00:00
return;
error:
JANET_EXIT("failed to initialize event loop");
}
void janet_ev_deinit(void) {
janet_ev_deinit_common();
More typo and syntax fixes.
2021-09-03 21:29:39 +00:00
close(janet_vm.kq);
Add kqueue support to Janet Note that this is a work in progress and simply a first attempt at getting some code into place before being able to test it. This code follows of sorts both the poll and epoll sections of the codebase hoping to achieve the exact same.
2021-09-03 19:29:13 +00:00
janet_ev_cleanup_selfpipe();
janet_vm.kq = 0;
}
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
#elif defined(JANET_EV_POLL)
Add net/flush. Useful for simple TCP protocols (netrepl), which benefit from being able to immediately send a message.
2020-05-29 00:14:35 +00:00
Quick blind take on getting kqueue similar to poll and poll.
2023-10-06 05:48:05 +00:00
/* Simple poll implementation. Efficiency is not the goal here, although the poll implementation should be farily efficient
* for low numbers of concurrent file descriptors. Rather, the code should be simple, portable, correct, and mirror the
* epoll and kqueue code. */
Merge branch 'master' into ev Also add poll implementation for ev.
2020-09-07 17:52:50 +00:00
static JanetTimestamp ts_now(void) {
struct timespec now;
janet_assert(-1 != clock_gettime(CLOCK_REALTIME, &now), "failed to get time");
uint64_t res = 1000 * now.tv_sec;
res += now.tv_nsec / 1000000;
return res;
}
Don't use gcroot/unroot for tracking IO operations. This could have bad effects in higher load situations, and duplicates code. It is better to keep a dedicated list of scheduled IO operations which can be efficiently added and removed from. It also provides and easy way to enumerate scheduled IO operations.
2020-11-16 15:30:04 +00:00
/* Wait for the next event */
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
void janet_register_stream(JanetStream *stream) {
struct pollfd ev = {0};
stream->index = (uint32_t) janet_vm.stream_count;
size_t new_count = janet_vm.stream_count + 1;
if (new_count > janet_vm.stream_capacity) {
size_t new_cap = new_count * 2;
janet_vm.fds = janet_realloc(janet_vm.fds, (1 + new_cap) * sizeof(struct pollfd));
janet_vm.streams = janet_realloc(janet_vm.streams, new_cap * sizeof(JanetStream *));
if (!janet_vm.fds || !janet_vm.streams) {
Merge branch 'master' into ev Also add poll implementation for ev.
2020-09-07 17:52:50 +00:00
JANET_OUT_OF_MEMORY;
}
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_vm.stream_capacity = new_cap;
Merge branch 'master' into ev Also add poll implementation for ev.
2020-09-07 17:52:50 +00:00
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
ev.fd = stream->handle;
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
ev.events = POLLIN | POLLOUT;
janet_vm.fds[janet_vm.stream_count + 1] = ev;
janet_vm.streams[janet_vm.stream_count] = stream;
janet_vm.stream_count = new_count;
Merge branch 'master' into ev Also add poll implementation for ev.
2020-09-07 17:52:50 +00:00
}
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
void janet_loop1_impl(int has_timeout, JanetTimestamp timeout) {
/* set event flags */
for (size_t i = 0; i < janet_vm.stream_count; i++) {
JanetStream *stream = janet_vm.streams[i];
janet_vm.fds[i + 1].events = 0;
janet_vm.fds[i + 1].revents = 0;
if (stream->read_fiber) janet_vm.fds[i + 1].events |= POLLIN;
if (stream->write_fiber) janet_vm.fds[i + 1].events |= POLLOUT;
Make poll work by going back to array of listeners for gc keeping.
2023-09-27 03:57:01 +00:00
}
Merge branch 'master' into ev Also add poll implementation for ev.
2020-09-07 17:52:50 +00:00
/* Poll for events */
int ready;
do {
Don't use gcroot/unroot for tracking IO operations. This could have bad effects in higher load situations, and duplicates code. It is better to keep a dedicated list of scheduled IO operations which can be efficiently added and removed from. It also provides and easy way to enumerate scheduled IO operations.
2020-11-16 15:30:04 +00:00
int to = -1;
Merge branch 'master' into ev Also add poll implementation for ev.
2020-09-07 17:52:50 +00:00
if (has_timeout) {
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
JanetTimestamp now = ts_now();
Don't use gcroot/unroot for tracking IO operations. This could have bad effects in higher load situations, and duplicates code. It is better to keep a dedicated list of scheduled IO operations which can be efficiently added and removed from. It also provides and easy way to enumerate scheduled IO operations.
2020-11-16 15:30:04 +00:00
to = now > timeout ? 0 : (int)(timeout - now);
Merge branch 'master' into ev Also add poll implementation for ev.
2020-09-07 17:52:50 +00:00
}
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
ready = poll(janet_vm.fds, janet_vm.stream_count + 1, to);
Merge branch 'master' into ev Also add poll implementation for ev.
2020-09-07 17:52:50 +00:00
} while (ready == -1 && errno == EINTR);
if (ready == -1) {
JANET_EXIT("failed to poll events");
}
Add net/flush. Useful for simple TCP protocols (netrepl), which benefit from being able to immediately send a message.
2020-05-29 00:14:35 +00:00
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
/* Check selfpipe */
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
if (janet_vm.fds[0].revents & POLLIN) {
janet_vm.fds[0].revents = 0;
Working implementation of process waiting with threads. Does not require all sorts of signal handling code that is not thread-safe and can "steal processes". However, there is a much simpler way to add this functionality by creating a new stream and thread for each subprocess when it is waited on. This is perhaps _slightly_ less efficient but oh so much simpler, since we can reuse all of our concepts from streams and there is no need to implement a whole system around the selfpipe.
2020-12-31 17:22:18 +00:00
janet_ev_handle_selfpipe();
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
}
Merge branch 'master' into ev Also add poll implementation for ev.
2020-09-07 17:52:50 +00:00
/* Step state machines */
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
for (size_t i = 0; i < janet_vm.stream_count; i++) {
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
struct pollfd *pfd = janet_vm.fds + i + 1;
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
JanetStream *stream = janet_vm.streams[i];
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
int mask = pfd->revents;
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
if (!mask) continue;
int has_err = mask & POLLERR;
int has_hup = mask & POLLHUP;
JanetFiber *rf = stream->read_fiber;
JanetFiber *wf = stream->write_fiber;
if (rf) {
if (rf->ev_callback && (mask & POLLIN)) {
rf->ev_callback(rf, JANET_ASYNC_EVENT_READ);
} else if (rf->ev_callback && has_hup) {
rf->ev_callback(rf, JANET_ASYNC_EVENT_HUP);
} else if (rf->ev_callback && has_err) {
rf->ev_callback(rf, JANET_ASYNC_EVENT_ERR);
}
}
if (wf) {
if (wf->ev_callback && (mask & POLLOUT)) {
wf->ev_callback(wf, JANET_ASYNC_EVENT_WRITE);
} else if (wf->ev_callback && has_hup) {
wf->ev_callback(wf, JANET_ASYNC_EVENT_HUP);
} else if (wf->ev_callback && has_err) {
wf->ev_callback(wf, JANET_ASYNC_EVENT_ERR);
}
Add TOCLOSE back.
2023-09-24 19:28:35 +00:00
}
janet_stream_checktoclose(stream);
Merge branch 'master' into ev Also add poll implementation for ev.
2020-09-07 17:52:50 +00:00
}
}
void janet_ev_init(void) {
janet_ev_init_common();
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
janet_vm.fds = NULL;
Working implementation of process waiting with threads. Does not require all sorts of signal handling code that is not thread-safe and can "steal processes". However, there is a much simpler way to add this functionality by creating a new stream and thread for each subprocess when it is waited on. This is perhaps _slightly_ less efficient but oh so much simpler, since we can reuse all of our concepts from streams and there is no need to implement a whole system around the selfpipe.
2020-12-31 17:22:18 +00:00
janet_ev_setup_selfpipe();
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
janet_vm.fds = janet_malloc(sizeof(struct pollfd));
if (NULL == janet_vm.fds) {
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
JANET_OUT_OF_MEMORY;
}
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
janet_vm.fds[0].fd = janet_vm.selfpipe[0];
janet_vm.fds[0].events = POLLIN;
janet_vm.fds[0].revents = 0;
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_vm.streams = NULL;
janet_vm.stream_count = 0;
janet_vm.stream_capacity = 0;
Merge branch 'master' into ev Also add poll implementation for ev.
2020-09-07 17:52:50 +00:00
return;
}
void janet_ev_deinit(void) {
janet_ev_deinit_common();
Working implementation of process waiting with threads. Does not require all sorts of signal handling code that is not thread-safe and can "steal processes". However, there is a much simpler way to add this functionality by creating a new stream and thread for each subprocess when it is waited on. This is perhaps _slightly_ less efficient but oh so much simpler, since we can reuse all of our concepts from streams and there is no need to implement a whole system around the selfpipe.
2020-12-31 17:22:18 +00:00
janet_ev_cleanup_selfpipe();
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
janet_free(janet_vm.fds);
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_free(janet_vm.streams);
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
janet_vm.fds = NULL;
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_vm.streams = NULL;
Merge branch 'master' into ev Also add poll implementation for ev.
2020-09-07 17:52:50 +00:00
}
#endif
Add net/flush. Useful for simple TCP protocols (netrepl), which benefit from being able to immediately send a message.
2020-05-29 00:14:35 +00:00
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
/*
* End poll implementation
*/
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
/*
* Generic Callback system. Post a function pointer + data to the event loop (from another
* thread or even a signal handler). Allows posting events from another thread or signal handler.
*/
void janet_ev_post_event(JanetVM *vm, JanetCallback cb, JanetEVGenericMessage msg) {
vm = vm ? vm : &janet_vm;
#ifdef JANET_WINDOWS
JanetHandle iocp = vm->iocp;
JanetSelfPipeEvent *event = janet_malloc(sizeof(JanetSelfPipeEvent));
if (NULL == event) {
JANET_OUT_OF_MEMORY;
}
event->msg = msg;
event->cb = cb;
janet_assert(PostQueuedCompletionStatus(iocp,
Fix linux issues with epoll on normal files. We use the selfpipe trick if epoll fails with EPERM when trying to register a file descriptor.
2021-07-26 02:47:52 +00:00
sizeof(JanetSelfPipeEvent),
0,
(LPOVERLAPPED) event),
"failed to post completion event");
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
#else
JanetSelfPipeEvent event;
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
memset(&event, 0, sizeof(event));
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
event.msg = msg;
event.cb = cb;
Fix linux issues with epoll on normal files. We use the selfpipe trick if epoll fails with EPERM when trying to register a file descriptor.
2021-07-26 02:47:52 +00:00
int fd = vm->selfpipe[1];
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
/* handle a bit of back pressure before giving up. */
int tries = 4;
while (tries > 0) {
int status;
do {
status = write(fd, &event, sizeof(event));
} while (status == -1 && errno == EINTR);
if (status > 0) break;
Working example for threaded channels. Still no marshalling more complex values.
2021-08-15 20:25:07 +00:00
sleep(0);
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
tries--;
}
Add marshal/unmarshal to items pushed to threaded channel.
2021-08-15 21:12:40 +00:00
janet_assert(tries > 0, "failed to write event to self-pipe");
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
#endif
}
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
/*
* Threaded calls
*/
#ifdef JANET_WINDOWS
static DWORD WINAPI janet_thread_body(LPVOID ptr) {
JanetEVThreadInit *init = (JanetEVThreadInit *)ptr;
JanetEVGenericMessage msg = init->msg;
JanetThreadedSubroutine subr = init->subr;
JanetThreadedCallback cb = init->cb;
JanetHandle iocp = init->write_pipe;
/* Reuse memory from thread init for returning data */
init->msg = subr(msg);
init->cb = cb;
Begin implementing async subproccesses for windows.
2021-01-03 17:21:44 +00:00
janet_assert(PostQueuedCompletionStatus(iocp,
sizeof(JanetSelfPipeEvent),
0,
(LPOVERLAPPED) init),
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
"failed to post completion event");
return 0;
}
#else
static void *janet_thread_body(void *ptr) {
JanetEVThreadInit *init = (JanetEVThreadInit *)ptr;
JanetEVGenericMessage msg = init->msg;
JanetThreadedSubroutine subr = init->subr;
JanetThreadedCallback cb = init->cb;
int fd = init->write_pipe;
Add config support for custom allocators.
2021-03-23 10:00:48 +00:00
janet_free(init);
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
JanetSelfPipeEvent response;
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
memset(&response, 0, sizeof(response));
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
response.msg = subr(msg);
response.cb = cb;
/* handle a bit of back pressure before giving up. */
int tries = 4;
while (tries > 0) {
int status;
do {
status = write(fd, &response, sizeof(response));
} while (status == -1 && errno == EINTR);
if (status > 0) break;
sleep(1);
tries--;
}
return NULL;
}
#endif
void janet_ev_threaded_call(JanetThreadedSubroutine fp, JanetEVGenericMessage arguments, JanetThreadedCallback cb) {
Add config support for custom allocators.
2021-03-23 10:00:48 +00:00
JanetEVThreadInit *init = janet_malloc(sizeof(JanetEVThreadInit));
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
if (NULL == init) {
JANET_OUT_OF_MEMORY;
}
init->msg = arguments;
init->subr = fp;
init->cb = cb;
#ifdef JANET_WINDOWS
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
init->write_pipe = janet_vm.iocp;
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
HANDLE thread_handle = CreateThread(NULL, 0, janet_thread_body, init, 0, NULL);
if (NULL == thread_handle) {
Add config support for custom allocators.
2021-03-23 10:00:48 +00:00
janet_free(init);
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
janet_panic("failed to create thread");
}
CloseHandle(thread_handle); /* detach from thread */
#else
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
init->write_pipe = janet_vm.selfpipe[1];
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
pthread_t waiter_thread;
Create pthread threads with detached attribute. Rather than calling pthread_detach on a default thread.
2022-11-11 17:01:59 +00:00
int err = pthread_create(&waiter_thread, &janet_vm.new_thread_attr, janet_thread_body, init);
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
if (err) {
Add config support for custom allocators.
2021-03-23 10:00:48 +00:00
janet_free(init);
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
janet_panicf("%s", strerror(err));
}
#endif
/* Increment ev refcount so we don't quit while waiting for a subprocess */
janet_ev_inc_refcount();
}
/* Default callback for janet_ev_threaded_await. */
void janet_ev_default_threaded_callback(JanetEVGenericMessage return_value) {
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
janet_ev_dec_refcount();
Add option to build janet without thread library.
2021-07-17 20:13:28 +00:00
if (return_value.fiber == NULL) {
return;
}
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
switch (return_value.tag) {
default:
case JANET_EV_TCTAG_NIL:
janet_schedule(return_value.fiber, janet_wrap_nil());
break;
case JANET_EV_TCTAG_INTEGER:
janet_schedule(return_value.fiber, janet_wrap_integer(return_value.argi));
break;
case JANET_EV_TCTAG_STRING:
case JANET_EV_TCTAG_STRINGF:
janet_schedule(return_value.fiber, janet_cstringv((const char *) return_value.argp));
Add config support for custom allocators.
2021-03-23 10:00:48 +00:00
if (return_value.tag == JANET_EV_TCTAG_STRINGF) janet_free(return_value.argp);
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
break;
case JANET_EV_TCTAG_KEYWORD:
janet_schedule(return_value.fiber, janet_ckeywordv((const char *) return_value.argp));
break;
case JANET_EV_TCTAG_ERR_STRING:
case JANET_EV_TCTAG_ERR_STRINGF:
janet_cancel(return_value.fiber, janet_cstringv((const char *) return_value.argp));
Add config support for custom allocators.
2021-03-23 10:00:48 +00:00
if (return_value.tag == JANET_EV_TCTAG_STRINGF) janet_free(return_value.argp);
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
break;
case JANET_EV_TCTAG_ERR_KEYWORD:
janet_cancel(return_value.fiber, janet_ckeywordv((const char *) return_value.argp));
break;
Update os/shell to be non-blocking as well.
2021-01-18 22:44:22 +00:00
case JANET_EV_TCTAG_BOOLEAN:
janet_schedule(return_value.fiber, janet_wrap_boolean(return_value.argi));
break;
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
}
janet_gcunroot(janet_wrap_fiber(return_value.fiber));
}
/* Convenience method for common case */
Update os/shell to be non-blocking as well.
2021-01-18 22:44:22 +00:00
JANET_NO_RETURN
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
void janet_ev_threaded_await(JanetThreadedSubroutine fp, int tag, int argi, void *argp) {
JanetEVGenericMessage arguments;
Working example for threaded channels. Still no marshalling more complex values.
2021-08-15 20:25:07 +00:00
memset(&arguments, 0, sizeof(arguments));
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
arguments.tag = tag;
arguments.argi = argi;
arguments.argp = argp;
arguments.fiber = janet_root_fiber();
janet_gcroot(janet_wrap_fiber(arguments.fiber));
janet_ev_threaded_call(fp, arguments, janet_ev_default_threaded_callback);
janet_await();
}
/*
* C API helpers for reading and writing from streams.
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
* There is some networking code in here as well as generic
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
* reading and writing primitives.
*/
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
Remove some flags in os/open.
2020-11-15 16:06:20 +00:00
void janet_stream_flags(JanetStream *stream, uint32_t flags) {
if (stream->flags & JANET_STREAM_CLOSED) {
janet_panic("stream is closed");
}
if ((stream->flags & flags) != flags) {
const char *rmsg = "", *wmsg = "", *amsg = "", *dmsg = "", *smsg = "stream";
if (flags & JANET_STREAM_READABLE) rmsg = "readable ";
if (flags & JANET_STREAM_WRITABLE) wmsg = "writable ";
if (flags & JANET_STREAM_ACCEPTABLE) amsg = "server ";
if (flags & JANET_STREAM_UDPSERVER) dmsg = "datagram ";
if (flags & JANET_STREAM_SOCKET) smsg = "socket";
janet_panicf("bad stream, expected %s%s%s%s%s", rmsg, wmsg, amsg, dmsg, smsg);
}
}
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
/* When there is an IO error, we need to be able to convert it to a Janet
* string to raise a Janet error. */
#ifdef JANET_WINDOWS
#define JANET_EV_CHUNKSIZE 4096
Janet janet_ev_lasterr(void) {
int code = GetLastError();
char msgbuf[256];
msgbuf[0] = '\0';
FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS,
NULL,
code,
MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
msgbuf,
sizeof(msgbuf),
NULL);
if (!*msgbuf) sprintf(msgbuf, "%d", code);
char *c = msgbuf;
while (*c) {
if (*c == '\n' || *c == '\r') {
*c = '\0';
break;
}
c++;
}
return janet_cstringv(msgbuf);
}
#else
Janet janet_ev_lasterr(void) {
return janet_cstringv(strerror(errno));
}
#endif
/* State machine for read/recv/recvfrom */
typedef enum {
JANET_ASYNC_READMODE_READ,
JANET_ASYNC_READMODE_RECV,
JANET_ASYNC_READMODE_RECVFROM
} JanetReadMode;
typedef struct {
#ifdef JANET_WINDOWS
OVERLAPPED overlapped;
Fix windows build with JANET_NO_NET - #1055
2023-01-21 16:55:42 +00:00
DWORD flags;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
#ifdef JANET_NET
WSABUF wbuf;
struct sockaddr from;
int fromlen;
#endif
uint8_t chunk_buf[JANET_EV_CHUNKSIZE];
#else
int flags;
#endif
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
int32_t bytes_left;
int32_t bytes_read;
JanetBuffer *buf;
int is_chunk;
JanetReadMode mode;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
} StateRead;
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
void ev_callback_read(JanetFiber *fiber, JanetAsyncEvent event) {
JanetStream *stream = fiber->ev_stream;
StateRead *state = (StateRead *) fiber->ev_state;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
switch (event) {
default:
break;
case JANET_ASYNC_EVENT_MARK:
janet_mark(janet_wrap_buffer(state->buf));
break;
case JANET_ASYNC_EVENT_CLOSE:
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_schedule(fiber, janet_wrap_nil());
janet_async_end(fiber);
break;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
#ifdef JANET_WINDOWS
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
case JANET_ASYNC_EVENT_FAILED:
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
case JANET_ASYNC_EVENT_COMPLETE: {
/* Called when read finished */
Update ev.c for removing msvc warnings
2023-10-07 19:11:14 +00:00
uint32_t ev_bytes = (uint32_t) state->overlapped.InternalHigh;
Reuse overlapped overhead on windows for something useful.
2023-10-07 18:25:20 +00:00
state->bytes_read += ev_bytes;
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
if (state->bytes_read == 0 && (state->mode != JANET_ASYNC_READMODE_RECVFROM)) {
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
janet_schedule(fiber, janet_wrap_nil());
janet_async_end(fiber);
return;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
Reuse overlapped overhead on windows for something useful.
2023-10-07 18:25:20 +00:00
janet_buffer_push_bytes(state->buf, state->chunk_buf, ev_bytes);
state->bytes_left -= ev_bytes;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
Reuse overlapped overhead on windows for something useful.
2023-10-07 18:25:20 +00:00
if (state->bytes_left == 0 || !state->is_chunk || ev_bytes == 0) {
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
Janet resume_val;
#ifdef JANET_NET
if (state->mode == JANET_ASYNC_READMODE_RECVFROM) {
void *abst = janet_abstract(&janet_address_type, state->fromlen);
memcpy(abst, &state->from, state->fromlen);
resume_val = janet_wrap_abstract(abst);
} else
#endif
{
resume_val = janet_wrap_buffer(state->buf);
}
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
janet_schedule(fiber, resume_val);
janet_async_end(fiber);
return;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
}
/* fallthrough */
case JANET_ASYNC_EVENT_USER: {
int32_t chunk_size = state->bytes_left > JANET_EV_CHUNKSIZE ? JANET_EV_CHUNKSIZE : state->bytes_left;
memset(&(state->overlapped), 0, sizeof(OVERLAPPED));
int status;
#ifdef JANET_NET
More work on windows networking code. Remove use of WSARecv and WSASend since for whatever reason they seem suspect. We may want to revisit this later.
2021-01-12 00:00:31 +00:00
if (state->mode == JANET_ASYNC_READMODE_RECVFROM) {
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
state->wbuf.len = (ULONG) chunk_size;
Initial Mingw support with Makefile. Also add a macro JANET_MSVC to distinguish between a windows build (JANET_WINDOWS) and a build with msvc.
2023-01-21 16:37:34 +00:00
state->wbuf.buf = (char *) state->chunk_buf;
fix for crash on windows in src/core/ev.c: initialze state->fromlen before doing WSARecvFrom() to prevent crash (likely caused by the memcpy() of `state->from` at line 2301 with the memcpy length set to -1)
2023-05-30 17:33:34 +00:00
state->fromlen = sizeof(state->from);
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
status = WSARecvFrom((SOCKET) stream->handle, &state->wbuf, 1,
More work on windows networking code. Remove use of WSARecv and WSASend since for whatever reason they seem suspect. We may want to revisit this later.
2021-01-12 00:00:31 +00:00
NULL, &state->flags, &state->from, &state->fromlen, &state->overlapped, NULL);
Fix some windows issues.
2020-11-14 20:40:39 +00:00
if (status && (WSA_IO_PENDING != WSAGetLastError())) {
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
janet_cancel(fiber, janet_ev_lasterr());
janet_async_end(fiber);
return;
Fix some windows issues.
2020-11-14 20:40:39 +00:00
}
} else
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
#endif
Fix some windows issues.
2020-11-14 20:40:39 +00:00
{
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
/* Some handles (not all) read from the offset in lpOverlapped
Merge branch 'master' of github.com:janet-lang/janet
2022-01-08 17:27:14 +00:00
* if its not set before calling `ReadFile` these streams will always read from offset 0 */
windows: fix ev/read hang when called on fs stream handles returned by CreateFileA and FILE_FLAG_OVERLAPPED support reading from arbitrary offsets. The offset is passed to ReadFile in through the OVERLAPPED structure. Since state->overlapped is zeroed ev_machine_read ReadFile would always read from the start of the file and never finish This commit changes ev_machine_read to update the offset to the number of bytes read before calling ReadFile.
2022-01-07 21:32:39 +00:00
state->overlapped.Offset = (DWORD) state->bytes_read;
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
status = ReadFile(stream->handle, state->chunk_buf, chunk_size, NULL, &state->overlapped);
Fix windows build with JANET_NO_NET - #1055
2023-01-21 16:55:42 +00:00
if (!status && (ERROR_IO_PENDING != GetLastError())) {
if (GetLastError() == ERROR_BROKEN_PIPE) {
Add more network and subprocess testing with redirection.
2021-01-12 00:32:56 +00:00
if (state->bytes_read) {
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
janet_schedule(fiber, janet_wrap_buffer(state->buf));
Add more network and subprocess testing with redirection.
2021-01-12 00:32:56 +00:00
} else {
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
janet_schedule(fiber, janet_wrap_nil());
Add more network and subprocess testing with redirection.
2021-01-12 00:32:56 +00:00
}
Broken Pipe error on windows should just be end of stream.
2020-11-14 22:03:51 +00:00
} else {
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
janet_cancel(fiber, janet_ev_lasterr());
Broken Pipe error on windows should just be end of stream.
2020-11-14 22:03:51 +00:00
}
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
janet_async_end(fiber);
return;
Fix some windows issues.
2020-11-14 20:40:39 +00:00
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
}
Remove some extra fiber state and use a flag.
2023-10-07 19:07:05 +00:00
fiber->flags |= JANET_FIBER_EV_FLAG_IN_FLIGHT;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
Fix some windows issues.
2020-11-14 20:40:39 +00:00
break;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
#else
Fix #610 - POLLHUP should cause us to continue reading.
2021-01-23 20:26:24 +00:00
case JANET_ASYNC_EVENT_ERR: {
Try again for windows build.
2020-11-14 21:01:52 +00:00
if (state->bytes_read) {
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_schedule(fiber, janet_wrap_buffer(state->buf));
Try again for windows build.
2020-11-14 21:01:52 +00:00
} else {
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_schedule(fiber, janet_wrap_nil());
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
}
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_async_end(fiber);
break;
Try again for windows build.
2020-11-14 21:01:52 +00:00
}
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
read_more:
Fix #610 - POLLHUP should cause us to continue reading.
2021-01-23 20:26:24 +00:00
case JANET_ASYNC_EVENT_HUP:
Make epoll event loop use EPOLLET and not reregister file descriptors. This results in fewer system calls and presumably more effcient code. It also brings the epoll (and kqueue) code more in line with how the windows IOCP code works, incidentally.
2023-10-05 03:35:04 +00:00
case JANET_ASYNC_EVENT_USER:
Add :all option to ev/read. Brings ev/read more in line with file/read.
2020-12-30 02:37:59 +00:00
case JANET_ASYNC_EVENT_READ: {
Try again for windows build.
2020-11-14 21:01:52 +00:00
JanetBuffer *buffer = state->buf;
int32_t bytes_left = state->bytes_left;
Remove accidental limit on max read size.
2021-09-21 14:55:40 +00:00
int32_t read_limit = state->is_chunk ? (bytes_left > 4096 ? 4096 : bytes_left) : bytes_left;
Add :all option to ev/read. Brings ev/read more in line with file/read.
2020-12-30 02:37:59 +00:00
janet_buffer_extra(buffer, read_limit);
Try again for windows build.
2020-11-14 21:01:52 +00:00
ssize_t nread;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
#ifdef JANET_NET
Try again for windows build.
2020-11-14 21:01:52 +00:00
char saddr[256];
socklen_t socklen = sizeof(saddr);
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
#endif
Try again for windows build.
2020-11-14 21:01:52 +00:00
do {
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
#ifdef JANET_NET
Try again for windows build.
2020-11-14 21:01:52 +00:00
if (state->mode == JANET_ASYNC_READMODE_RECVFROM) {
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
nread = recvfrom(stream->handle, buffer->data + buffer->count, read_limit, state->flags,
Try again for windows build.
2020-11-14 21:01:52 +00:00
(struct sockaddr *)&saddr, &socklen);
} else if (state->mode == JANET_ASYNC_READMODE_RECV) {
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
nread = recv(stream->handle, buffer->data + buffer->count, read_limit, state->flags);
Try again for windows build.
2020-11-14 21:01:52 +00:00
} else
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
#endif
Try again for windows build.
2020-11-14 21:01:52 +00:00
{
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
nread = read(stream->handle, buffer->data + buffer->count, read_limit);
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
Try again for windows build.
2020-11-14 21:01:52 +00:00
} while (nread == -1 && errno == EINTR);
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
Try again for windows build.
2020-11-14 21:01:52 +00:00
/* Check for errors - special case errors that can just be waited on to fix */
if (nread == -1) {
When reading from a stream, EPIPE is considered EOS. Before, EPIPE caused an error, but in most cases it is better to consider it an end of stream. In the future, we may want to allow cusomtization of this behavior with flags on the stream but for now let's keep it simpler.
2020-11-16 22:42:09 +00:00
if (errno == EAGAIN || errno == EWOULDBLOCK) {
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
break;
When reading from a stream, EPIPE is considered EOS. Before, EPIPE caused an error, but in most cases it is better to consider it an end of stream. In the future, we may want to allow cusomtization of this behavior with flags on the stream but for now let's keep it simpler.
2020-11-16 22:42:09 +00:00
}
/* In stream protocols, a pipe error is end of stream */
if (errno == EPIPE && (state->mode != JANET_ASYNC_READMODE_RECVFROM)) {
nread = 0;
} else {
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_cancel(fiber, janet_ev_lasterr());
janet_async_end(fiber);
break;
When reading from a stream, EPIPE is considered EOS. Before, EPIPE caused an error, but in most cases it is better to consider it an end of stream. In the future, we may want to allow cusomtization of this behavior with flags on the stream but for now let's keep it simpler.
2020-11-16 22:42:09 +00:00
}
Try again for windows build.
2020-11-14 21:01:52 +00:00
}
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
Try again for windows build.
2020-11-14 21:01:52 +00:00
/* Only allow 0-length packets in recv-from. In stream protocols, a zero length packet is EOS. */
state->bytes_read += nread;
if (state->bytes_read == 0 && (state->mode != JANET_ASYNC_READMODE_RECVFROM)) {
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_schedule(fiber, janet_wrap_nil());
janet_async_end(fiber);
break;
Try again for windows build.
2020-11-14 21:01:52 +00:00
}
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
Try again for windows build.
2020-11-14 21:01:52 +00:00
/* Increment buffer counts */
buffer->count += nread;
bytes_left -= nread;
state->bytes_left = bytes_left;
/* Resume if done */
if (!state->is_chunk || bytes_left == 0 || nread == 0) {
Janet resume_val;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
#ifdef JANET_NET
Try again for windows build.
2020-11-14 21:01:52 +00:00
if (state->mode == JANET_ASYNC_READMODE_RECVFROM) {
void *abst = janet_abstract(&janet_address_type, socklen);
memcpy(abst, &saddr, socklen);
resume_val = janet_wrap_abstract(abst);
} else
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
#endif
Try again for windows build.
2020-11-14 21:01:52 +00:00
{
resume_val = janet_wrap_buffer(buffer);
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_schedule(fiber, resume_val);
janet_async_end(fiber);
break;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
/* Read some more if possible */
goto read_more;
Try again for windows build.
2020-11-14 21:01:52 +00:00
}
break;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
#endif
Try again for windows build.
2020-11-14 21:01:52 +00:00
}
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
static void janet_ev_read_generic(JanetStream *stream, JanetBuffer *buf, int32_t nbytes, int is_chunked, JanetReadMode mode, int flags) {
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
JanetFiber *f = janet_vm.root_fiber;
StateRead *state = (StateRead *) janet_async_start(f, stream, JANET_ASYNC_LISTEN_READ, ev_callback_read, sizeof(StateRead));
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
state->is_chunk = is_chunked;
state->buf = buf;
state->bytes_left = nbytes;
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
state->bytes_read = 0;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
state->mode = mode;
#ifdef JANET_WINDOWS
state->flags = (DWORD) flags;
#else
state->flags = flags;
#endif
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
ev_callback_read(f, JANET_ASYNC_EVENT_USER);
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
void janet_ev_read(JanetStream *stream, JanetBuffer *buf, int32_t nbytes) {
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
janet_ev_read_generic(stream, buf, nbytes, 0, JANET_ASYNC_READMODE_READ, 0);
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
void janet_ev_readchunk(JanetStream *stream, JanetBuffer *buf, int32_t nbytes) {
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
janet_ev_read_generic(stream, buf, nbytes, 1, JANET_ASYNC_READMODE_READ, 0);
}
#ifdef JANET_NET
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
void janet_ev_recv(JanetStream *stream, JanetBuffer *buf, int32_t nbytes, int flags) {
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
janet_ev_read_generic(stream, buf, nbytes, 0, JANET_ASYNC_READMODE_RECV, flags);
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
void janet_ev_recvchunk(JanetStream *stream, JanetBuffer *buf, int32_t nbytes, int flags) {
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
janet_ev_read_generic(stream, buf, nbytes, 1, JANET_ASYNC_READMODE_RECV, flags);
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
void janet_ev_recvfrom(JanetStream *stream, JanetBuffer *buf, int32_t nbytes, int flags) {
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
janet_ev_read_generic(stream, buf, nbytes, 0, JANET_ASYNC_READMODE_RECVFROM, flags);
}
#endif
/*
* State machine for write/send/send-to
*/
typedef enum {
JANET_ASYNC_WRITEMODE_WRITE,
JANET_ASYNC_WRITEMODE_SEND,
Redo async connect code to be moved out of ev.c. Async connect is different than write.
2023-09-24 17:08:40 +00:00
JANET_ASYNC_WRITEMODE_SENDTO
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
} JanetWriteMode;
typedef struct {
#ifdef JANET_WINDOWS
OVERLAPPED overlapped;
Fix windows build with JANET_NO_NET - #1055
2023-01-21 16:55:42 +00:00
DWORD flags;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
#ifdef JANET_NET
WSABUF wbuf;
#endif
#else
int flags;
int32_t start;
#endif
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
union {
JanetBuffer *buf;
const uint8_t *str;
} src;
int is_buffer;
JanetWriteMode mode;
void *dest_abst;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
} StateWrite;
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
void ev_callback_write(JanetFiber *fiber, JanetAsyncEvent event) {
JanetStream *stream = fiber->ev_stream;
StateWrite *state = (StateWrite *) fiber->ev_state;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
switch (event) {
default:
break;
Quick fix for discussion #1253 Protect against garbage collection during connect.
2023-08-13 16:59:21 +00:00
case JANET_ASYNC_EVENT_MARK: {
Redo async connect code to be moved out of ev.c. Async connect is different than write.
2023-09-24 17:08:40 +00:00
janet_mark(state->is_buffer
? janet_wrap_buffer(state->src.buf)
: janet_wrap_string(state->src.str));
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
if (state->mode == JANET_ASYNC_WRITEMODE_SENDTO) {
janet_mark(janet_wrap_abstract(state->dest_abst));
}
break;
Quick fix for discussion #1253 Protect against garbage collection during connect.
2023-08-13 16:59:21 +00:00
}
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
case JANET_ASYNC_EVENT_CLOSE:
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_cancel(fiber, janet_cstringv("stream closed"));
janet_async_end(fiber);
break;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
#ifdef JANET_WINDOWS
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
case JANET_ASYNC_EVENT_FAILED:
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
case JANET_ASYNC_EVENT_COMPLETE: {
/* Called when write finished */
Update ev.c for removing msvc warnings
2023-10-07 19:11:14 +00:00
uint32_t ev_bytes = (uint32_t) state->overlapped.InternalHigh;
Reuse overlapped overhead on windows for something useful.
2023-10-07 18:25:20 +00:00
if (ev_bytes == 0 && (state->mode != JANET_ASYNC_WRITEMODE_SENDTO)) {
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
janet_cancel(fiber, janet_cstringv("disconnect"));
janet_async_end(fiber);
return;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
janet_schedule(fiber, janet_wrap_nil());
janet_async_end(fiber);
return;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
break;
case JANET_ASYNC_EVENT_USER: {
/* Begin write */
Try again for windows build.
2020-11-14 21:01:52 +00:00
int32_t len;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
const uint8_t *bytes;
if (state->is_buffer) {
/* If buffer, convert to string. */
/* TODO - be more efficient about this */
JanetBuffer *buffer = state->src.buf;
JanetString str = janet_string(buffer->data, buffer->count);
bytes = str;
len = buffer->count;
state->is_buffer = 0;
state->src.str = str;
} else {
bytes = state->src.str;
len = janet_string_length(bytes);
}
memset(&(state->overlapped), 0, sizeof(WSAOVERLAPPED));
int status;
#ifdef JANET_NET
More work on windows networking code. Remove use of WSARecv and WSASend since for whatever reason they seem suspect. We may want to revisit this later.
2021-01-12 00:00:31 +00:00
if (state->mode == JANET_ASYNC_WRITEMODE_SENDTO) {
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
SOCKET sock = (SOCKET) stream->handle;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
state->wbuf.buf = (char *) bytes;
state->wbuf.len = len;
More work on windows networking code. Remove use of WSARecv and WSASend since for whatever reason they seem suspect. We may want to revisit this later.
2021-01-12 00:00:31 +00:00
const struct sockaddr *to = state->dest_abst;
int tolen = (int) janet_abstract_size((void *) to);
status = WSASendTo(sock, &state->wbuf, 1, NULL, state->flags, to, tolen, &state->overlapped, NULL);
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
if (status) {
if (WSA_IO_PENDING == WSAGetLastError()) {
Remove some extra fiber state and use a flag.
2023-10-07 19:07:05 +00:00
fiber->flags |= JANET_FIBER_EV_FLAG_IN_FLIGHT;
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
} else {
janet_cancel(fiber, janet_ev_lasterr());
janet_async_end(fiber);
return;
}
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
} else
#endif
{
Add utitities for interrupting the event loop. janet_loop1_interrupt makes the event loop compatible with safe interruptions for custom scheduling. Does this by exposing custom events on the event loop. A custom event schedules a function pointer to run in a way that can interrupt epoll_wait/poll/GetQueuedCompletionStatus.
2021-07-25 01:30:36 +00:00
/*
* File handles in IOCP need to specify this if they are writing to the
Add :d switch to os/spawn. This allows for starting processes that can be turned into zombies.
2021-07-24 16:55:04 +00:00
* ends of files, like how this is used here.
* If the underlying resource doesn't support seeking
* byte offsets, they will be ignored
* but this otherwise writes to the end of the file in question
* Right now, os/open streams aren't seekable, so this works.
* for more details see the lpOverlapped parameter in
* https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-writefile
*/
Add fixes for :write on filestreams that come from os/open
2021-07-24 08:30:00 +00:00
state->overlapped.Offset = (DWORD) 0xFFFFFFFF;
state->overlapped.OffsetHigh = (DWORD) 0xFFFFFFFF;
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
status = WriteFile(stream->handle, bytes, len, NULL, &state->overlapped);
if (!status) {
if (ERROR_IO_PENDING == GetLastError()) {
Remove some extra fiber state and use a flag.
2023-10-07 19:07:05 +00:00
fiber->flags |= JANET_FIBER_EV_FLAG_IN_FLIGHT;
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
} else {
janet_cancel(fiber, janet_ev_lasterr());
janet_async_end(fiber);
return;
}
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
}
}
break;
#else
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
case JANET_ASYNC_EVENT_ERR:
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_cancel(fiber, janet_cstringv("stream err"));
janet_async_end(fiber);
break;
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
case JANET_ASYNC_EVENT_HUP:
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_cancel(fiber, janet_cstringv("stream hup"));
janet_async_end(fiber);
break;
Make epoll event loop use EPOLLET and not reregister file descriptors. This results in fewer system calls and presumably more effcient code. It also brings the epoll (and kqueue) code more in line with how the windows IOCP code works, incidentally.
2023-10-05 03:35:04 +00:00
case JANET_ASYNC_EVENT_USER:
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
case JANET_ASYNC_EVENT_WRITE: {
int32_t start, len;
const uint8_t *bytes;
start = state->start;
if (state->is_buffer) {
JanetBuffer *buffer = state->src.buf;
bytes = buffer->data;
len = buffer->count;
} else {
bytes = state->src.str;
len = janet_string_length(bytes);
}
ssize_t nwrote = 0;
if (start < len) {
int32_t nbytes = len - start;
void *dest_abst = state->dest_abst;
do {
#ifdef JANET_NET
if (state->mode == JANET_ASYNC_WRITEMODE_SENDTO) {
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
nwrote = sendto(stream->handle, bytes + start, nbytes, state->flags,
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
(struct sockaddr *) dest_abst, janet_abstract_size(dest_abst));
} else if (state->mode == JANET_ASYNC_WRITEMODE_SEND) {
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
nwrote = send(stream->handle, bytes + start, nbytes, state->flags);
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
} else
#endif
{
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
nwrote = write(stream->handle, bytes + start, nbytes);
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
} while (nwrote == -1 && errno == EINTR);
/* Handle write errors */
if (nwrote == -1) {
if (errno == EAGAIN || errno == EWOULDBLOCK) break;
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_cancel(fiber, janet_ev_lasterr());
janet_async_end(fiber);
break;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
/* Unless using datagrams, empty message is a disconnect */
if (nwrote == 0 && !dest_abst) {
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_cancel(fiber, janet_cstringv("disconnect"));
janet_async_end(fiber);
break;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
if (nwrote > 0) {
start += nwrote;
} else {
start = len;
}
}
state->start = start;
if (start >= len) {
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
janet_schedule(fiber, janet_wrap_nil());
janet_async_end(fiber);
break;
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
break;
}
break;
#endif
}
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
static void janet_ev_write_generic(JanetStream *stream, void *buf, void *dest_abst, JanetWriteMode mode, int is_buffer, int flags) {
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
JanetFiber *f = janet_vm.root_fiber;
StateWrite *state = (StateWrite *) janet_async_start(f, stream, JANET_ASYNC_LISTEN_WRITE,
ev_callback_write, sizeof(StateWrite));
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
state->is_buffer = is_buffer;
state->src.buf = buf;
state->dest_abst = dest_abst;
state->mode = mode;
#ifdef JANET_WINDOWS
state->flags = (DWORD) flags;
#else
state->flags = flags;
Make epoll event loop use EPOLLET and not reregister file descriptors. This results in fewer system calls and presumably more effcient code. It also brings the epoll (and kqueue) code more in line with how the windows IOCP code works, incidentally.
2023-10-05 03:35:04 +00:00
state->start = 0;
Lots of work to make iocp work again. Big issue with IOCP vs. poll variants is that the overlapped structures have a longer lifetime than intermediate state needed for epoll. One cannot free overlapped structures after closing a handle/socket, like one can do with any intermediate state when using readiness-based IO.
2023-10-07 15:01:35 +00:00
#endif
Simpler async model that is better suited to epoll
2023-10-06 05:37:19 +00:00
ev_callback_write(f, JANET_ASYNC_EVENT_USER);
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
void janet_ev_write_buffer(JanetStream *stream, JanetBuffer *buf) {
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
janet_ev_write_generic(stream, buf, NULL, JANET_ASYNC_WRITEMODE_WRITE, 1, 0);
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
void janet_ev_write_string(JanetStream *stream, JanetString str) {
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
janet_ev_write_generic(stream, (void *) str, NULL, JANET_ASYNC_WRITEMODE_WRITE, 0, 0);
}
#ifdef JANET_NET
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
void janet_ev_send_buffer(JanetStream *stream, JanetBuffer *buf, int flags) {
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
janet_ev_write_generic(stream, buf, NULL, JANET_ASYNC_WRITEMODE_SEND, 1, flags);
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
void janet_ev_send_string(JanetStream *stream, JanetString str, int flags) {
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
janet_ev_write_generic(stream, (void *) str, NULL, JANET_ASYNC_WRITEMODE_SEND, 0, flags);
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
void janet_ev_sendto_buffer(JanetStream *stream, JanetBuffer *buf, void *dest, int flags) {
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
janet_ev_write_generic(stream, buf, dest, JANET_ASYNC_WRITEMODE_SENDTO, 1, flags);
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
void janet_ev_sendto_string(JanetStream *stream, JanetString str, void *dest, int flags) {
Move read/write functions into ev.c from net.c This code can also be used for non-network streams.
2020-11-14 17:48:23 +00:00
janet_ev_write_generic(stream, (void *) str, dest, JANET_ASYNC_WRITEMODE_SENDTO, 0, flags);
}
#endif
Several changes to move pipe creation back into ev.c
2020-11-18 16:53:36 +00:00
/* For a pipe ID */
#ifdef JANET_WINDOWS
static volatile long PipeSerialNumber;
#endif
Pass non-blocking pipes to subprocesses on non-windows platform.
2021-10-14 18:57:51 +00:00
/*
* mode = 0: both sides non-blocking.
* mode = 1: only read side non-blocking: write side sent to subprocess
* mode = 2: only write side non-blocking: read side sent to subprocess
*/
Add os/proc-close to close all pipes associated with a subprocess. This will not leak handles until the GC runs in most use cases.
2021-01-16 21:11:07 +00:00
int janet_make_pipe(JanetHandle handles[2], int mode) {
Several changes to move pipe creation back into ev.c
2020-11-18 16:53:36 +00:00
#ifdef JANET_WINDOWS
/*
* On windows, the built in CreatePipe function doesn't support overlapped IO
* so we lift from the windows source code and modify for our own version.
*/
Add os/proc-close to close all pipes associated with a subprocess. This will not leak handles until the GC runs in most use cases.
2021-01-16 21:11:07 +00:00
JanetHandle shandle, chandle;
Initial Mingw support with Makefile. Also add a macro JANET_MSVC to distinguish between a windows build (JANET_WINDOWS) and a build with msvc.
2023-01-21 16:37:34 +00:00
CHAR PipeNameBuffer[MAX_PATH];
Fix subprocess spawning on windows. Also fix (:read stream :all)
2021-01-11 17:10:23 +00:00
SECURITY_ATTRIBUTES saAttr;
memset(&saAttr, 0, sizeof(saAttr));
saAttr.nLength = sizeof(saAttr);
saAttr.bInheritHandle = TRUE;
Several changes to move pipe creation back into ev.c
2020-11-18 16:53:36 +00:00
sprintf(PipeNameBuffer,
"\\\\.\\Pipe\\JanetPipeFile.%08x.%08x",
Initial Mingw support with Makefile. Also add a macro JANET_MSVC to distinguish between a windows build (JANET_WINDOWS) and a build with msvc.
2023-01-21 16:37:34 +00:00
(unsigned int) GetCurrentProcessId(),
(unsigned int) InterlockedIncrement(&PipeSerialNumber));
Add os/proc-close to close all pipes associated with a subprocess. This will not leak handles until the GC runs in most use cases.
2021-01-16 21:11:07 +00:00
/* server handle goes to subprocess */
shandle = CreateNamedPipeA(
Several changes to move pipe creation back into ev.c
2020-11-18 16:53:36 +00:00
PipeNameBuffer,
Add os/proc-close to close all pipes associated with a subprocess. This will not leak handles until the GC runs in most use cases.
2021-01-16 21:11:07 +00:00
(mode == 2 ? PIPE_ACCESS_INBOUND : PIPE_ACCESS_OUTBOUND) | FILE_FLAG_OVERLAPPED,
PIPE_TYPE_BYTE | PIPE_WAIT,
255, /* Max number of pipes for duplication. */
Several changes to move pipe creation back into ev.c
2020-11-18 16:53:36 +00:00
4096, /* Out buffer size */
4096, /* In buffer size */
120 * 1000, /* Timeout in ms */
Fix subprocess spawning on windows. Also fix (:read stream :all)
2021-01-11 17:10:23 +00:00
&saAttr);
Add os/proc-close to close all pipes associated with a subprocess. This will not leak handles until the GC runs in most use cases.
2021-01-16 21:11:07 +00:00
if (shandle == INVALID_HANDLE_VALUE) {
return -1;
}
/* we keep client handle */
chandle = CreateFileA(
Several changes to move pipe creation back into ev.c
2020-11-18 16:53:36 +00:00
PipeNameBuffer,
Add os/proc-close to close all pipes associated with a subprocess. This will not leak handles until the GC runs in most use cases.
2021-01-16 21:11:07 +00:00
(mode == 2 ? GENERIC_WRITE : GENERIC_READ),
Several changes to move pipe creation back into ev.c
2020-11-18 16:53:36 +00:00
0,
Fix subprocess spawning on windows. Also fix (:read stream :all)
2021-01-11 17:10:23 +00:00
&saAttr,
Several changes to move pipe creation back into ev.c
2020-11-18 16:53:36 +00:00
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL | FILE_FLAG_OVERLAPPED,
NULL);
Add os/proc-close to close all pipes associated with a subprocess. This will not leak handles until the GC runs in most use cases.
2021-01-16 21:11:07 +00:00
if (chandle == INVALID_HANDLE_VALUE) {
CloseHandle(shandle);
Several changes to move pipe creation back into ev.c
2020-11-18 16:53:36 +00:00
return -1;
}
Add os/proc-close to close all pipes associated with a subprocess. This will not leak handles until the GC runs in most use cases.
2021-01-16 21:11:07 +00:00
if (mode == 2) {
handles[0] = shandle;
handles[1] = chandle;
} else {
handles[0] = chandle;
handles[1] = shandle;
}
Several changes to move pipe creation back into ev.c
2020-11-18 16:53:36 +00:00
return 0;
#else
if (pipe(handles)) return -1;
Fix leaking file descriptors to subprocess causing hangs.
2021-11-27 00:44:11 +00:00
if (mode != 2 && fcntl(handles[0], F_SETFD, FD_CLOEXEC)) goto error;
if (mode != 1 && fcntl(handles[1], F_SETFD, FD_CLOEXEC)) goto error;
Pass non-blocking pipes to subprocesses on non-windows platform.
2021-10-14 18:57:51 +00:00
if (mode != 2 && fcntl(handles[0], F_SETFL, O_NONBLOCK)) goto error;
if (mode != 1 && fcntl(handles[1], F_SETFL, O_NONBLOCK)) goto error;
Several changes to move pipe creation back into ev.c
2020-11-18 16:53:36 +00:00
return 0;
Use PostQueuedCompletionStatus for threaded calls on windows. Ore efficient than using a self pipe.
2021-01-03 17:04:21 +00:00
error:
close(handles[0]);
close(handles[1]);
return -1;
Several changes to move pipe creation back into ev.c
2020-11-18 16:53:36 +00:00
#endif
}
Start with ev module.
2020-05-28 15:39:40 +00:00
/* C functions */
Update ev.c to use the new binding style that provides source information
2021-07-26 05:25:38 +00:00
JANET_CORE_FN(cfun_ev_go,
Add doc for ev/go Document that you can pass a function instead of a fiber to ev/go
2022-09-14 00:17:53 +00:00
"(ev/go fiber-or-fun &opt value supervisor)",
Fix some docstring typos.
2023-01-19 03:41:06 +00:00
"Put a fiber on the event loop to be resumed later. If a function is used, it is wrapped "
Add doc for ev/go Document that you can pass a function instead of a fiber to ev/go
2022-09-14 00:17:53 +00:00
"with `fiber/new` first. "
"Optionally pass a value to resume with, otherwise resumes with nil. Returns the fiber. "
Fix typos in ev/go ev/select
2021-08-27 07:06:14 +00:00
"An optional `core/channel` can be provided as a supervisor. When various "
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
"events occur in the newly scheduled fiber, an event will be pushed to the supervisor. "
"If not provided, the new fiber will inherit the current supervisor.") {
Go back to a single supervisor channel per fiber. We now also use the fiber mask to figure out which flags to wait for.
2021-01-13 03:35:28 +00:00
janet_arity(argc, 1, 3);
Fix bug with ev/go when passing supervisor and value. value was incorrectly set to nil in these cases. Also fix some typos in core docstrings.
2021-06-25 23:58:19 +00:00
Janet value = argc >= 2 ? argv[1] : janet_wrap_nil();
Fix threaded supervisor channels - Fix #766 Some pointer casting with abstract types was incorrect, resulting in strange behavior when trying to use supervisor channels that were threaded. This fix also adds the ability to supply a supervisor channel directly when creating a thread.
2021-08-17 02:14:06 +00:00
void *supervisor = janet_optabstract(argv, argc, 2, &janet_channel_type, janet_vm.root_fiber->supervisor_channel);
Allow passing a function to directly to ev/go. Makes ev/call less useful but ev/go more useful. No need to construct as many identical intermediate fibers.
2021-08-31 03:22:22 +00:00
JanetFiber *fiber;
if (janet_checktype(argv[0], JANET_FUNCTION)) {
/* Create a fiber for the user */
JanetFunction *func = janet_unwrap_function(argv[0]);
if (func->def->min_arity > 1) {
janet_panicf("task function must accept 0 or 1 arguments");
}
fiber = janet_fiber(func, 64, func->def->min_arity, &value);
fiber->flags |=
JANET_FIBER_MASK_ERROR |
JANET_FIBER_MASK_USER0 |
JANET_FIBER_MASK_USER1 |
JANET_FIBER_MASK_USER2 |
JANET_FIBER_MASK_USER3 |
JANET_FIBER_MASK_USER4;
if (!janet_vm.fiber->env) {
janet_vm.fiber->env = janet_table(0);
}
fiber->env = janet_table(0);
fiber->env->proto = janet_vm.fiber->env;
} else {
fiber = janet_getfiber(argv, 0);
}
Fix threaded supervisor channels - Fix #766 Some pointer casting with abstract types was incorrect, resulting in strange behavior when trying to use supervisor channels that were threaded. This fix also adds the ability to supply a supervisor channel directly when creating a thread.
2021-08-17 02:14:06 +00:00
fiber->supervisor_channel = supervisor;
Update os/shell to be non-blocking as well.
2021-01-18 22:44:22 +00:00
janet_schedule(fiber, value);
Allow passing a function to directly to ev/go. Makes ev/call less useful but ev/go more useful. No need to construct as many identical intermediate fibers.
2021-08-31 03:22:22 +00:00
return janet_wrap_fiber(fiber);
Start with ev module.
2020-05-28 15:39:40 +00:00
}
Allow shorthand for setting task-id on new threads with flag. Avoids the need to wrap function bodies in closures in many cases.
2022-07-03 17:08:21 +00:00
#define JANET_THREAD_SUPERVISOR_FLAG 0x100
Add ev/thread and ev/do-thread. - Also fix setting supervisor with net/accept-loop.
2021-01-22 18:52:45 +00:00
/* For ev/thread - Run an interpreter in the new thread. */
static JanetEVGenericMessage janet_go_thread_subr(JanetEVGenericMessage args) {
JanetBuffer *buffer = (JanetBuffer *) args.argp;
const uint8_t *nextbytes = buffer->data;
const uint8_t *endbytes = nextbytes + buffer->count;
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
uint32_t flags = args.tag;
args.tag = 0;
Add ev/thread and ev/do-thread. - Also fix setting supervisor with net/accept-loop.
2021-01-22 18:52:45 +00:00
janet_init();
Add sandboxing API. The sandboxing API is meant to make janet a bit more attractive for certain application embedding use cases. The sandboxing API puts limits on what system resources the interpreter can access.
2023-02-06 14:41:04 +00:00
janet_vm.sandbox_flags = (uint32_t) args.argi;
Add ev/thread and ev/do-thread. - Also fix setting supervisor with net/accept-loop.
2021-01-22 18:52:45 +00:00
JanetTryState tstate;
JanetSignal signal = janet_try(&tstate);
if (!signal) {
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
/* Set abstract registry */
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
if (!(flags & 0x2)) {
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
Janet aregv = janet_unmarshal(nextbytes, endbytes - nextbytes,
JANET_MARSHAL_UNSAFE, NULL, &nextbytes);
if (!janet_checktype(aregv, JANET_TABLE)) janet_panic("expected table for abstract registry");
janet_vm.abstract_registry = janet_unwrap_table(aregv);
Fix memory leak and use after free Use after free was caused by missing janet_gcroot call when setting up thread.
2021-08-20 02:51:53 +00:00
janet_gcroot(janet_wrap_table(janet_vm.abstract_registry));
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
}
Fix threaded supervisor channels - Fix #766 Some pointer casting with abstract types was incorrect, resulting in strange behavior when trying to use supervisor channels that were threaded. This fix also adds the ability to supply a supervisor channel directly when creating a thread.
2021-08-17 02:14:06 +00:00
/* Get supervsior */
Allow shorthand for setting task-id on new threads with flag. Avoids the need to wrap function bodies in closures in many cases.
2022-07-03 17:08:21 +00:00
if (flags & JANET_THREAD_SUPERVISOR_FLAG) {
Fix threaded supervisor channels - Fix #766 Some pointer casting with abstract types was incorrect, resulting in strange behavior when trying to use supervisor channels that were threaded. This fix also adds the ability to supply a supervisor channel directly when creating a thread.
2021-08-17 02:14:06 +00:00
Janet sup =
janet_unmarshal(nextbytes, endbytes - nextbytes,
JANET_MARSHAL_UNSAFE, NULL, &nextbytes);
Fix ev thread swallowing error in some case.
2021-08-27 16:46:42 +00:00
/* Hack - use a global variable to avoid longjmp clobber */
janet_vm.user = janet_unwrap_pointer(sup);
Fix threaded supervisor channels - Fix #766 Some pointer casting with abstract types was incorrect, resulting in strange behavior when trying to use supervisor channels that were threaded. This fix also adds the ability to supply a supervisor channel directly when creating a thread.
2021-08-17 02:14:06 +00:00
}
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
/* Set cfunction registry */
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
if (!(flags & 0x4)) {
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
uint32_t count1;
memcpy(&count1, nextbytes, sizeof(count1));
size_t count = (size_t) count1;
if (count > (endbytes - nextbytes) * sizeof(JanetCFunRegistry)) {
janet_panic("thread message invalid");
}
janet_vm.registry_count = count;
janet_vm.registry_cap = count;
janet_vm.registry = janet_malloc(count * sizeof(JanetCFunRegistry));
if (janet_vm.registry == NULL) {
JANET_OUT_OF_MEMORY;
}
janet_vm.registry_dirty = 1;
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
nextbytes += sizeof(uint32_t);
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
memcpy(janet_vm.registry, nextbytes, count * sizeof(JanetCFunRegistry));
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
nextbytes += count * sizeof(JanetCFunRegistry);
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
}
Add ev/thread and ev/do-thread. - Also fix setting supervisor with net/accept-loop.
2021-01-22 18:52:45 +00:00
Janet fiberv = janet_unmarshal(nextbytes, endbytes - nextbytes,
JANET_MARSHAL_UNSAFE, NULL, &nextbytes);
Janet value = janet_unmarshal(nextbytes, endbytes - nextbytes,
JANET_MARSHAL_UNSAFE, NULL, &nextbytes);
Allow passing function to ev/thread. Convenient when there is no need to create an entire fiber.
2021-08-31 03:04:15 +00:00
JanetFiber *fiber;
if (!janet_checktype(fiberv, JANET_FIBER)) {
if (!janet_checktype(fiberv, JANET_FUNCTION)) {
changed some error messages 'x|y' -> 'x or y'
2023-05-23 04:58:52 +00:00
janet_panicf("expected function or fiber, got %v", fiberv);
Allow passing function to ev/thread. Convenient when there is no need to create an entire fiber.
2021-08-31 03:04:15 +00:00
}
JanetFunction *func = janet_unwrap_function(fiberv);
Don't do fiber double arity check.
2023-06-04 17:55:44 +00:00
fiber = janet_fiber(func, 64, func->def->min_arity, &value);
if (fiber == NULL) {
Allow passing function to ev/thread. Convenient when there is no need to create an entire fiber.
2021-08-31 03:04:15 +00:00
janet_panicf("thread function must accept 0 or 1 arguments");
}
fiber->flags |=
JANET_FIBER_MASK_ERROR |
JANET_FIBER_MASK_USER0 |
JANET_FIBER_MASK_USER1 |
JANET_FIBER_MASK_USER2 |
JANET_FIBER_MASK_USER3 |
JANET_FIBER_MASK_USER4;
} else {
fiber = janet_unwrap_fiber(fiberv);
}
Allow shorthand for setting task-id on new threads with flag. Avoids the need to wrap function bodies in closures in many cases.
2022-07-03 17:08:21 +00:00
if (flags & 0x8) {
if (NULL == fiber->env) fiber->env = janet_table(0);
janet_table_put(fiber->env, janet_ckeywordv("task-id"), value);
}
Fix ev thread swallowing error in some case.
2021-08-27 16:46:42 +00:00
fiber->supervisor_channel = janet_vm.user;
Add ev/thread and ev/do-thread. - Also fix setting supervisor with net/accept-loop.
2021-01-22 18:52:45 +00:00
janet_schedule(fiber, value);
janet_loop();
args.tag = JANET_EV_TCTAG_NIL;
} else {
Fix ev thread swallowing error in some case.
2021-08-27 16:46:42 +00:00
void *supervisor = janet_vm.user;
if (NULL != supervisor) {
/* Got a supervisor, write error there */
Janet pair[] = {
janet_ckeywordv("error"),
tstate.payload
};
janet_channel_push((JanetChannel *)supervisor,
Prepare for 1.17.1 release.
2021-08-29 16:29:09 +00:00
janet_wrap_tuple(janet_tuple_n(pair, 2)), 2);
Fix ev thread swallowing error in some case.
2021-08-27 16:46:42 +00:00
} else if (flags & 0x1) {
/* No wait, just print to stderr */
janet_eprintf("thread start failure: %v\n", tstate.payload);
Add ev/thread and ev/do-thread. - Also fix setting supervisor with net/accept-loop.
2021-01-22 18:52:45 +00:00
} else {
Fix ev thread swallowing error in some case.
2021-08-27 16:46:42 +00:00
/* Make ev/thread call from parent thread error */
if (janet_checktype(tstate.payload, JANET_STRING)) {
args.tag = JANET_EV_TCTAG_ERR_STRINGF;
args.argp = strdup((const char *) janet_unwrap_string(tstate.payload));
} else {
args.tag = JANET_EV_TCTAG_ERR_STRING;
args.argp = "failed to start thread";
}
Add ev/thread and ev/do-thread. - Also fix setting supervisor with net/accept-loop.
2021-01-22 18:52:45 +00:00
}
}
Fix destruction order.
2021-08-21 18:30:09 +00:00
janet_restore(&tstate);
Add ev/thread and ev/do-thread. - Also fix setting supervisor with net/accept-loop.
2021-01-22 18:52:45 +00:00
janet_buffer_deinit(buffer);
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
janet_free(buffer);
Add ev/thread and ev/do-thread. - Also fix setting supervisor with net/accept-loop.
2021-01-22 18:52:45 +00:00
janet_deinit();
return args;
}
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
JANET_CORE_FN(cfun_ev_thread,
Allow passing function to ev/thread. Convenient when there is no need to create an entire fiber.
2021-08-31 03:04:15 +00:00
"(ev/thread main &opt value flags supervisor)",
"Run `main` in a new operating system thread, optionally passing `value` "
"to resume with. The parameter `main` can either be a fiber, or a function that accepts "
"0 or 1 arguments. "
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
"Unlike `ev/go`, this function will suspend the current fiber until the thread is complete. "
"If you want to run the thread without waiting for a result, pass the `:n` flag to return nil immediately. "
Fix #773 - fix docstring.
2021-08-27 16:52:03 +00:00
"Otherwise, returns nil. Available flags:\n\n"
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
"* `:n` - return immediately\n"
Allow shorthand for setting task-id on new threads with flag. Avoids the need to wrap function bodies in closures in many cases.
2022-07-03 17:08:21 +00:00
"* `:t` - set the task-id of the new thread to value. The task-id is passed in messages to the supervisor channel.\n"
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
"* `:a` - don't copy abstract registry to new thread (performance optimization)\n"
"* `:c` - don't copy cfunction registry to new thread (performance optimization)") {
Fix threaded supervisor channels - Fix #766 Some pointer casting with abstract types was incorrect, resulting in strange behavior when trying to use supervisor channels that were threaded. This fix also adds the ability to supply a supervisor channel directly when creating a thread.
2021-08-17 02:14:06 +00:00
janet_arity(argc, 1, 4);
Add option to build janet without thread library.
2021-07-17 20:13:28 +00:00
Janet value = argc >= 2 ? argv[1] : janet_wrap_nil();
Allow passing function to ev/thread. Convenient when there is no need to create an entire fiber.
2021-08-31 03:04:15 +00:00
if (!janet_checktype(argv[0], JANET_FUNCTION)) janet_getfiber(argv, 0);
Add option to build janet without thread library.
2021-07-17 20:13:28 +00:00
uint64_t flags = 0;
if (argc >= 3) {
Allow shorthand for setting task-id on new threads with flag. Avoids the need to wrap function bodies in closures in many cases.
2022-07-03 17:08:21 +00:00
flags = janet_getflags(argv, 2, "nact");
Add option to build janet without thread library.
2021-07-17 20:13:28 +00:00
}
Fix threaded supervisor channels - Fix #766 Some pointer casting with abstract types was incorrect, resulting in strange behavior when trying to use supervisor channels that were threaded. This fix also adds the ability to supply a supervisor channel directly when creating a thread.
2021-08-17 02:14:06 +00:00
void *supervisor = janet_optabstract(argv, argc, 3, &janet_channel_type, janet_vm.root_fiber->supervisor_channel);
Allow shorthand for setting task-id on new threads with flag. Avoids the need to wrap function bodies in closures in many cases.
2022-07-03 17:08:21 +00:00
if (NULL != supervisor) flags |= JANET_THREAD_SUPERVISOR_FLAG;
Fix threaded supervisor channels - Fix #766 Some pointer casting with abstract types was incorrect, resulting in strange behavior when trying to use supervisor channels that were threaded. This fix also adds the ability to supply a supervisor channel directly when creating a thread.
2021-08-17 02:14:06 +00:00
Add ev/thread and ev/do-thread. - Also fix setting supervisor with net/accept-loop.
2021-01-22 18:52:45 +00:00
/* Marshal arguments for the new thread. */
Add config support for custom allocators.
2021-03-23 10:00:48 +00:00
JanetBuffer *buffer = janet_malloc(sizeof(JanetBuffer));
Add ev/thread and ev/do-thread. - Also fix setting supervisor with net/accept-loop.
2021-01-22 18:52:45 +00:00
if (NULL == buffer) {
JANET_OUT_OF_MEMORY;
}
janet_buffer_init(buffer, 0);
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
if (!(flags & 0x2)) {
janet_marshal(buffer, janet_wrap_table(janet_vm.abstract_registry), NULL, JANET_MARSHAL_UNSAFE);
}
Allow shorthand for setting task-id on new threads with flag. Avoids the need to wrap function bodies in closures in many cases.
2022-07-03 17:08:21 +00:00
if (flags & JANET_THREAD_SUPERVISOR_FLAG) {
Fix threaded supervisor channels - Fix #766 Some pointer casting with abstract types was incorrect, resulting in strange behavior when trying to use supervisor channels that were threaded. This fix also adds the ability to supply a supervisor channel directly when creating a thread.
2021-08-17 02:14:06 +00:00
janet_marshal(buffer, janet_wrap_abstract(supervisor), NULL, JANET_MARSHAL_UNSAFE);
}
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
if (!(flags & 0x4)) {
Fix ev thread swallowing error in some case.
2021-08-27 16:46:42 +00:00
janet_assert(janet_vm.registry_count <= INT32_MAX, "assert failed size check");
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
uint32_t temp = (uint32_t) janet_vm.registry_count;
janet_buffer_push_bytes(buffer, (uint8_t *) &temp, sizeof(temp));
Update for windows compiler warning.
2021-08-21 00:48:48 +00:00
janet_buffer_push_bytes(buffer, (uint8_t *) janet_vm.registry, (int32_t) janet_vm.registry_count * sizeof(JanetCFunRegistry));
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
}
Add ev/thread and ev/do-thread. - Also fix setting supervisor with net/accept-loop.
2021-01-22 18:52:45 +00:00
janet_marshal(buffer, argv[0], NULL, JANET_MARSHAL_UNSAFE);
janet_marshal(buffer, value, NULL, JANET_MARSHAL_UNSAFE);
Add option to build janet without thread library.
2021-07-17 20:13:28 +00:00
if (flags & 0x1) {
/* Return immediately */
JanetEVGenericMessage arguments;
Working example for threaded channels. Still no marshalling more complex values.
2021-08-15 20:25:07 +00:00
memset(&arguments, 0, sizeof(arguments));
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
arguments.tag = (uint32_t) flags;
Add sandboxing API. The sandboxing API is meant to make janet a bit more attractive for certain application embedding use cases. The sandboxing API puts limits on what system resources the interpreter can access.
2023-02-06 14:41:04 +00:00
arguments.argi = (uint32_t) janet_vm.sandbox_flags;
Add option to build janet without thread library.
2021-07-17 20:13:28 +00:00
arguments.argp = buffer;
arguments.fiber = NULL;
janet_ev_threaded_call(janet_go_thread_subr, arguments, janet_ev_default_threaded_callback);
return janet_wrap_nil();
} else {
Add sandboxing API. The sandboxing API is meant to make janet a bit more attractive for certain application embedding use cases. The sandboxing API puts limits on what system resources the interpreter can access.
2023-02-06 14:41:04 +00:00
janet_ev_threaded_await(janet_go_thread_subr, (uint32_t) flags, (uint32_t) janet_vm.sandbox_flags, buffer);
Add option to build janet without thread library.
2021-07-17 20:13:28 +00:00
}
Add ev/thread and ev/do-thread. - Also fix setting supervisor with net/accept-loop.
2021-01-22 18:52:45 +00:00
}
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
JANET_CORE_FN(cfun_ev_give_supervisor,
"(ev/give-supervisor tag & payload)",
Fix some docstring typos.
2023-01-19 03:41:06 +00:00
"Send a message to the current supervisor channel if there is one. The message will be a "
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
"tuple of all of the arguments combined into a single message, where the first element is tag. "
"By convention, tag should be a keyword indicating the type of message. Returns nil.") {
Fix #593. Also add ev/give-supervisor to the core.
2021-01-16 13:18:07 +00:00
janet_arity(argc, 1, -1);
Fix threaded supervisor channels - Fix #766 Some pointer casting with abstract types was incorrect, resulting in strange behavior when trying to use supervisor channels that were threaded. This fix also adds the ability to supply a supervisor channel directly when creating a thread.
2021-08-17 02:14:06 +00:00
void *chanv = janet_vm.root_fiber->supervisor_channel;
if (NULL != chanv) {
JanetChannel *chan = janet_channel_unwrap(chanv);
Fix #593. Also add ev/give-supervisor to the core.
2021-01-16 13:18:07 +00:00
if (janet_channel_push(chan, janet_wrap_tuple(janet_tuple_n(argv, argc)), 0)) {
janet_await();
}
}
return janet_wrap_nil();
}
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
JANET_NO_RETURN void janet_sleep_await(double sec) {
More updates to the ev library.
2020-05-28 21:51:11 +00:00
JanetTimeout to;
to.when = ts_delta(ts_now(), sec);
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
to.fiber = janet_vm.root_fiber;
Merge branch 'master' into ev
2020-08-16 22:29:57 +00:00
to.is_error = 0;
Overhaul of poll loop, redo ev/select.
2020-10-11 14:07:11 +00:00
to.sched_id = to.fiber->sched_id;
Add ev/deadline and ev/with-deadline. This should be more useful than timeouts in real-world use cases. The deadline system is based on fibers and is target to much more coarse-grained (and therfor reliable) timeouts than things like ev/sleep and timeout arguments.
2020-12-05 16:32:34 +00:00
to.curr_fiber = NULL;
More updates to the ev library.
2020-05-28 21:51:11 +00:00
add_timeout(to);
janet_await();
}
Update ev.c to use the new binding style that provides source information
2021-07-26 05:25:38 +00:00
JANET_CORE_FN(cfun_ev_sleep,
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
"(ev/sleep sec)",
"Suspend the current fiber for sec seconds without blocking the event loop.") {
Make os/execute cooperate with ev module. os/execute, os/proc-wait do not block (currently posix only). This uses the self-pipe trick to turn signals into a pollable entity.
2020-11-29 21:36:21 +00:00
janet_fixarity(argc, 1);
double sec = janet_getnumber(argv, 0);
janet_sleep_await(sec);
}
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
JANET_CORE_FN(cfun_ev_deadline,
"(ev/deadline sec &opt tocancel tocheck)",
"Set a deadline for a fiber `tocheck`. If `tocheck` is not finished after `sec` seconds, "
"`tocancel` will be canceled as with `ev/cancel`. "
"If `tocancel` and `tocheck` are not given, they default to `(fiber/root)` and "
"`(fiber/current)` respectively. Returns `tocancel`.") {
Add ev/deadline and ev/with-deadline. This should be more useful than timeouts in real-world use cases. The deadline system is based on fibers and is target to much more coarse-grained (and therfor reliable) timeouts than things like ev/sleep and timeout arguments.
2020-12-05 16:32:34 +00:00
janet_arity(argc, 1, 3);
double sec = janet_getnumber(argv, 0);
Add janet_vm_save and janet_vm_load. This lets a user multiplex multiple Janet VMs on a single thread or process if they are willing to implement context switching themselves.
2021-07-17 01:59:03 +00:00
JanetFiber *tocancel = janet_optfiber(argv, argc, 1, janet_vm.root_fiber);
JanetFiber *tocheck = janet_optfiber(argv, argc, 2, janet_vm.fiber);
Add ev/deadline and ev/with-deadline. This should be more useful than timeouts in real-world use cases. The deadline system is based on fibers and is target to much more coarse-grained (and therfor reliable) timeouts than things like ev/sleep and timeout arguments.
2020-12-05 16:32:34 +00:00
JanetTimeout to;
to.when = ts_delta(ts_now(), sec);
to.fiber = tocancel;
to.curr_fiber = tocheck;
to.is_error = 0;
to.sched_id = to.fiber->sched_id;
add_timeout(to);
return janet_wrap_fiber(tocancel);
}
Update ev.c to use the new binding style that provides source information
2021-07-26 05:25:38 +00:00
JANET_CORE_FN(cfun_ev_cancel,
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
"(ev/cancel fiber err)",
Address #820 - ev/cancel to work on already scheduled fibers.
2021-09-28 20:41:41 +00:00
"Cancel a suspended fiber in the event loop. Differs from cancel in that it returns the canceled fiber immediately.") {
Merge branch 'master' into ev
2020-08-23 16:25:04 +00:00
janet_fixarity(argc, 2);
JanetFiber *fiber = janet_getfiber(argv, 0);
Janet err = argv[1];
janet_cancel(fiber, err);
return argv[0];
}
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
JANET_CORE_FN(janet_cfun_stream_close,
"(ev/close stream)",
"Close a stream. This should be the same as calling (:close stream) for all streams.") {
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
janet_fixarity(argc, 1);
JanetStream *stream = janet_getabstract(argv, 0, &janet_stream_type);
janet_stream_close(stream);
return argv[0];
}
Update ev.c to use the new binding style that provides source information
2021-07-26 05:25:38 +00:00
JANET_CORE_FN(janet_cfun_stream_read,
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
"(ev/read stream n &opt buffer timeout)",
"Read up to n bytes into a buffer asynchronously from a stream. `n` can also be the keyword "
"`:all` to read into the buffer until end of stream. "
"Optionally provide a buffer to write into "
"as well as a timeout in seconds after which to cancel the operation and raise an error. "
"Returns the buffer if the read was successful or nil if end-of-stream reached. Will raise an "
"error if there are problems with the IO operation.") {
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
janet_arity(argc, 2, 4);
JanetStream *stream = janet_getabstract(argv, 0, &janet_stream_type);
janet_stream_flags(stream, JANET_STREAM_READABLE);
JanetBuffer *buffer = janet_optbuffer(argv, argc, 2, 10);
double to = janet_optnumber(argv, argc, 3, INFINITY);
Add :all option to ev/read. Brings ev/read more in line with file/read.
2020-12-30 02:37:59 +00:00
if (janet_keyeq(argv[1], "all")) {
if (to != INFINITY) janet_addtimeout(to);
Fix subprocess spawning on windows. Also fix (:read stream :all)
2021-01-11 17:10:23 +00:00
janet_ev_readchunk(stream, buffer, INT32_MAX);
Add :all option to ev/read. Brings ev/read more in line with file/read.
2020-12-30 02:37:59 +00:00
} else {
int32_t n = janet_getnat(argv, 1);
if (to != INFINITY) janet_addtimeout(to);
janet_ev_read(stream, buffer, n);
}
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
janet_await();
}
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
JANET_CORE_FN(janet_cfun_stream_chunk,
"(ev/chunk stream n &opt buffer timeout)",
"Same as ev/read, but will not return early if less than n bytes are available. If an end of "
"stream is reached, will also return early with the collected bytes.") {
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
janet_arity(argc, 2, 4);
JanetStream *stream = janet_getabstract(argv, 0, &janet_stream_type);
janet_stream_flags(stream, JANET_STREAM_READABLE);
int32_t n = janet_getnat(argv, 1);
JanetBuffer *buffer = janet_optbuffer(argv, argc, 2, 10);
double to = janet_optnumber(argv, argc, 3, INFINITY);
if (to != INFINITY) janet_addtimeout(to);
janet_ev_readchunk(stream, buffer, n);
janet_await();
}
Put source mapping info in stack traces.
2021-07-30 02:29:08 +00:00
JANET_CORE_FN(janet_cfun_stream_write,
"(ev/write stream data &opt timeout)",
"Write data to a stream, suspending the current fiber until the write "
"completes. Takes an optional timeout in seconds, after which will return nil. "
"Returns nil, or raises an error if the write failed.") {
Add ev/pipe and move stream code into ev.c Also adds a lot to the C API and changes things up.
2020-11-14 20:28:10 +00:00
janet_arity(argc, 2, 3);
JanetStream *stream = janet_getabstract(argv, 0, &janet_stream_type);
janet_stream_flags(stream, JANET_STREAM_WRITABLE);
double to = janet_optnumber(argv, argc, 2, INFINITY);
if (janet_checktype(argv[1], JANET_BUFFER)) {
if (to != INFINITY) janet_addtimeout(to);
janet_ev_write_buffer(stream, janet_getbuffer(argv, 1));
} else {
JanetByteView bytes = janet_getbytes(argv, 1);
if (to != INFINITY) janet_addtimeout(to);
janet_ev_write_string(stream, bytes.bytes);
}
janet_await();
}
Add some thread coordination primitives. Due to the nature of event loops, it is a bit difficult to integrate lock and other primitives such that they don't block fibers on the same thread.
2022-06-05 20:24:34 +00:00
static int mutexgc(void *p, size_t size) {
(void) size;
Remove pthread.h from janet.h Should make janet a bit easier to use. Also changes the header to not expose the size of native mutexes and rwlocks, except with janet_os_mutex_size and janet_os_rwlock_size.
2022-06-17 22:13:58 +00:00
janet_os_mutex_deinit(p);
Add some thread coordination primitives. Due to the nature of event loops, it is a bit difficult to integrate lock and other primitives such that they don't block fibers on the same thread.
2022-06-05 20:24:34 +00:00
return 0;
}
const JanetAbstractType janet_mutex_type = {
"core/lock",
mutexgc,
JANET_ATEND_GC
};
JANET_CORE_FN(janet_cfun_mutex,
Add stubs that are precursor to FFI. FFI may be best implemented as an external library (libffi has incompatible license to Janet) or as code that takes void * and wraps then into Janet C functions given a function signature. Either way, we need to some way to load symbols from arbitrary dynamic libraries.
2022-06-06 15:49:30 +00:00
"(ev/lock)",
"Create a new lock to coordinate threads.") {
Add some thread coordination primitives. Due to the nature of event loops, it is a bit difficult to integrate lock and other primitives such that they don't block fibers on the same thread.
2022-06-05 20:24:34 +00:00
janet_fixarity(argc, 0);
(void) argv;
Remove pthread.h from janet.h Should make janet a bit easier to use. Also changes the header to not expose the size of native mutexes and rwlocks, except with janet_os_mutex_size and janet_os_rwlock_size.
2022-06-17 22:13:58 +00:00
void *mutex = janet_abstract_threaded(&janet_mutex_type, janet_os_mutex_size());
janet_os_mutex_init(mutex);
Add some thread coordination primitives. Due to the nature of event loops, it is a bit difficult to integrate lock and other primitives such that they don't block fibers on the same thread.
2022-06-05 20:24:34 +00:00
return janet_wrap_abstract(mutex);
}
JANET_CORE_FN(janet_cfun_mutex_acquire,
Add stubs that are precursor to FFI. FFI may be best implemented as an external library (libffi has incompatible license to Janet) or as code that takes void * and wraps then into Janet C functions given a function signature. Either way, we need to some way to load symbols from arbitrary dynamic libraries.
2022-06-06 15:49:30 +00:00
"(ev/acquire-lock lock)",
"Acquire a lock such that this operating system thread is the only thread with access to this resource."
" This will block this entire thread until the lock becomes available, and will not yield to other fibers "
"on this system thread.") {
Add some thread coordination primitives. Due to the nature of event loops, it is a bit difficult to integrate lock and other primitives such that they don't block fibers on the same thread.
2022-06-05 20:24:34 +00:00
janet_fixarity(argc, 1);
Remove pthread.h from janet.h Should make janet a bit easier to use. Also changes the header to not expose the size of native mutexes and rwlocks, except with janet_os_mutex_size and janet_os_rwlock_size.
2022-06-17 22:13:58 +00:00
void *mutex = janet_getabstract(argv, 0, &janet_mutex_type);
janet_os_mutex_lock(mutex);
Add some thread coordination primitives. Due to the nature of event loops, it is a bit difficult to integrate lock and other primitives such that they don't block fibers on the same thread.
2022-06-05 20:24:34 +00:00
return argv[0];
}
JANET_CORE_FN(janet_cfun_mutex_release,
Add stubs that are precursor to FFI. FFI may be best implemented as an external library (libffi has incompatible license to Janet) or as code that takes void * and wraps then into Janet C functions given a function signature. Either way, we need to some way to load symbols from arbitrary dynamic libraries.
2022-06-06 15:49:30 +00:00
"(ev/release-lock lock)",
"Release a lock such that other threads may acquire it.") {
Add some thread coordination primitives. Due to the nature of event loops, it is a bit difficult to integrate lock and other primitives such that they don't block fibers on the same thread.
2022-06-05 20:24:34 +00:00
janet_fixarity(argc, 1);
Remove pthread.h from janet.h Should make janet a bit easier to use. Also changes the header to not expose the size of native mutexes and rwlocks, except with janet_os_mutex_size and janet_os_rwlock_size.
2022-06-17 22:13:58 +00:00
void *mutex = janet_getabstract(argv, 0, &janet_mutex_type);
janet_os_mutex_unlock(mutex);
Add some thread coordination primitives. Due to the nature of event loops, it is a bit difficult to integrate lock and other primitives such that they don't block fibers on the same thread.
2022-06-05 20:24:34 +00:00
return argv[0];
}
Add rwlocks.
2022-06-05 21:42:18 +00:00
static int rwlockgc(void *p, size_t size) {
(void) size;
Remove pthread.h from janet.h Should make janet a bit easier to use. Also changes the header to not expose the size of native mutexes and rwlocks, except with janet_os_mutex_size and janet_os_rwlock_size.
2022-06-17 22:13:58 +00:00
janet_os_rwlock_deinit(p);
Add rwlocks.
2022-06-05 21:42:18 +00:00
return 0;
}
const JanetAbstractType janet_rwlock_type = {
"core/rwlock",
rwlockgc,
JANET_ATEND_GC
};
JANET_CORE_FN(janet_cfun_rwlock,
Add stubs that are precursor to FFI. FFI may be best implemented as an external library (libffi has incompatible license to Janet) or as code that takes void * and wraps then into Janet C functions given a function signature. Either way, we need to some way to load symbols from arbitrary dynamic libraries.
2022-06-06 15:49:30 +00:00
"(ev/rwlock)",
"Create a new read-write lock to coordinate threads.") {
Add rwlocks.
2022-06-05 21:42:18 +00:00
janet_fixarity(argc, 0);
(void) argv;
Remove pthread.h from janet.h Should make janet a bit easier to use. Also changes the header to not expose the size of native mutexes and rwlocks, except with janet_os_mutex_size and janet_os_rwlock_size.
2022-06-17 22:13:58 +00:00
void *rwlock = janet_abstract_threaded(&janet_rwlock_type, janet_os_rwlock_size());
janet_os_rwlock_init(rwlock);
Add rwlocks.
2022-06-05 21:42:18 +00:00
return janet_wrap_abstract(rwlock);
}
JANET_CORE_FN(janet_cfun_rwlock_read_lock,
Add stubs that are precursor to FFI. FFI may be best implemented as an external library (libffi has incompatible license to Janet) or as code that takes void * and wraps then into Janet C functions given a function signature. Either way, we need to some way to load symbols from arbitrary dynamic libraries.
2022-06-06 15:49:30 +00:00
"(ev/acquire-rlock rwlock)",
"Acquire a read lock an a read-write lock.") {
Add rwlocks.
2022-06-05 21:42:18 +00:00
janet_fixarity(argc, 1);
Remove pthread.h from janet.h Should make janet a bit easier to use. Also changes the header to not expose the size of native mutexes and rwlocks, except with janet_os_mutex_size and janet_os_rwlock_size.
2022-06-17 22:13:58 +00:00
void *rwlock = janet_getabstract(argv, 0, &janet_rwlock_type);
janet_os_rwlock_rlock(rwlock);
Add rwlocks.
2022-06-05 21:42:18 +00:00
return argv[0];
}
JANET_CORE_FN(janet_cfun_rwlock_write_lock,
Add stubs that are precursor to FFI. FFI may be best implemented as an external library (libffi has incompatible license to Janet) or as code that takes void * and wraps then into Janet C functions given a function signature. Either way, we need to some way to load symbols from arbitrary dynamic libraries.
2022-06-06 15:49:30 +00:00
"(ev/acquire-wlock rwlock)",
"Acquire a write lock on a read-write lock.") {
Add rwlocks.
2022-06-05 21:42:18 +00:00
janet_fixarity(argc, 1);
Remove pthread.h from janet.h Should make janet a bit easier to use. Also changes the header to not expose the size of native mutexes and rwlocks, except with janet_os_mutex_size and janet_os_rwlock_size.
2022-06-17 22:13:58 +00:00
void *rwlock = janet_getabstract(argv, 0, &janet_rwlock_type);
janet_os_rwlock_wlock(rwlock);
Add rwlocks.
2022-06-05 21:42:18 +00:00
return argv[0];
}
JANET_CORE_FN(janet_cfun_rwlock_read_release,
Add stubs that are precursor to FFI. FFI may be best implemented as an external library (libffi has incompatible license to Janet) or as code that takes void * and wraps then into Janet C functions given a function signature. Either way, we need to some way to load symbols from arbitrary dynamic libraries.
2022-06-06 15:49:30 +00:00
"(ev/release-rlock rwlock)",
"Release a read lock on a read-write lock") {
Add rwlocks.
2022-06-05 21:42:18 +00:00
janet_fixarity(argc, 1);
Remove pthread.h from janet.h Should make janet a bit easier to use. Also changes the header to not expose the size of native mutexes and rwlocks, except with janet_os_mutex_size and janet_os_rwlock_size.
2022-06-17 22:13:58 +00:00
void *rwlock = janet_getabstract(argv, 0, &janet_rwlock_type);
janet_os_rwlock_runlock(rwlock);
Add rwlocks.
2022-06-05 21:42:18 +00:00
return argv[0];
}
JANET_CORE_FN(janet_cfun_rwlock_write_release,
Add stubs that are precursor to FFI. FFI may be best implemented as an external library (libffi has incompatible license to Janet) or as code that takes void * and wraps then into Janet C functions given a function signature. Either way, we need to some way to load symbols from arbitrary dynamic libraries.
2022-06-06 15:49:30 +00:00
"(ev/release-wlock rwlock)",
"Release a write lock on a read-write lock") {
Add rwlocks.
2022-06-05 21:42:18 +00:00
janet_fixarity(argc, 1);
Remove pthread.h from janet.h Should make janet a bit easier to use. Also changes the header to not expose the size of native mutexes and rwlocks, except with janet_os_mutex_size and janet_os_rwlock_size.
2022-06-17 22:13:58 +00:00
void *rwlock = janet_getabstract(argv, 0, &janet_rwlock_type);
janet_os_rwlock_wunlock(rwlock);
Add rwlocks.
2022-06-05 21:42:18 +00:00
return argv[0];
}
Add `ev/all-tasks` to get running and pending root fibers.
2023-04-01 23:57:13 +00:00
JANET_CORE_FN(janet_cfun_ev_all_tasks,
"(ev/all-tasks)",
"Get an array of all active fibers that are being used by the scheduler.") {
janet_fixarity(argc, 0);
(void) argv;
JanetArray *array = janet_array(janet_vm.active_tasks.count);
for (int32_t i = 0; i < janet_vm.active_tasks.capacity; i++) {
if (!janet_checktype(janet_vm.active_tasks.data[i].key, JANET_NIL)) {
janet_array_push(array, janet_vm.active_tasks.data[i].key);
}
}
return janet_wrap_array(array);
}
Start with ev module.
2020-05-28 15:39:40 +00:00
void janet_lib_ev(JanetTable *env) {
Update ev.c to use the new binding style that provides source information
2021-07-26 05:25:38 +00:00
JanetRegExt ev_cfuns_ext[] = {
JANET_CORE_REG("ev/give", cfun_channel_push),
JANET_CORE_REG("ev/take", cfun_channel_pop),
JANET_CORE_REG("ev/full", cfun_channel_full),
JANET_CORE_REG("ev/capacity", cfun_channel_capacity),
JANET_CORE_REG("ev/count", cfun_channel_count),
JANET_CORE_REG("ev/select", cfun_channel_choice),
Add rselect bindings
2021-07-27 06:48:28 +00:00
JANET_CORE_REG("ev/rselect", cfun_channel_rchoice),
Update ev.c to use the new binding style that provides source information
2021-07-26 05:25:38 +00:00
JANET_CORE_REG("ev/chan", cfun_channel_new),
Initial working draft of threaded channels. Introduces close semtantics to channels as well, but otherwise threaded channels behave much like non-threaded channels. They have different marshalling behavior though, and can only send values over by packing and unpacking them. For now, this means only primitive values although this will be expanded. Also missing some implementation for closing threaded channels, and a whole lot of testing. Achtung!, Caveat emptor, here be dragons and bugs.
2021-08-15 16:48:13 +00:00
JANET_CORE_REG("ev/thread-chan", cfun_channel_new_threaded),
JANET_CORE_REG("ev/chan-close", cfun_channel_close),
Update ev.c to use the new binding style that provides source information
2021-07-26 05:25:38 +00:00
JANET_CORE_REG("ev/go", cfun_ev_go),
JANET_CORE_REG("ev/thread", cfun_ev_thread),
JANET_CORE_REG("ev/give-supervisor", cfun_ev_give_supervisor),
JANET_CORE_REG("ev/sleep", cfun_ev_sleep),
JANET_CORE_REG("ev/deadline", cfun_ev_deadline),
JANET_CORE_REG("ev/cancel", cfun_ev_cancel),
JANET_CORE_REG("ev/close", janet_cfun_stream_close),
JANET_CORE_REG("ev/read", janet_cfun_stream_read),
JANET_CORE_REG("ev/chunk", janet_cfun_stream_chunk),
JANET_CORE_REG("ev/write", janet_cfun_stream_write),
Add some thread coordination primitives. Due to the nature of event loops, it is a bit difficult to integrate lock and other primitives such that they don't block fibers on the same thread.
2022-06-05 20:24:34 +00:00
JANET_CORE_REG("ev/lock", janet_cfun_mutex),
JANET_CORE_REG("ev/acquire-lock", janet_cfun_mutex_acquire),
JANET_CORE_REG("ev/release-lock", janet_cfun_mutex_release),
Add rwlocks.
2022-06-05 21:42:18 +00:00
JANET_CORE_REG("ev/rwlock", janet_cfun_rwlock),
JANET_CORE_REG("ev/acquire-rlock", janet_cfun_rwlock_read_lock),
JANET_CORE_REG("ev/acquire-wlock", janet_cfun_rwlock_write_lock),
JANET_CORE_REG("ev/release-rlock", janet_cfun_rwlock_read_release),
JANET_CORE_REG("ev/release-wlock", janet_cfun_rwlock_write_release),
Add `ev/all-tasks` to get running and pending root fibers.
2023-04-01 23:57:13 +00:00
JANET_CORE_REG("ev/all-tasks", janet_cfun_ev_all_tasks),
Update ev.c to use the new binding style that provides source information
2021-07-26 05:25:38 +00:00
JANET_REG_END
};
janet_core_cfuns_ext(env, NULL, ev_cfuns_ext);
Update for sending streams to new threads.
2021-01-24 22:48:46 +00:00
janet_register_abstract_type(&janet_stream_type);
Fix some issues and improve channel closing. Still not fully working, seems to be deadlock/channel issue when sending events between threads.
2021-08-15 18:14:33 +00:00
janet_register_abstract_type(&janet_channel_type);
Add some thread coordination primitives. Due to the nature of event loops, it is a bit difficult to integrate lock and other primitives such that they don't block fibers on the same thread.
2022-06-05 20:24:34 +00:00
janet_register_abstract_type(&janet_mutex_type);
Add rwlocks.
2022-06-05 21:42:18 +00:00
janet_register_abstract_type(&janet_rwlock_type);
Start with ev module.
2020-05-28 15:39:40 +00:00
}
#endif
Reference in New Issue Copy Permalink