1
0
mirror of https://github.com/janet-lang/janet synced 2024-06-16 10:19:55 +00:00
Commit Graph

118 Commits

Author SHA1 Message Date
Max Schillinger
c29ab22e6d Fix documentation of peg/replace 2024-02-23 12:46:45 +01:00
Ian Henry
61f38fab37
add a new (split) PEG special
This works similarly to string/split, but the separator is a PEG.
2024-01-05 22:02:52 -08:00
Calvin Rose
e8ed961572
Merge pull request #1344 from ianthehenry/peg-sub-special
Add a new (sub) PEG special
2023-12-31 18:40:47 -06:00
Ian Henry
ea75086300
add a new (sub) PEG special
(sub) will first match one pattern, then match another pattern against the
text that the first pattern advanced over.
2023-12-28 22:15:54 -08:00
Josef Pospíšil
938f5a689e Fix arity typo in peg 2023-12-07 14:08:03 +01:00
Calvin Rose
2fde34b519 Remove extra function call that cannot ever trigger. 2023-06-18 09:41:53 -05:00
Chloe Kudryavtsev
e54ea7a1d8 fixup! peg: add support for "true" and "false" primitives to always/never match 2023-06-11 12:38:40 +02:00
Chloe Kudryavtsev
f9ab91511d peg: add support for "true" and "false" primitives to always/never match
The use cases involve user-expandable grammars.
For example, consider the IRC nickname specification.
> They SHOULD NOT contain any dot character ('.', 0x2E).
> Servers MAY have additional implementation-specific nickname restrictions.

To implement this, we can do something along these lines:
```janet
(def nickname @{:main '(some :allowed)
                :allowed (! (+ :forbidden/dot :forbidden/user))
	        # for lax mode, (put nickname :forbidden/dot false)
	        :forbidden/dot "."
		# to add your own requirements
		# (put nickname :forbidden/user 'something)
		:forbidden/user false})
```

Additionally, it's common in parsing theory to allow matches of the
empty string (epsilon). `true` essentially allows for this.

Note that this does not strictly add new functionality, you could
emulate this previously using `0` and `(! 0)` respectively, but this
should be faster and more intuitive.
The speed improvement primarily comes from `(! 0)` which is now a single
step.
2023-06-11 10:44:39 +02:00
Ico Doornekamp
e9f2d1aca7 changed some error messages 'x|y' -> 'x or y' 2023-05-23 06:58:52 +02:00
Ian Henry
9dc7e8ed3a
peg replacement functions have access to captures
When peg/replace or peg/replace-all are given a function to serve as the text
replacement, any captures produced by the PEG are passed as additional
arguments to that function.
2023-04-23 09:15:46 -07:00
Ian Henry
485099fd6e
string and peg replacement functions can now take functions
Functions will be invoked with the matched text, and their result will be
coerced to a string and used as the new replacement text.

This also allows passing non-function, non-byteviewable values, which will be
converted into strings during replacement (only once, and only if at least
one match is found).
2023-04-23 08:36:17 -07:00
Ian Henry
b442b21d3f
fix a segfault in the (number) special
This was an error about the base argument that referenced the tag argument,
which might not exist.
2023-02-20 17:07:42 -08:00
Calvin Rose
a0f40042cb Update copyright year. 2023-01-07 15:03:35 -06:00
Calvin Rose
914a4360e7 Indicate version bump since header file changed. 2022-11-05 16:38:52 -05:00
Calvin Rose
dfc0aefd87
Merge pull request #1028 from autumnull/master
Made peg 'not' and 'if-not' drop their captures on success
2022-09-13 15:20:10 -05:00
Calvin Rose
8da7bb6b68 Fix peg/replace-all and family - Fix #1027 2022-09-12 18:58:48 -05:00
Autumn!
9341081a4d Made peg 'not' and 'if-not' drop their captures on success 2022-09-12 23:07:56 +01:00
Calvin Rose
ff26e3a8ba Remove end of string check that is now redudant.
The addition of some code to avoid valgrind warnings made this code
redundant.
2022-09-05 20:13:15 -05:00
Calvin Rose
14657a762c Fix peg RULE_SET op code when at tail of string in some cases. 2022-09-05 14:11:03 -05:00
Calvin Rose
9fa945ad93 Don't include captures of last match from to combinator. 2022-04-29 19:21:10 -05:00
Calvin Rose
a895219d2f Fix #971 - remove to rule optimization
For to and thru, we need to restore eveytime through the loop since rules need
run with the right captures on the stack, especially if they have any
sort of backrefs.
2022-04-29 19:15:56 -05:00
Calvin Rose
3715d7a184 Auto update copyright date. 2022-03-21 18:22:59 -05:00
Calvin Rose
eb4595158d Allow compiling tables as peg grammars. 2021-10-23 09:59:36 -05:00
Calvin Rose
9985f787eb Add custom base option to number peg combinator.
Allows parsing custom bases without needed Janet specific prefixes.
2021-09-21 18:02:42 -05:00
Calvin Rose
cccbdc164c Add (number combinator to peg).
This allows using Janet's number parser without creating
intermediate strings.
2021-09-19 13:02:16 -05:00
Calvin Rose
eb84200f28 Fix linux issues with epoll on normal files.
We use the selfpipe trick if epoll fails with EPERM when trying to
register a file descriptor.
2021-07-25 21:47:52 -05:00
bakpakin
7e8154e648 Update peg.c with new style core function declarations. 2021-07-25 14:54:25 -05:00
Calvin Rose
bbae43f259 Update copyright dates. 2021-05-31 13:46:02 -05:00
Calvin Rose
7ff204ec44 Work on system for adding compiler warnings.
This is the beginning of a system for compiler warnings. This includes
linting, deprecation notices, and other compiler warnings that are best
detected by the `compile` function and don't require the partial
evalutaion of the flychecker.
2021-05-28 15:12:05 -05:00
Calvin Rose
2db7945d6f Fix peg bug when there is no default grammar set.
This could result in a segfault when we attempt to
read from a NULL pointer.
2021-05-20 21:57:22 -05:00
Andrew Chambers
f4c9064b79 Add config support for custom allocators. 2021-03-23 23:00:48 +13:00
Calvin Rose
96513665d6 Address #641 - add undef combinator.
The (undef rule :tag) combinator lets a user "scope" tagged captures.
After the rule has matched, all captures with tag :tag can no longer be
refered to by their tag. However, such captures from outside
rule are kept as is. If no tag is given, all tagged captures from rule
are unreferenced. Note that this doesn't `drop` the captures, merely
removes their association with the tag. This means subsequent calls to
`backref` and `backmatch` will no longer "see" these tagged captures.
2021-02-26 17:25:09 -06:00
Calvin Rose
742469a8bc Address #640.
Allow for a zero length match at the end of a string when using the
to or thru combinators.
2021-02-19 16:10:03 -06:00
sogaiu
60e2992158 Tweak spec_readint 2021-02-10 08:33:46 +09:00
Calvin Rose
85155bb2b4 Reference #478 Update peg/compile to use dyn for default grammar. 2021-01-23 13:54:02 -06:00
Calvin Rose
61cca10cf6 Allow iterating through the properties of core abstract types. 2021-01-11 23:14:07 -06:00
Calvin Rose
adc41e31f4 Address #547 - don't drop references.
Keep a separate stack for tagged references. May cause pegs to
use more memory but makes the backref and backmatch features much more
powerful.

Also disables the second stack if backref and backmatch are not used in the peg.
2021-01-05 20:27:15 -06:00
Calvin Rose
a0964d44d5 Fix some valgrind errors.
A null pointer dereference and a memory leak with the line/col mapping.
2020-11-27 12:21:23 -06:00
Calvin Rose
d76f671d37 Update changelog and make arg to peg's error optional. 2020-11-26 18:57:24 -06:00
Calvin Rose
776ce586bc Add line and column combinators to peg.
These capture the line and column number of the current position
in the matched text. This is useful for error reporting as well
as indentation checking.

This works by lazily creating an index on first use that stores all
newline character indices in order. We can then do a binary search on
this to get both line number and column number in log(n) time.
This is good enough for most use cases and doesn't slow down the common case at all
- these will not be commonly used patterns in a hot loop so it is not worth to try and
optimize this at all. Constant time look up should be possible but at
the cost of complicating code and slowing down all matching to check for
new lines.
2020-11-26 18:32:56 -06:00
Calvin Rose
45feb55483 Add integer parsing to pegs. 2020-09-27 12:19:00 -05:00
Calvin Rose
6ea530cc48 Address compilation warnings and errors. 2020-07-03 12:25:24 -05:00
Calvin Rose
e548e1f6e0 Add peg/replace and peg/replace-all 2020-07-01 21:29:24 -05:00
Calvin Rose
17a131ac21 Add peg/find and peg/find-all.
These peg functions should make pegs a bit easier to use
and more efficient in some common cases.
2020-06-29 19:13:06 -05:00
Calvin Rose
86e12369b6 Add alias for PEG repeat.
A tuple where the first element is an integer is
a shortand for this.
2020-06-11 11:23:43 -05:00
Calvin Rose
6d096551f0 Add Peg combinators 'to' and 'thru'.
Inpsired by the REBOL operators of the same name, these
combinators match bytes up to or inculding a given pattern.
(to patt) is (almost) equalivalent to (any (if-not patt 1)), and
(thru patt) is equivalent to (* (to patt) patt). The one difference
is that if the end of the input is reached and patt is not
matched, the entire pattern does not match.
2020-06-10 21:18:50 -05:00
Calvin Rose
677737d345 Fixes #412 Lookahead does not move cursor. 2020-05-21 01:22:08 -05:00
Calvin Rose
235605bfa4 1.9.0 Release.
Fix up some documentation as well.
2020-05-10 16:45:33 -05:00
Calvin Rose
8b5bcaee3c Add lenprefix combinator to pegs.
This lets peg match n repeitions of a pattern, where
n is supplied from other parsed input and is not a constant.
2020-05-02 10:39:35 -05:00
Calvin Rose
6a3a983f43 Expose abstract type definitions in janet.h
This makes certain operations easier, and allows
more access to built in APIs.
2020-03-14 10:12:47 -05:00