Some minor changes to the filter docs (#9709)

* Some minor changes to the filter docs * Fix wording in Intersection Filter Run Prefix documentation
2026-06-14 17:28:51 +00:00 · 2026-03-24 09:46:03 +01:00
parent 1d915389d9
commit c4991a54a0
8 changed files with 43 additions and 10 deletions
@@ -19,3 +19,5 @@ A <<.def "filter expression">> is the outermost level of the [[filter syntax|Fil
 <<.tip """If the diagram has a single start and end line, as shown above, it means there is more info in the linked level above. The breadcrumbs can be used to navigate""">>

 <<.tip """If the diagram has no start and no end, as used in lower levels, it means that higher level syntax elements have been removed, to increase readability and simplicity. The breadcrumbs can be used to navigate""">>
+
+<<.note """Filter expressions have a maximum recursion depth of 300. Filters that recursively invoke themselves (e.g. via the <<.olink subfilter>> or <<.olink filter>> operators) beyond this limit will return the error message `/**-- Excessive filter recursion --**/`. This protects against infinite loops.""">>
@@ -12,6 +12,8 @@ type: text/vnd.tiddlywiki
  "{" [: <-"indirect"-> /"anything but }"/] "}"
  |
  "<" [: <-"variable"-> /"anything but >"/] ">"
+  |
+  "(" [: <-"multi-valued variable"-> /"anything but )"/  ] ")" /"v5.4.0"/
 )
 """/>

@@ -20,6 +22,7 @@ The parameter to a [[filter operator|Filter Operators]] can be:
 ;<<.def hard>>
 : `[like this]`
 : The parameter is the exact text that appears between the square brackets.
+
 ;<<.def soft>>
 : <<.def indirect>>
 :: `{like this}`
@@ -28,6 +31,9 @@ The parameter to a [[filter operator|Filter Operators]] can be:
 :: `<like this>`
 :: The parameter is the current value of the [[variable|Variables]] whose name appears between the angle brackets. Macro parameters are <<.em not>> supported up to v5.2.0
 ::<<.from-version "5.2.0">> Literal macro parameters are supported. For example: `[<now [UTC]YYYY0MM0DD0hh0mm0ssXXX>]`.
+: <<.def "multi-valued variable">>
+:: `(like this)`
+:: <<.from-version "5.4.0">> The parameter expands to the complete list of values stored in the [[multi-valued variable|Multi-Valued Variables]] whose name appears between the round brackets. When accessed this way, the filter operator receives all values, not just the first. See [[Multi-Valued Variables]] for details.

 <<.note """Every [[filter Operator]] must be followed by a parameter expression. In the case of [[Operators without parameters]], that expression is empty, as with the filter Operator <<.olink links>> in `[<currentTiddler>links[]]`.""">>

@@ -35,4 +41,8 @@ The parameter to a [[filter operator|Filter Operators]] can be:

 <<.from-version "5.1.23">>  [[Filter Step]]s support multiple parameters which are separated by a `,` character.

-For example: `[param1],[param2]` or `<param1>,{param2}`
+For example: `[param1],[param2]` or `<param1>,{param2}` or `[param1],(param2)`
+
+---
+
+<<.warning """The `/regexp/(flags)` operand syntax is deprecated and will log a warning to the browser console. Use the <<.olink regexp>> operator instead.""">>
@@ -18,17 +18,17 @@ In programming terms, it is akin to a function call to which the step's input is
 { [[parameter|"Filter Parameter"]] + "," }
 """/>

-The step's <<.def operator>> is drawn from a list of predefined keywoards which are known as [[filter operators|Filter Operators]].
+The step's <<.def operator>> is drawn from a list of predefined keywords which are known as [[filter operators|Filter Operators]].

 Many steps require an explicit <<.def parameter>>, that further defines what the step is to do. 

-The <<.def suffix>> is additional text, often the name of a [[field|TiddlerFields]], that extends the meaning of certain operators.
+The <<.def suffix>> is additional text, often the name of a [[field|TiddlerFields]], that extends the meaning of certain operators. Suffixes are separated by `:` characters, and each suffix group can contain multiple comma-separated values. For example, in `compare:number:gteq`, the first suffix is `number` and the second is `gteq`. In `:sort:string:reverse,casesensitive`, the first suffix is `string` and the second suffix group contains both `reverse` and `casesensitive`.

-If a step's <<.def operator>> and <<.def suffix>> are //omitted// altogether, it defaults to the [[title|title Operator]] operator.
+If a step's <<.def operator>> and <<.def suffix>> are //omitted// altogether, it defaults to the [[title|title Operator]] operator. If a suffix is present but the operator name before the colon is empty (e.g. `[:fieldname[value]]`), the operator defaults to <<.olink field>>.

 <<.from-version "5.1.23">> Some steps accept multiple <<.def parameter>>s which are separated by a `,` character.

-Any unrecognised operator is treated as if it was the suffix to the <<.olink field>> operator. 
+Any unrecognised operator is treated as if it was the suffix to the <<.olink field>> operator. <<.from-version "5.3.0">> However, unrecognised operators whose name contains a `.` (dot) character are treated as user-defined filter operators, resolved by looking up a variable or function with that name.

 Filter operators can be extended by plugins.

@@ -43,3 +43,5 @@ For the difference between `+` and `:intersection`, see [[Intersection Filter Ru
 !! For Developers

 To create a new filter run prefix, create a [[Javascript module|Modules]] with a [[module-type|ModuleType]] of `filterrunprefix`.
+
+Compiled filter expressions are cached for performance. The cache holds up to 2000 entries and is reset in its entirety when this limit is exceeded.
@@ -15,6 +15,15 @@ type: text/vnd.tiddlywiki
 [[run|"Filter Run"]]
 """/>

-The filter output from previous runs is set aside. The `:intersection` filter run is started with all tiddler titles as input. Once this latest filter run has completed, the latest output is compared to the set-aside output. A new output is produced that contains only titles that appeared in both the set-aside output and the latest output.
+The filter output from previous runs is set aside. The `:intersection` filter run is started with all tiddler titles as input. Once this filter run has completed, the output is compared to the set-aside output. A new output is produced that contains only titles that appeared in both the set-aside output and the latest output. The order of titles in the output is preserved from the accumulated results.
+
+!! Difference from `:and` / `+`
+
+The key difference between `:intersection` and `:and` (or `+`) is what they receive as ''input'':
+
+* `:and` / `+` feeds the ''accumulated results so far'' as input to the filter run. Operators in the run can only see and work with titles already in the results.
+* `:intersection` feeds ''all tiddler titles'' as input to the filter run (just like an unprefixed run), then keeps only titles that also appear in the accumulated results.
+
+This means `:intersection` can match titles using operators that need to start from the full tiddler pool. See [[Interchangeable Filter Run Prefixes]] for more details.

 [[Intersection Filter Run Prefix (Examples)]]
@@ -34,4 +34,14 @@ A named filter run prefix can precede any [[run|Filter Run]] of a [[filter expre

 <<.tip """Within the filter runs prefixed with `:reduce`, `:sort`, `:map` and `:filter`, the <<.var currentTiddler>> variable is set to the title of the tiddler being processed.<br>The value of currentTiddler outside the subfilter is available in the variable <<.var "..currentTiddler">> <<.from-version "5.2.0">>""" >>

+!! Suffixes
+
+Named filter run prefixes can accept suffixes separated by `:` characters, each optionally containing comma-separated values. The general syntax is `:prefixname:suffix1:suffix2,...`. Currently the following prefixes accept suffixes:
+
+|!Prefix |!Suffixes |!Example |
+|`:map` |`flat` — return all results instead of only the first per item |`:map:flat[...]` |
+|`:sort` |type (''string'', ''alphanumeric'', ''number'', ''integer'', ''version'', ''date'') and flags (''reverse'', ''casesensitive'', ''caseinsensitive'') |`:sort:number:reverse[...]` |
+
+All other named prefixes (`:all`, `:and`, `:cascade`, `:else`, `:except`, `:filter`, `:intersection`, `:let`, `:or`, `:reduce`, `:then`) do not currently accept suffixes.
+
 Also see: [[Interchangeable Filter Run Prefixes]]
@@ -21,10 +21,10 @@ If a run has:

 * the prefix `-`, output titles are <<.em removed>> from the filter's output (if such tiddlers exist)

-* the prefix `~`, if the filter output so far is an empty list then the output titles of the run are [[dominantly appended|Dominant Append]] to the filter's output. If the filter output so far is not an empty list then the run is ignored. <<.from-version "5.1.18">>
+* the prefix `~`, <<.from-version "5.1.18">> if the filter output so far is an empty list then the output titles of the run are [[dominantly appended|Dominant Append]] to the filter's output. If the filter output so far is not an empty list then the run is ignored.

-* the prefix `=`, output titles are appended to the filter's output without de-duplication. <<.from-version "5.1.20">>
+* the prefix `=`, <<.from-version "5.1.20">> output titles are appended to the filter's output without de-duplication.

-* the prefix `=>`, the input is assigned to the variable named with the output title. <<.from-version "5.4.0">>
+* the prefix `=>`, <<.from-version "5.4.0">> the accumulated results from all previous runs are assigned as a [[multi-valued variable|Multi-Valued Variables]] whose name is the first result of this run. The accumulated results are then cleared.

 {{Interchangeable Filter Run Prefixes}}
@@ -1,7 +1,7 @@
 created: 20210618133745003
 from-version: 5.3.0
 modified: 20230710074225410
-rp-input: <<.olink all>> tiddler titles
+rp-input: <<.olink all>> tiddler titles (only evaluated when the output from previous runs is non-empty)
 rp-output: the output of this filter run replaces the output of previous runs unless it is an empty list (see below)
 rp-purpose: replace any input to this filter run with its output, only evaluating the run when there is any input
 search: