Docs for filesystem adaptor update (#5169)

* rebased to master

* cleanup typos

* typo

editions/tw5.com/tiddlers/nodejs/Customising Tiddler File Naming.tid

@@ -1,25 +1,51 @@
 created: 20160424181300000
-modified: 20160424181300000
+modified: 20201201000000000
 tags: [[TiddlyWiki on Node.js]]
 title: Customising Tiddler File Naming
 type: text/vnd.tiddlywiki
 
-By default, a [[TiddlyWiki on Node.js]] instance using a [[wiki folder|TiddlyWikiFolders]] will create new tiddler files by using the sanitised and disambiguated title as filename.
+By default, a [[TiddlyWiki on Node.js]] instance using a [[wiki folder|TiddlyWikiFolders]] will create new tiddler files by using the sanitised and disambiguated title as filename and write them to the wiki folder's `tiddlers/` directory. The default file extension of `.tid` is used for tiddlers that are missing the `type` field, or for tiddlers of type "text/vnd.tiddlywiki". Tidders of other types are saved according to their IMIE types (defined at boot startup). Both the logical path (directory and file name) and the file extension can be customized independently by creating optional tiddlers: [[$:/config/FileSystemPaths]] and [[$:/config/FileSystemExtensions]].
 
-This can be customised by creating a tiddler [[$:/config/FileSystemPaths]] containing one or more [[filter expressions|Filter Syntax]], each on a line of its own. Newly created tiddlers are matched to each filter in turn, and the first output of the first filter to produce any output is taken as a logical path to be used for the tiddler file. Logical paths don't include the `.tid` extension, and they can use `/` or `\` as directory separator (when generating the physical path, this is replaced by the correct separator for the platform ~TiddlyWiki is running on). If none of the filters matches, the logical path is simply the title with all occurences of `/` replaced by `_` (for backwards compatibility).
+! File System Paths
 
-In both cases, the characters `<>:"\|?*^` are replaced by `_` in order to guarantee that the resulting path is legal on all supported platforms.
+The logical path can be customised by creating a tiddler [[$:/config/FileSystemPaths]] containing one or more [[filter expressions|Filter Syntax]], each on a line of its own. Newly created tiddlers are matched to each filter in turn, and the first output of the first filter to produce any output is taken as a logical path to be used for the tiddler file. Tiddlers are also tested against the [[$:/config/FileSystemPaths]] on every save to disk, and if the logical path has changed a new file is created and the old file deleted.
+
+Tiddlers are limited to being written to the [[wiki folder|TiddlyWikiFolders]]. Any error saving a tiddler to disk, with a logical path that does not start with the wiki folder's path the most common error, causes the filepath to be encoded via Javascript's `encodeURIComponent()` method and the tiddler is saved as this file in the wiki folder's `tiddlers/` directory.
+
+Logical paths do not include the file-on-disk's extension (see below), and they can use `/` or `\` as directory separator (when generating the physical path, this is replaced by the correct separator for the platform ~TiddlyWiki is running on). If none of the filters matches, the logical path is simply the title with all occurences of `/` replaced by `_` (for backwards compatibility).
+
+In both cases, the characters `<>~:"\|?*^` are replaced by `_` in order to guarantee that the resulting path is legal on all supported platforms.
 
 !! Example
 
 ```
-[is[system]removeprefix[$:/]addprefix[_system/]]
+[is[system]!has[draft.of]removeprefix[$:/]addprefix[_system/]]
+[is[draft]search-replace:g:regexp[/|\\],[_]addprefix[drafts/]]
 [tag[task]addprefix[mytasks/]]
-[!has[draft.of]]
+[tag[externalnote]addprefix[../externalnotes/]]
+[addprefix[wiki/]]
 ```
 
-This will store newly created system tiddlers in `tiddlers/_system` (after stripping the `$:/` prefix), tiddlers tagged [[task]] in a subdirectory `tiddlers/mytasks`, and also create subdirectory structures for all other non-draft tiddlers.
+This will store newly created system tiddlers that are not drafts of other tiddlers in `tiddlers/_system` (after stripping the `$:/` prefix). Next, all drafts have the path seperator characters in their titles replaced by "_" and are stored in `tiddlers/drafts/`. Then tiddlers tagged [[task]] are stored in a subdirectory `tiddlers/mytasks/`. Next, all tiddlers tagged "externalnote" will be written to `<wikiFolder>/tiddlers/../externalnotes/`, which resolves to `<wikiFolder>/externalnotes/` and places these tiddlers outside of the tiddler folder. Use [[tiddlywiki.files Files|tiddlywiki.files_Files]] to reimport these tiddlers when the wiki server is restarted. Finally, all tidders will match the final `[addprefix[wiki/]]` storing these in `<wikiFolder>/tiddlers/wiki/`.
 
-Thus, $:/config/FileSystemPaths itself will end up in `tiddlers/_system/config/FileSystemPaths.tid` or `tiddlers\_system\config\FileSystemPaths.tid`, depending on the platform.
+Because there was a filter match, any `/` or `\` in the tiddler title is mapped to a path separator. With the above filters, the non-system, non-draft tiddler `some/thing/entirely/new` (with no tags) will be saved to `<wikiFolder>/tiddlers/wiki/some/thing/entirely/new.tid` (ie, the file `new.tid` in a directory called `entirely/`). Thus, $:/config/FileSystemPaths itself will end up in `tiddlers/_system/config/FileSystemPaths.tid` or `tiddlers\_system\config\FileSystemPaths.tid`, depending on the platform.
 
-The final `[!has[draft.of]]` will match all remaining non-draft tiddlers. Because there was a match, any `/` or `\` in the tiddler title is mapped to a path separator. Thus, `some/thing/entirely/new` will be saved to `tiddlers/some/thing/entirely/new.tid` (ie, the file `new.tid` in a directory called `entirely`).
+! File System Extensions
+
+Normally, the file system extension of a tiddler on disk is determined by the existance of bad fields (multi-line fields other than the text field, fields that can be trimmed of spaces from the fron or back, etc), in which case the single-file ".json" tiddler-file format is used. If the tiddler does not have bad fields, then the `type` field is referenced to find a matching file-type. Tiddlywiki's boot engine defines a set of these tiddler-type to file-type relationships in the [[$:/boot/boot.js]] tiddler. Search for `// Add file extension information` to find the section of code that defines these relationships.
+
+The file extension of individual tidders can be customised by creating a tiddler [[$:/config/FileSystemExtensions]] containing one or more [[filter expressions|Filter Syntax]], each on a line of its own. Newly created tiddlers are matched to each filter in turn, and the first output of the first filter to produce any output is taken as the file extension to be used for the tiddler file. Extensions should always start with a leading dot (see example). Tiddlers are also tested against the $:/config/FileSystemExtensions on every save to disk, and if the extension has changed a new file is created and the old file deleted. If no filter matches, the default extension is used.
+
+Two special cases should be noted: Result of ".tid" will force the tiddler to be written to disk as a single-file text tiddler. A result of ".json" will force the tiddler to be written to disk as a single-file tiddler in json-format (a single tiddler fields object in an array), NOT as a tiddler of type "application/json". All other recognized file-types will be saved using their defined extention along with an acompanying *.meta file of the same name which describes all fields but the "text" field.
+
+!! Example
+
+```
+[tag[.txt]then[.txt]]
+[tag[.json]then[.json]]
+[tag[.tid]then[.tid]]
+```
+
+This will cause all tidders that have the tag ".txt" in their tags field to be saved at the filepath determined by the File System Paths filters, but with their text field saved as a *.txt file, and all other fields saved as a *.txt.meta file.
+
+Next, all tiddlers that have the ".json" tag are saved as *.json single-file tiddlers. Finally, all tiddlers that have tag ".tid" are saved as single-file text tiddlers. If a tiddler matches none of the filters, the default extension determined by the tiddler's `type` field would be used.

editions/tw5.com/tiddlers/nodejs/tiddlywiki.files_Files.tid

@@ -1,5 +1,5 @@
 created: 20161015114118243
-modified: 20161015170604353
+modified: 20201201000000000
 tags: TiddlyWikiFolders
 title: tiddlywiki.files Files
 type: text/vnd.tiddlywiki
@@ -51,7 +51,7 @@ Directory specifications in the `directories` array may take the following forms
 ** ''path'' - (required) the absolute or relative path to the directory containing the tiddler files (relative paths are interpreted relative to the path of the `tiddlywiki.files` file). Note that the directory is not recursively searched; sub-directories are ignored
 ** ''filesRegExp'' - (optional) a [[regular expression|https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Regular_Expressions]] that matches the filenames of the files that should be processed within the directory
 ** ''isTiddlerFile'' - (required) if `true`, the file will be treated as a [[tiddler file|TiddlerFiles]] and deserialised to extract the tiddlers. Otherwise, the raw content of the file is assigned to the `text` field without any parsing
-** ''isEditableFile'' - (optional) if `true`, changes to the tiddler be saved back to the original file. <<.from-version "5.1.23">>
+** ''isEditableFile'' - (optional) if `true`, changes to the tiddler be saved back to the original file. The ''path'' of the current directory being read must be within the wiki's base directory, and a $:/config/FileSystemPath filter is required to add the correct logical path to the tiddler's title (see second **Example**). <<.from-version "5.1.23">>
 ** ''fields'' - (required) an object containing values that override or customise the fields provided in the tiddler file (see above)
 
 Fields can be overridden for particular files by creating a file with the same name plus the suffix `.meta` -- see TiddlerFiles.
@@ -64,7 +64,7 @@ There are also several examples of `tiddlywiki.files` files in the main [[Tiddly
 
 !! Importing a folder of PDFs
 
-This example retrieves all the files with the extension `.pdf` from a folder specified by a relative path. Each tiddler is set up for LazyLoading with the following fields:
+This example retrieves all the files with the extension `.pdf` from a folder specified by a relative path. This path starts with "../../../" indicating 3 directory levels above the folder holdng this confog fole. Each tiddler is set up for LazyLoading with the following fields:
 
 * ''title'' - set to the URI decoded base filename of the PDF file. [[URI decoding|https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/decodeURIComponent]] allows characters like "/" to be included in titles by URI encoding them as "%2F"
 * ''created'' - set to the creation date/time of the PDF file
@@ -94,3 +94,39 @@ This example retrieves all the files with the extension `.pdf` from a folder spe
     ]
 }
 ```
+!! Importing a folder of text files
+
+This example retrieves all the files with the extension `.txt` from a folder specified by a relative path. This folder is within the wiki's base directory, and the current config file is in a directory within the wiki's "tiddlers/" directory. So, in this case the path starts with "../../" to traverse upwards two directory levels, and then down into the "externalnotes/" directory. Each tiddler is set up with the following fields:
+
+* ''title'' - set to the URI decoded base filename of the text file. [[URI decoding|https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/decodeURIComponent]] allows characters like "/" to be included in titles by URI encoding them as "%2F"
+* ''created'' - set to the creation date/time of the text file
+* ''modified'' - set to the modification date/time of the text file
+* ''type'' - set to `text/plain`
+* ''tags'' - set to `[[note]] [[externalnote]]` (using array notation)
+* ''text'' - not set, thus the content of the file is loaded as the text field
+
+```
+{
+    "directories": [
+        {
+            "path": "../../externalnotes",
+            "filesRegExp": ".+\\.txt",
+            "isTiddlerFile": false,
+            "isEditableFile": true,
+            "fields": {
+                "title": {"source": "basename-uri-decoded"},
+                "created": {"source": "created"},
+                "modified": {"source": "modified"},
+                "type": "text/plain",
+                "tags": ["note", "externalnote", ".txt"]
+            }
+        }
+  ]
+}
+```
+
+This will load all text files in the `<wikiBase>/externalnotes/` directory into the wiki as individual tiddlers. These can be a collection of snippets in various markup-languages. Then, the `type` field of each of these tiddlers can be changed to match their languages For example, "text/vnd.tiddlywiki" for wikitext, or "text/markdown" for markdown files. Then, using $:/config/FileSystemPaths and $:/config/FileSystemExtentions tiddlers with the following lines we can cause any changes to these tiddlers to be saved back to the directory they started from, and as "*.txt" files with accompanying "*.txt.meta" files. These meta files will then over-ride any fields generated from the config `tiddlywiki.files` file (such as the tiddler's `type` field) when the server is restarted.
+
+From the examples in [[Customising Tiddler File Naming]] we see that the `[tag[externalnote]addprefix[../externalnotes/]]` filter in the $:/config/FileSystemPaths tiddler catches all tiddlers tagged with `externalnotes` (that have not matched an earlier filter). These tiddlers have "../externalnotes/" appended to their titles to render the final logical path. As this path starts in the wiki's "tiddlers/" folder by default (one folder above the folder holding the above config file) it differes by one set of "../".
+
+Then, the `[tag[.txt]then[.txt]]` filter in the $:/config/FileSystemExtensions tiddler forces all these text files (which start with tag ".txt") to be saved back to disk as *.txt and accompanying *.txt.meta files (overriding the normal tiddler-type to file-type mapping). In this case, allowing the snippets of Tiddlywiki wikitext or markdown-text to be saved back to "text" *.txt files.