The code here had got a bit broken by some PRs that I should have
checked more carefully. I’ve done a major refactoring which will
hopefully make it easier to understand, and fixes a number of problems:
* Problem with eg .md tiddlers not being deleted correctly
* Problem with Windows path separators not being usable within
$:/config/FileSystemPaths on Windows
* Problem with filename clashes not being detected correctly when
saving to a different directory via $:/config/FileSystemPaths
* Enables slashes within tiddler titles to be mapped into folders
* Enables plain text files like .md and .css to be saved with .meta
files instead of as .tid files (see #2558)
* No longer replaces spaces with underscores
As this is such a major update, I’d be grateful if Node.js users could
give it a careful run through — in particular, you’ll need to try
creating new tiddlers of various types and ensure that the expected
files are created.
We were using `String.prototype.replace()` without addressing the
wrinkle that dollar signs in the replacement string have special
handling. This caused problems in situations where the replacement
string is derived from user input and contains dollar signs.
Fixes#2517
* Save binary tiddlers with meta file
The filesystemadaptor plugin was a little simplistic in its
understanding of a binary file. It was using the typeInfo dictionary to
choose what tiddler types were binary (and hence needed a meta file when
saving).
I looked as if it was trying to be smart by looking for the hasMetaFile
*OR* had the encoding of base64. Unfortunately the typeInfo only defined
image/jpeg and so any other base64 encoded tiddler was assumed to be of
type text/vnd.tiddlywiki.
The net effect was only JPG images got a meta file and everything else
were saved as .tid files with base64 encoding. It all still worked but
made working with binary data in a Git repo a bit daunting.
There is enough information in the $tw.config.contentTypeInfo to
determine if a tiddler type is encoded with base64 or not. A better list
is available from boot/boot.js who registers all the types thorough the
registerFileType and marks then with base64 were appropriate.
This commit uses the typeInfo dictionary first for any filesystem
specific overrides, then the contentTypeInfo, and finally defaults to
the typeInfo["text/vnd.tiddlywiki"]. It also eliminates the now
unnecessary override for image/jpeg.
I think this might have been the original intent from commit 10b192e7.
From my limited testing all files described in boot/boot.js (lines
1832-1856) with an encoding of base64 now save as the original binary
and a meta file. Meaning that when you start the node server and then
drag-n-drop a binary file (i.e. image/png) it will PUT to the server
and then save it on the filesystem as-is allowing the file to be managed
as a binary file and not a text file. (Binary diffs are better and
GitHub supports them as well).
* Prevent duplicate file extensions
A side effects of using the $tw.config.contentFileInfo in the previous
commit is that it will always append a file extension to the tiddler
title when saving. In most cases this is the correct course of action.
However, sometimes that title is already a proper filename with an
extension (for example importing 'foobar.png' would save a file named
'foobar.png.png') which seemed silly.
This commit simply checks to make sure the title does not already end
with the file extension before appending it to the filename. A little
convenience really.
Since IE apparently doesn't have the String endsWith method I took the
liberty to add a helper method to $tw.utils trying to follow the other
polyfill patterns. I figured this was more generic and readable then
attempting to use a one-off solution inline. I got the polyfill code
from MDN.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/endsWith#Polyfill
Is strEndsWith the best method name?
When saving new tiddlers on node.js, allow the user to override the path of the
generated .tid file. This is done by creating a tiddler
$:/config/FileSystemPaths which contains one or more filter expressions, one
per line. These filters are applied in turn to the tiddler to be saved, and
the first output produced is taken as a logical path relative to the wiki's
tiddlers directory. Any occurences of "/" in the logical path are replaced with
the platform's path separator, the extension ".tid" is appended, illegal
characters are replaced by "_" and the path is disambiguated (if necessary) in
order to arrive at the final tiddler file path. If none of the filters matches,
or the configuration tiddler does not exist, fall back to the previous file
naming scheme (i.e. replacing "/" by "_").
This implies we will now, for tiddlers matching the user-specified filters,
create directory trees below the tiddlers directory. In order to avoid
cluttering it with empty directory trees when renaming or removing tiddlers, any
directories that become empty by deleting a tiddler file are removed
(recursively).
Benefits of this configuration option include the ability to organize git
repositories of TiddlyWikis running on node.js, ability to replace characters
that cause trouble with particular operating systems or workflows (e.g. '$' on
unix) and the ability to replicate tiddler "paths" in the filesystem (by
including a filter like "[!has[draft.of]]") without forcing such a (potentially
problematic) change on all users.
Entities such as `👷` were broken because
`String.fromCharCode()` is not fully Unicode aware. The fix is to use
`String.fromCodePoint()` where available.
Noted by @ericshulman
Now we process the rendered HTML of tiddlers, which allows us to
process HTML generated by MS Word. In fact, the HTML that MS Word
generates is so awful, I’ve instead been using Mammoth to do the
conversion: https://github.com/mwilliamson/mammoth.js
Also some necessary improvements to the fake dom implementation.
They don’t get automatically decoded when the browser reads the
resulting HTML. So, instead, we’ll solve
1e9e1a1fdc by switching to double quotes
for attribute values.
This may cause backwards compatibility problems for people relying on
the block mode parsing, but it’s much better for the rest of us as it
avoids an unsightly paragraph tag
Stop the logger from creating alert tiddlers on the server. They
propagate to the client but are not deletable from the client because
they are in the `$:/temp` namespace.
cc @loleg
The previous fix changed the scrolling behaviour such that it only
scrolled to the top of a tiddler if the tiddler was entirely offscreen.
It wasn’t entirely satisfactory because scrolling was prevented even if
only a few pixels of a tiddler are in view. This commit ensures that
the scroll does occur if less than 50 pixels of the target is in view
@aelocson here’s an alternative fix for #981, as discussed in
https://github.com/Jermolene/TiddlyWiki5/commit/691e5719a4ff74a04d389bd1
26ba2a69e7651a2a#commitcomment-9848682
It does seem a lot nicer. It avoids the problem you raised, and also
avoids scrolling when a permalink is used.
I suspect that we’d need to make the hard-coded 50 pixels be
configurable.
