* Module-ize server routes and add static file support (#2510)
* Refactor server routes to modules
New module type: serverroute
Caveats: Loading order is not deterministic but this would only matter
if two route modules attempted to use the same path regexp (that would
be silly).
* Add static assets plugin
This plugin allows the node server to fetch static assets in the /assets
directory. I felt that this was a feature that goes above the core
functionality. That is why I added it as a plugin. with the modular
route extensions this was a breeze.
* Add serverroute description to ModuleTypes
* Coding standards tweaks
* Fix filename typo
* Move support for attachments from a plugin into the core
* Missing "else"
* Refactor server handling
* Introduce a new named parameter scheme for commands
* Move the SimpleServer class into it's own module
* Deprecate the --server command because of the unwieldy syntax
* Add a new --listen command using the new syntax
For example:
tiddlywiki mywiki --listen host:0.0.0.0 port:8090
* Add check for unknown parameters
* Add support for multiple basic authentication credentials in a CSV file
Beware: Passwords are stored in plain text. If that's a problem, use an authenticating proxy and the trusted header authentication approach.
* Refactor module locations
* Rename "serverroute" module type to "route"
* Remove support for verifying optional named command parameters
The idea was to be able to flag unknown parameter names, but requiring a command to pre-specify all the parameter names makes it harder for (say) the listen command to be extensible so that plugins can add new optional parameters that they handle. (This is particularly in the context of work in progress to encapsulate authenticators into their own modules).
* Refactor the two authenticators into separate modules and add support for authorization
* Correct mistaken path.join vs. path.resolve
See https://stackoverflow.com/a/39836259
* Docs for the named command parameters
I'd be grateful if anyone with sufficient Windows experience could confirm that the note about double quotes in "NamedCommandParameters" is correct.
* Be consistent about lower case parameter names
* Do the right thing when we have a username but no password
With a username parameter but no password parameter we'll attribute edits to that username, but not require authentication.
* Remove obsolete code
* Add support for requiring authentication without restricting the username
* Refactor authorization checks
* Return read_only status in /status response
* Fix two code typos
* Add basic support for detecting readonly status and avoiding write errors
We now have syncadaptors returning readonly status and avoid attempting to write to the server if it's going to fail
* Add readonly-styles
We hide editing-related buttons in read only mode
I've made this part of the tiddlyweb plugin but I think a case could be made for putting it into the core.
* Add custom request header as CSRF mitigation
By default we require the header X-Requested-With to be set to TiddlyWiki. Can be overriden by setting csrfdisable to "yes"
See https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)_Prevention_Cheat_Sheet#Protecting_REST_Services:_Use_of_Custom_Request_Headers
* Add support for HTTPS
* First pass at a route for serving rendered tiddlers
cc @Drakor
* Tweaks to the single tiddler static view
Adding a simple sidebar
* Switch to "dash" separated parameter names
* Typo
* Docs: Update ServerCommand and ListenCommand
* First pass at docs for the new web server stuff
Writing the docs is turning out to be quite an undertaking, much harder than writing the code!
* Get rid of extraneous paragraphs in static renderings
* Rejig anonymous user handling
Now we can support wikis that are read-only for anonymous access, but allow a user to login for read/write access.
* More docs
Slowly getting there...
* Static tiddler rendering: Fix HTML content in page title
* Docs updates
* Fix server command parameter names
Missed off 30ce7ea
* Docs: Missing quotes
* Avoid inadvertent dependency on Node.js > v9.6.0
The listenOptions parameter of the plain HTTP version of CreateServer was only introduced in v9.6.0
cc @Drakor @pmario
* Typo
* Refactored the is operator for simplicity and efficiency.
* Improve `is` filter documentation.
* Update is.js
* extracted `subops.length` to `num_of_subops`
* renamed `subop` to `operator` for clarity/differentiation from `subops`
* refactored to avoid using a `Set` object.
* Update is.js
* Fix WebDAV by requesting new ETag conditionally
For me. this was saving only the first time and subsequently failing.
Having revised the requests, I noticed it didn't get a new ETag after saving.
Seems not all WebDAV implementations return a new ETag in PUT requests.
In my WebDAV service (WsgiDAV) - ETag is only served from a HEAD
request.
So if no ETag is found with PUT - we request one with HEAD.
This patch fixes error handling and should also work with servers that
provide ETag directly upon PUT.
* Add tweak from PMario
* Adds a range widget
Adds a range widget that supports all the range attributes on a input[type=range] element. These include min max and increment.
* Update range.js
* Fix spaces vs tabs.
* Added documentation.
* Modify the is operator to allow multiple types to be specified.
* Fixed indentation.
* Fixed indentation.
* Rewritten to maintain input order when multiple filters provided.
* Updated documentation.
* Update is.tid
* Invoke hook when server starts
Invokes the `th-server-command-start` hook when the server is started, with the server object as the parameter. This allows adding a WebSocket listener to the server.
* Return the HTTP server from the listen function
Returns the node HTTP server created in the listen function to allow extension
* Add node HTTP server to server-command-start hook
* Change hook to post start in case we add a pre-start hook
* Create Hook__th-server-command-post-start.tid
* sets checked ckass for radio widget wrapper
* added tc-radio as standard class for radio widgets
* removed selectedClass again, as suggested
@pmario ;-)
* fixed the "0 is not a number bug" in listops and x-listops
* Fixed one comment
* "default" is not a good name for a variable
* Following code styles.
Moving getInt to utils.
* Removing unwanted spaces introduced by me
When occuring in a tiddler named "New Tiddler", the following construct should return ++ instead of +[[New Tiddler]]+
<$set name="myVariable" filter="[all[current]field:title[New Tiddler]]" value="">
+<<myVariable>>+
</$set>
Spending a bit more time with Chrome dev tools, and further to 254e1ca, this optimisation reduces the rendering time for the sample TOC from 1.9s to about 0.9s...
* extend ListWidget to enforce classes
Previously, the undocumented *class* attribute only allowed to specify
additional classes to be set.
Especially for use within a LinkCatcher, you can now apply / enforce
only the custom classes and avoid any of the defaults being applied
depending on the link target.
This will allow to implement #1161 more gracefully.
* use setClass insted of exclamation mark syntax
update docs & fix typo in docs
Remove URI encoding from PUT saver and let the browser handle it as necessary. This seems to be the normal way of doing things. We have confirmed that several WebDAV servers do not expect the file names to be double-encoded.
A first pass at fixing #2973 - comments are welcome. If everybody is
happy, the next step is to deprecate the existing
rendertiddler(s)/savetiddler(s) commands and replace them in all
examples and tiddlywiki.info files with the new equivalents.
This change makes it possible to perform the conversion from target
tiddler title to `href` value as a filter expression (previously a
JavaScript macro was needed to use the tv-get-export-link variable)
* Add ability to now macro to return same UTC string used in the modified and created fields.
* Revert "Add ability to now macro to return same UTC string used in the modified and created fields."
This reverts commit 7b6ad7db4a.
* Mods to formatDateString to adjust displayed date to UTC for [UTC]
format. Corresponding doc changes, and optimization for special
case.
document.location gives us an URL encoded version of the original
filename, so we decode it to recover the original filename for saving.
Fixes#2828Fixes#2819
* Trim tiddler titles imported via JSON
Otherwise, it's possible to create a tiddler with a trailing space (or a
leading one, I suppose) in its title. TiddlyWiki, in general, trims
titles before operating on a tiddler, so having a tiddler with a
trailing space ends up making that tiddler, for all intents and
purposes, uneditable.
Fixes GH #2850
* Signing the CLA
A couple of fixes:
* Fixed problem with loading saver in a file URI within Beaker
* Fixed problem when saving to a directory-style URI with an implicit
“/index.html”
* Switched to double quotes for strings
These changes allow drag and drop to work with one issue: <a> links are
not draggable; draggable divs, spans, buttons etc. seem to work fine.
There’s some issue with IE11 that I don’t understand.
For testing, you can force links to become spans by changing line 64 of
$:/core/modules/widgets/link.js to:
var domNode = this.document.createElement("span");
We do this so that we don’t accidentally modify shadow tiddlers when we
drag them to reorder them within their tag parent. Otherwise, moving a
toolbar button like $:/core/ui/Buttons/permaview in the control panel
will override the shadow tiddler.
At the moment, we support JSON files containing an array of tiddlers.
With this change the core will import files containing a single
tiddler. Also adding templates for saving individual tiddlers in JSON
format
The problem was that `this.responseText` crashes for non-text data. We
fix it by letting the client specify which property should be returned.
@ericshulman does this work for you?
* provides a "close plugin library" button
* starting from #1718 by @inmysocks
* possibly fixes all of #1718, #1597, and #2067
* corrected code comment
* add back title
No idea why I overlooked this beforehand.
Next time I will not suggest such changes that have little to do with
the PR, but rather just comment the code.
* ah, sorry, "fixed" wrong spot
now
```
<$action-sendmessage $message="tm-load-plugin-from-library"
url={{!!url}} title={{$(assetInfo)$!!original-title}}/>
```
...should be back at the right spot.
* mhhh... still fixing the mess
load, unload, puh... let's see if I got it now
* added unloadIFrame and minor syntax fixes
We now use highlight.js in raw HTML mode on the server, rather than
trying to use it with the fakedom. This causes problems with fakedoms
inability to get textContent for a node that has been created by
assigning innerHTML. So we extend the fakedom to allow the original
text content to be saved.
See #2778 for discussion.
I used this test:
console.time();for(var t=0; t<200; t++)
{$tw.wiki.filterTiddlers("[all[tiddlers+shadows]sameday[20170210]]");};c
onsole.timeEnd()
Before this patch, I got speeds of approx 190ms, versus 140ms
afterwards.
Note that the ability to add a cache property like this is only
possible because tiddler objects are immutable.
Like the load command except retrieves the file over HTTP/HTTPS.
Allows experimentation with server-side twederation
This is a cleaned up version of code that I wrote last year at TWEUM
2016 @inmysocks @pmario @twMat @xcazin
* putSaver: detect edit conflicts to prevent clobbering, if possible
if the server supplies an ETag, we send it back when saving, allowing
the server to detect edit conflicts and respond with 412 (cf.
https://www.w3.org/1999/04/Editing/)
caveats:
* this only kicks in after the first save, as we don't have access to
the ETag when first loading the document
* there's no recovery mechanism (e.g. resetting `this.etag` in order to
force clobbering), other than manually reloading the document
* putSaver: retrieve ETag upon initialization for clobbering protection
this addresses one of the caveats from the previous commit
(2d75cb83af) - while theoretically prone
to a race condition, it seems unlikely that saving will be triggered
before the server responds
* putSaver: simplify URI extraction
this simplifies the approach introduced in
f51f6bf774, with the purpose of removing
the fragment identifier
* putSaver: localize error message
* putSaver: switch to built-in HTTP helper
in the process, fixed ETag assignment in `#save` method (was
`this.etag`, now `self.etag`) as well as a syntax error due to a missing
closing brace
* putSaver: consolidate URI handling
The code here had got a bit broken by some PRs that I should have
checked more carefully. I’ve done a major refactoring which will
hopefully make it easier to understand, and fixes a number of problems:
* Problem with eg .md tiddlers not being deleted correctly
* Problem with Windows path separators not being usable within
$:/config/FileSystemPaths on Windows
* Problem with filename clashes not being detected correctly when
saving to a different directory via $:/config/FileSystemPaths
* Enables slashes within tiddler titles to be mapped into folders
* Enables plain text files like .md and .css to be saved with .meta
files instead of as .tid files (see #2558)
* No longer replaces spaces with underscores
As this is such a major update, I’d be grateful if Node.js users could
give it a careful run through — in particular, you’ll need to try
creating new tiddlers of various types and ensure that the expected
files are created.
The request handler may be used by ExpressJS apps directly and can do most of the heavy lifting without any modification. Note that the self variable must be assignee using `[Function].bind(null,SimpleServer instance)`.
* allow radio widget to set an index in a data tiddler
* updated RadioWidget docs, with same demo macro as for CheckboxWidget
in #2103
* removed docs in widget code (seems the wrong place)
* added from version to docs
* revert doc maros to master
* using wikitext-example-without-html and .tip macro now
* fix quotes
given a list `A B C D` if I run `A B C D +[move:-1 [A]]` I get `B C A D`. However, if I were to do `A B C D +[move:1[D]]` it doesn't wrap around, and I get `A B C D`. This fixes that such that `A B C D +[move:-1 [A]]` gives 'A B C D`
When renaming an existing tiddler, the edit template now shows a
checkbox that determines whether or not to relink references to the
tiddler in the list or tags fields of other tiddlers.
This fixes a problem introduced in
c7b31b0242.
The changes by @tobibeer inadvertently made the regular expression
evaluation significantly more expensive because of lookahead. The is
less elegant but reverts the performance problem.
* return all wikiparserrules w/o operand
* simpler layout & code / updated instruction details
Also wanted to link each rule to the official docs using a dictionary at
`$:/language/Docs/ParserRules/`. However, without #2194 this is not
doable.
This commit permits language plugins to carry the field
“text-direction” with the value “rtl” to trigger right-to-left layout
of the entire page. We also adjust the sidebar layout to work in RTL
mode.
There are still a number of problems to be addressed:
* Brackets and other punctuation incorrectly placed within en-GB UI text
* System tiddler titles are rendered semi-back-to-front (eg
`languages/ca-ES/:$`)
Starting to address #1845 and the discussion in #2523.
We were using `String.prototype.replace()` without addressing the
wrinkle that dollar signs in the replacement string have special
handling. This caused problems in situations where the replacement
string is derived from user input and contains dollar signs.
Fixes#2517
* Save binary tiddlers with meta file
The filesystemadaptor plugin was a little simplistic in its
understanding of a binary file. It was using the typeInfo dictionary to
choose what tiddler types were binary (and hence needed a meta file when
saving).
I looked as if it was trying to be smart by looking for the hasMetaFile
*OR* had the encoding of base64. Unfortunately the typeInfo only defined
image/jpeg and so any other base64 encoded tiddler was assumed to be of
type text/vnd.tiddlywiki.
The net effect was only JPG images got a meta file and everything else
were saved as .tid files with base64 encoding. It all still worked but
made working with binary data in a Git repo a bit daunting.
There is enough information in the $tw.config.contentTypeInfo to
determine if a tiddler type is encoded with base64 or not. A better list
is available from boot/boot.js who registers all the types thorough the
registerFileType and marks then with base64 were appropriate.
This commit uses the typeInfo dictionary first for any filesystem
specific overrides, then the contentTypeInfo, and finally defaults to
the typeInfo["text/vnd.tiddlywiki"]. It also eliminates the now
unnecessary override for image/jpeg.
I think this might have been the original intent from commit 10b192e7.
From my limited testing all files described in boot/boot.js (lines
1832-1856) with an encoding of base64 now save as the original binary
and a meta file. Meaning that when you start the node server and then
drag-n-drop a binary file (i.e. image/png) it will PUT to the server
and then save it on the filesystem as-is allowing the file to be managed
as a binary file and not a text file. (Binary diffs are better and
GitHub supports them as well).
* Prevent duplicate file extensions
A side effects of using the $tw.config.contentFileInfo in the previous
commit is that it will always append a file extension to the tiddler
title when saving. In most cases this is the correct course of action.
However, sometimes that title is already a proper filename with an
extension (for example importing 'foobar.png' would save a file named
'foobar.png.png') which seemed silly.
This commit simply checks to make sure the title does not already end
with the file extension before appending it to the filename. A little
convenience really.
Since IE apparently doesn't have the String endsWith method I took the
liberty to add a helper method to $tw.utils trying to follow the other
polyfill patterns. I figured this was more generic and readable then
attempting to use a one-off solution inline. I got the polyfill code
from MDN.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/endsWith#Polyfill
Is strEndsWith the best method name?
It turns out that IE11 has a horrible bug whereby setting the
placeholder attribute before setting the text will trigger an input
event:
https://social.msdn.microsoft.com/Forums/ie/en-US/ae4832b0-8eee-4729-b93
3-a9977ea1b583/internet-explorer-input-event-get-fired-when-settingunset
ting-the-placeholder?forum=iewebdevelopment
There have been long standing problems with the buttons not working in
Firefox, and now a [further
problem](https://groups.google.com/d/msg/tiddlywiki/GlsruQyPOag/BAhnI2mt
BgAJ) has come to light, and been similarly hard to resolve.
On balance, I’ve decided to remove the undo/redo buttons from the
toolbar (the ctrl/cmd-Z shortcut is still operational), thus avoiding
the problems, and saving some space in the core.
This is quite a big change: a new way to invoke action widgets.
The advantage is that it solves #2217 and #1564, a long running problem
that prevented us from adding action widgets to widgets that modify the
store.
This commit adds the new technique for the button and keyboard widgets,
but also extends the select widget to trigger action widgets for the
first time
* Change the negation logic to address an edge case
Make it possible to get an interval ending with yesterday or starting with tomorrow.
* "days" filter: adjust documentation
When saving new tiddlers on node.js, allow the user to override the path of the
generated .tid file. This is done by creating a tiddler
$:/config/FileSystemPaths which contains one or more filter expressions, one
per line. These filters are applied in turn to the tiddler to be saved, and
the first output produced is taken as a logical path relative to the wiki's
tiddlers directory. Any occurences of "/" in the logical path are replaced with
the platform's path separator, the extension ".tid" is appended, illegal
characters are replaced by "_" and the path is disambiguated (if necessary) in
order to arrive at the final tiddler file path. If none of the filters matches,
or the configuration tiddler does not exist, fall back to the previous file
naming scheme (i.e. replacing "/" by "_").
This implies we will now, for tiddlers matching the user-specified filters,
create directory trees below the tiddlers directory. In order to avoid
cluttering it with empty directory trees when renaming or removing tiddlers, any
directories that become empty by deleting a tiddler file are removed
(recursively).
Benefits of this configuration option include the ability to organize git
repositories of TiddlyWikis running on node.js, ability to replace characters
that cause trouble with particular operating systems or workflows (e.g. '$' on
unix) and the ability to replicate tiddler "paths" in the filesystem (by
including a filter like "[!has[draft.of]]") without forcing such a (potentially
problematic) change on all users.
Entities such as `👷` were broken because
`String.fromCharCode()` is not fully Unicode aware. The fix is to use
`String.fromCodePoint()` where available.
Noted by @ericshulman
List fields (such as tags) when evaluated to produce tiddlers result in empty arrays. Using the exact not equals, an empty array is not the same as an empty string. By using equivelent not equals, we state that the field is either != "" or anything that can be coerced to "". Which, based on https://dorey.github.io/JavaScript-Equality-Table/ is `false` `0` `[]` or `[[]]`` neither `false` nor `0` can be set as a tiddler field as both will end up being quoted (`"false"`, `"0"`) so this should work.
When using the widget with only short form SetText attributes (field="value) -- the default field (text) of the default tiddler is set to the default value ("") -- thus clearing the text field of the current tiddler. I have inserted a conditional to test for the presence of the `field` attribute.
Whilst I was about this, I have inserted code to enable the setting of any number of TextReferences -- this fixes the issue of only being able to set one index per widget and also allows different tiddlers to be targeted by the same widget.
