1
0
mirror of https://github.com/Jermolene/TiddlyWiki5 synced 2024-11-27 12:07:19 +00:00
TiddlyWiki5/core/modules/server/server.js
Jeremy Ruston b95723a022
Fix syncer to handler errors properly (#4373)
* First commit

* Add throttling of saves

Now we refuse to save a tiddler more often than once per second.

* Wait for a timeout before trying again after an error

* Modest optimisations of isDirty() method

* Synchronise system tiddlers and deletions from the server

Fixes two long-standing issues:

* Changes to system tiddlers are not synchronised from the server to the browser
* Deletions of tiddlers on the server are not propagated to browser clients

* Make sure we update the dirty status even if there isn't a task to perform

* Replace save-wiki button with popup sync menu

* Remove the "Server" control panel tab

We don't need it with the enhanced sync dropdown

* Add indentation to the save-wiki button

* Fix spacing in dropdown menu items

* Switch between cloud icons according to dirty status

* Add a menu item to copy syncer logs to the clipboard

* Improve animated icon

* Remove indentation from save-wiki button

@pmario the annoying thing is that using `\trim whitespace` trims significant whitespace too, so it means we have to use <$text text=" "/> when we need a space that won't be trimmed. For the moment, I've removed the indentation but will keep thinking about it.

* Further icon, UI and copy text tweaks

Move the icons and styles from the core into the TiddlyWeb plugin

* Clean up PR diff

* Tweak animation durations

* Break the actions from the syncer dropdown into separate tiddlers

@pmario I think this makes things a bit easier to follow

* Refactor syncadaptor creation and logging

The goal is for the syncadaptor to be able to log to the same logger as the syncer, so that the "copy syncer logs to clipboard" data is more useful.

* Don't transition the dirty indicator container colour, just the SVG's colour

* Only trigger a sync for changes to tiddlers we're interested in

Otherwise it is triggered by the creation of the alert tiddlers used to display errors.

* Restore deleting local tiddlers removed from the server

(I had commented it out for some testing and accidentally commited it).

* Guard against missing adaptor info

* We still need to trigger a timeout when there was no task to process

* Avoid repeatedly polling for changes

Instead we only trigger a timeout call at if there is a pending task (ie a tiddler that has changed but isn't yet old enough to save).

* Lazy loading: include skinny versions of lazily loaded tiddlers in the index.html

* Introduce _is_skinny field for indicating that a tiddler is subject to lazy loading

* Remove savetrail plugin from prerelease

It doesn't yet work with the new syncer

* Make the savetrail plugin work again

* Clear outstanding alerts when synchronisation is restored

* Logger: only remove alerts from the same component

Missed off 9f5c0de07

* Make the saving throttle interval configurable (#4385)

After switching Bob to use the core syncer the throttle interval makes saving feel very sluggish compared to the message queue setup that I had before.
The editing lock that I use to prevent conflicts with multiple users doesn't go away until the save is completed, and with the 1 second delay it means that if you edit a tiddler and save it than you have to wait one second before you can edit it again.

* Tweaks to appearance of alerts

* Exclude temp tiddlers from offline snapshots

Otherwise alerts will persist

* Tweak appearance of status line in dropdown

* Update release note

* Web server: Don't include full path in error messages

Fixes #3724

* In change event handler check for deletions

* Disable the official plugin library when the tiddlyweb plugin is loaded

* Hide error details from browser for /files/ route

See https://github.com/Jermolene/TiddlyWiki5/issues/3724#issuecomment-565702492 -- thanks @pmario

* Revert all the changes to the relationship between the syncer and the syncadaptor

Previously we had some major rearrangements to make it possible for the syncadaptor to route it's logging to the logger used by the syncer. The motivation is so that the "copy logs to clipboard" button is more useful.

On reflection, changing the interface this drastically is undesirable from a backwards compatibility perspective, so I'm going to investigate other ways to achieve the logger sharing

* Make the tiddlyweb adaptor use the syncer's logger

So that both are availavble when copying the syncer logs to the clipboard

* Update release note

* Support setting port=0 to get an OS assigned port

Quite useful

* Update code comment

* UI: Use "Get latest changes from server" instead of "Refresh"

* Add getUpdatedTiddlers() method to syncadaptor API

See https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-573579495

* Refactor revision handling within the syncer

Thanks @pmario

* Fix typo in tiddlywebadaptor

* Improve presentation of errors

See https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-573695267

* Add docs for getTiddlerRevision()

* Remove unused error animation

* Update comment for GET /recipes/default/tiddlers/tiddlers.json

* Optimise SVG cloud image

* Add optional list of allowed filters for get all tiddlers route

An attempt to address @Arlen22's concern here:

https://github.com/Jermolene/TiddlyWiki5/pull/4373#pullrequestreview-342146190

* Fix network error alert text translatability

* Fix error code and logging for GET /recipes/default/tiddlers/tiddlers.json

Thanks @Arlen22

* Flip GET /recipes/default/tiddlers/tiddlers.json allowed filter handling to be secure by default

* Validate updates received from getUpdatedTiddlers()

* Add syncer method to force loading of a tiddler from the server

* Remove the release note update to remove the merge conflict

* Fix crash when there's no config section in the tiddlywiki.info file

* Use config tiddler title to check filter query (merge into fix-syncer) (#4478)

* Use config tiddler title to check filter query

* Create config-tiddlers-filter.tid

* Add config switch to enable all filters on GET /recipes/default/tiddlers/tiddlers.json

And update docs

* Fix bug when deleting a tiddler with a shadow

Reported by @kookma at https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-604027528

Co-authored-by: jed <inmysocks@fastmail.com>
Co-authored-by: Arlen22 <arlenbee@gmail.com>
2020-03-30 15:24:05 +01:00

274 lines
9.4 KiB
JavaScript

/*\
title: $:/core/modules/server/server.js
type: application/javascript
module-type: library
Serve tiddlers over http
\*/
(function(){
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
if($tw.node) {
var util = require("util"),
fs = require("fs"),
url = require("url"),
path = require("path"),
querystring = require("querystring");
}
/*
A simple HTTP server with regexp-based routes
options: variables - optional hashmap of variables to set (a misnomer - they are really constant parameters)
routes - optional array of routes to use
wiki - reference to wiki object
*/
function Server(options) {
var self = this;
this.routes = options.routes || [];
this.authenticators = options.authenticators || [];
this.wiki = options.wiki;
this.servername = $tw.utils.transliterateToSafeASCII(this.wiki.getTiddlerText("$:/SiteTitle") || "TiddlyWiki5");
// Initialise the variables
this.variables = $tw.utils.extend({},this.defaultVariables);
if(options.variables) {
for(var variable in options.variables) {
if(options.variables[variable]) {
this.variables[variable] = options.variables[variable];
}
}
}
$tw.utils.extend({},this.defaultVariables,options.variables);
// Initialise CSRF
this.csrfDisable = this.get("csrf-disable") === "yes";
// Initialize Gzip compression
this.enableGzip = this.get("gzip") === "yes";
// Initialise authorization
var authorizedUserName = (this.get("username") && this.get("password")) ? this.get("username") : "(anon)";
this.authorizationPrincipals = {
readers: (this.get("readers") || authorizedUserName).split(",").map($tw.utils.trim),
writers: (this.get("writers") || authorizedUserName).split(",").map($tw.utils.trim)
}
// Load and initialise authenticators
$tw.modules.forEachModuleOfType("authenticator", function(title,authenticatorDefinition) {
// console.log("Loading server route " + title);
self.addAuthenticator(authenticatorDefinition.AuthenticatorClass);
});
// Load route handlers
$tw.modules.forEachModuleOfType("route", function(title,routeDefinition) {
// console.log("Loading server route " + title);
self.addRoute(routeDefinition);
});
// Initialise the http vs https
this.listenOptions = null;
this.protocol = "http";
var tlsKeyFilepath = this.get("tls-key"),
tlsCertFilepath = this.get("tls-cert");
if(tlsCertFilepath && tlsKeyFilepath) {
this.listenOptions = {
key: fs.readFileSync(path.resolve($tw.boot.wikiPath,tlsKeyFilepath),"utf8"),
cert: fs.readFileSync(path.resolve($tw.boot.wikiPath,tlsCertFilepath),"utf8")
};
this.protocol = "https";
}
this.transport = require(this.protocol);
}
Server.prototype.defaultVariables = {
port: "8080",
host: "127.0.0.1",
"root-tiddler": "$:/core/save/all",
"root-render-type": "text/plain",
"root-serve-type": "text/html",
"tiddler-render-type": "text/html",
"tiddler-render-template": "$:/core/templates/server/static.tiddler.html",
"system-tiddler-render-type": "text/plain",
"system-tiddler-render-template": "$:/core/templates/wikified-tiddler",
"debug-level": "none",
"gzip": "no"
};
Server.prototype.get = function(name) {
return this.variables[name];
};
Server.prototype.addRoute = function(route) {
this.routes.push(route);
};
Server.prototype.addAuthenticator = function(AuthenticatorClass) {
// Instantiate and initialise the authenticator
var authenticator = new AuthenticatorClass(this),
result = authenticator.init();
if(typeof result === "string") {
$tw.utils.error("Error: " + result);
} else if(result) {
// Only use the authenticator if it initialised successfully
this.authenticators.push(authenticator);
}
};
Server.prototype.findMatchingRoute = function(request,state) {
var pathprefix = this.get("path-prefix") || "";
for(var t=0; t<this.routes.length; t++) {
var potentialRoute = this.routes[t],
pathRegExp = potentialRoute.path,
pathname = state.urlInfo.pathname,
match;
if(pathprefix) {
if(pathname.substr(0,pathprefix.length) === pathprefix) {
pathname = pathname.substr(pathprefix.length) || "/";
match = potentialRoute.path.exec(pathname);
} else {
match = false;
}
} else {
match = potentialRoute.path.exec(pathname);
}
if(match && request.method === potentialRoute.method) {
state.params = [];
for(var p=1; p<match.length; p++) {
state.params.push(match[p]);
}
return potentialRoute;
}
}
return null;
};
Server.prototype.methodMappings = {
"GET": "readers",
"OPTIONS": "readers",
"HEAD": "readers",
"PUT": "writers",
"POST": "writers",
"DELETE": "writers"
};
/*
Check whether a given user is authorized for the specified authorizationType ("readers" or "writers"). Pass null or undefined as the username to check for anonymous access
*/
Server.prototype.isAuthorized = function(authorizationType,username) {
var principals = this.authorizationPrincipals[authorizationType] || [];
return principals.indexOf("(anon)") !== -1 || (username && (principals.indexOf("(authenticated)") !== -1 || principals.indexOf(username) !== -1));
}
Server.prototype.requestHandler = function(request,response) {
// Compose the state object
var self = this;
var state = {};
state.wiki = self.wiki;
state.server = self;
state.urlInfo = url.parse(request.url);
state.queryParameters = querystring.parse(state.urlInfo.query);
// Get the principals authorized to access this resource
var authorizationType = this.methodMappings[request.method] || "readers";
// Check for the CSRF header if this is a write
if(!this.csrfDisable && authorizationType === "writers" && request.headers["x-requested-with"] !== "TiddlyWiki") {
response.writeHead(403,"'X-Requested-With' header required to login to '" + this.servername + "'");
response.end();
return;
}
// Check whether anonymous access is granted
state.allowAnon = this.isAuthorized(authorizationType,null);
// Authenticate with the first active authenticator
if(this.authenticators.length > 0) {
if(!this.authenticators[0].authenticateRequest(request,response,state)) {
// Bail if we failed (the authenticator will have sent the response)
return;
}
}
// Authorize with the authenticated username
if(!this.isAuthorized(authorizationType,state.authenticatedUsername)) {
response.writeHead(401,"'" + state.authenticatedUsername + "' is not authorized to access '" + this.servername + "'");
response.end();
return;
}
// Find the route that matches this path
var route = self.findMatchingRoute(request,state);
// Optionally output debug info
if(self.get("debug-level") !== "none") {
console.log("Request path:",JSON.stringify(state.urlInfo));
console.log("Request headers:",JSON.stringify(request.headers));
console.log("authenticatedUsername:",state.authenticatedUsername);
}
// Return a 404 if we didn't find a route
if(!route) {
response.writeHead(404);
response.end();
return;
}
// Receive the request body if necessary and hand off to the route handler
if(route.bodyFormat === "stream" || request.method === "GET" || request.method === "HEAD") {
// Let the route handle the request stream itself
route.handler(request,response,state);
} else if(route.bodyFormat === "string" || !route.bodyFormat) {
// Set the encoding for the incoming request
request.setEncoding("utf8");
var data = "";
request.on("data",function(chunk) {
data += chunk.toString();
});
request.on("end",function() {
state.data = data;
route.handler(request,response,state);
});
} else if(route.bodyFormat === "buffer") {
var data = [];
request.on("data",function(chunk) {
data.push(chunk);
});
request.on("end",function() {
state.data = Buffer.concat(data);
route.handler(request,response,state);
})
} else {
response.writeHead(400,"Invalid bodyFormat " + route.bodyFormat + " in route " + route.method + " " + route.path.source);
response.end();
}
};
/*
Listen for requests
port: optional port number (falls back to value of "port" variable)
host: optional host address (falls back to value of "host" variable)
prefix: optional prefix (falls back to value of "path-prefix" variable)
*/
Server.prototype.listen = function(port,host,prefix) {
var self = this;
// Handle defaults for port and host
port = port || this.get("port");
host = host || this.get("host");
prefix = prefix || this.get("path-prefix") || "";
// Check for the port being a string and look it up as an environment variable
if(parseInt(port,10).toString() !== port) {
port = process.env[port] || 8080;
}
// Warn if required plugins are missing
if(!$tw.wiki.getTiddler("$:/plugins/tiddlywiki/tiddlyweb") || !$tw.wiki.getTiddler("$:/plugins/tiddlywiki/filesystem")) {
$tw.utils.warning("Warning: Plugins required for client-server operation (\"tiddlywiki/filesystem\" and \"tiddlywiki/tiddlyweb\") are missing from tiddlywiki.info file");
}
// Create the server
var server;
if(this.listenOptions) {
server = this.transport.createServer(this.listenOptions,this.requestHandler.bind(this));
} else {
server = this.transport.createServer(this.requestHandler.bind(this));
}
// Display the port number after we've started listening (the port number might have been specified as zero, in which case we will get an assigned port)
server.on("listening",function() {
var address = server.address();
$tw.utils.log("Serving on " + self.protocol + "://" + address.address + ":" + address.port + prefix,"brown/orange");
$tw.utils.log("(press ctrl-C to exit)","red");
});
// Listen
return server.listen(port,host);
};
exports.Server = Server;
})();