Module-ize server routes, add static file support and other enhancements(#2679)

* Module-ize server routes and add static file support (#2510)

* Refactor server routes to modules

New module type: serverroute

Caveats: Loading order is not deterministic but this would only matter
if two route modules attempted to use the same path regexp (that would
be silly).

* Add static assets plugin

This plugin allows the node server to fetch static assets in the /assets
directory. I felt that this was a feature that goes above the core
functionality. That is why I added it as a plugin. with the modular
route extensions this was a breeze.

* Add serverroute description to ModuleTypes

* Coding standards tweaks

* Fix filename typo

* Move support for attachments from a plugin into the core

* Missing "else"

* Refactor server handling

* Introduce a new named parameter scheme for commands
* Move the SimpleServer class into it's own module
* Deprecate the --server command because of the unwieldy syntax
* Add a new --listen command using the new syntax

For example:

tiddlywiki mywiki --listen host:0.0.0.0 port:8090

* Add check for unknown parameters

* Add support for multiple basic authentication credentials in a CSV file

Beware: Passwords are stored in plain text. If that's a problem, use an authenticating proxy and the trusted header authentication approach.

* Refactor module locations

* Rename "serverroute" module type to "route"

* Remove support for verifying optional named command parameters

The idea was to be able to flag unknown parameter names, but requiring a command to pre-specify all the parameter names makes it harder for (say) the listen command to be extensible so that plugins can add new optional parameters that they handle. (This is particularly in the context of work in progress to encapsulate authenticators into their own modules).

* Refactor the two authenticators into separate modules and add support for authorization

* Correct mistaken path.join vs. path.resolve

See https://stackoverflow.com/a/39836259

* Docs for the named command parameters

I'd be grateful if anyone with sufficient Windows experience could confirm that the note about double quotes in "NamedCommandParameters" is correct.

* Be consistent about lower case parameter names

* Do the right thing when we have a username but no password

With a username parameter but no password parameter we'll attribute edits to that username, but not require authentication.

* Remove obsolete code

* Add support for requiring authentication without restricting the username

* Refactor authorization checks

* Return read_only status in /status response

* Fix two code typos

* Add basic support for detecting readonly status and avoiding write errors

We now have syncadaptors returning  readonly status and avoid attempting to write to the server if it's going to fail

* Add readonly-styles

We hide editing-related buttons in read only mode

I've made this part of the tiddlyweb plugin but I think a case could be made for putting it into the core.

* Add custom request header as CSRF mitigation

By default we require the header X-Requested-With to be set to TiddlyWiki. Can be overriden by setting csrfdisable to "yes"

See https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)_Prevention_Cheat_Sheet#Protecting_REST_Services:_Use_of_Custom_Request_Headers

* Add support for HTTPS

* First pass at a route for serving rendered tiddlers

cc @Drakor

* Tweaks to the single tiddler static view

Adding a simple sidebar

* Switch to "dash" separated parameter names

* Typo

* Docs: Update ServerCommand and ListenCommand

* First pass at docs for the new web server stuff

Writing the docs is turning out to be quite an undertaking, much harder than writing the code!

* Get rid of extraneous paragraphs in static renderings

* Rejig anonymous user handling

Now we can support wikis that are read-only for anonymous access, but allow a user to login for read/write access.

* More docs

Slowly getting there...

* Static tiddler rendering: Fix HTML content in page title

* Docs updates

* Fix server command parameter names

Missed off 30ce7ea

* Docs: Missing quotes

* Avoid inadvertent dependency on Node.js > v9.6.0

The listenOptions parameter of the plain HTTP version of CreateServer was only introduced in v9.6.0

cc @Drakor @pmario

* Typo
This commit is contained in:
snlhnk
2018-07-18 16:54:43 +01:00
committed by Jeremy Ruston
parent 9735e13dea
commit c05c0d3df6
68 changed files with 1555 additions and 336 deletions
@@ -2,6 +2,7 @@ title: $:/language/Docs/ModuleTypes/
allfilteroperator: A sub-operator for the ''all'' filter operator.
animation: Animations that may be used with the RevealWidget.
authenticator: Defines how requests are authenticated by the built-in HTTP server.
bitmapeditoroperation: A bitmap editor toolbar operation.
command: Commands that can be executed under Node.js.
config: Data to be inserted into `$tw.config`.
@@ -12,6 +13,7 @@ isfilteroperator: Operands for the ''is'' filter operator.
library: Generic module type for general purpose JavaScript modules.
macro: JavaScript macro definitions.
parser: Parsers for different content types.
route: Defines how individual URL patterns are handled by the built-in HTTP server.
saver: Savers handle different methods for saving files from the browser.
startup: Startup functions.
storyview: Story views customise the animation and behaviour of list widgets.
+30
View File
@@ -0,0 +1,30 @@
title: $:/language/Help/listen
description: Provides an HTTP server interface to TiddlyWiki
Serves a wiki over HTTP.
The listen command uses NamedCommandParameters:
```
--listen [<name>=<value>]...
```
All parameters are optional with safe defaults, and can be specified in any order. The recognised parameters are:
* ''host'' - optional hostname to serve from (defaults to "127.0.0.1" aka "localhost")
* ''path-prefix'' - optional prefix for paths
* ''port'' - port number on which to listen; non-numeric values are interpreted as a system environment variable from which the port number is extracted (defaults to "8080")
* ''credentials'' - pathname of credentials CSV file (relative to wiki folder)
* ''anon-username'' - the username for signing edits for anonymous users
* ''username'' - optional username for basic authentication
* ''password'' - optional password for basic authentication
* ''authenticated-user-header'' - optional name of header to be used for trusted authentication
* ''readers'' - comma separated list of principals allowed to write to this wiki
* ''writers'' - comma separated list of principals allowed to read from this wiki
* ''csrf-disable'' - set to "yes" to disable CSRF checks (defaults to "no")
* ''root-tiddler'' - the tiddler to serve at the root (defaults to "$:/core/save/all")
* ''root-render-type'' - the content type to which the root tiddler should be rendered (defaults to "text/plain")
* ''root-serve-type'' - the content type with which the root tiddler should be served (defaults to "text/html")
* ''tls-cert'' - pathname of TLS certificate file (relative to wiki folder)
* ''tls-key'' - pathname of TLS key file (relative to wiki folder)
* ''debug-level'' - optional debug level; set to "debug" to view request details (defaults to "none")
+9 -12
View File
@@ -1,27 +1,25 @@
title: $:/language/Help/server
description: Provides an HTTP server interface to TiddlyWiki
description: Provides an HTTP server interface to TiddlyWiki (deprecated in favour of the new listen command)
The server built in to TiddlyWiki5 is very simple. Although compatible with TiddlyWeb it doesn't support many of the features needed for robust Internet-facing usage.
At the root, it serves a rendering of a specified tiddler. Away from the root, it serves individual tiddlers encoded in JSON, and supports the basic HTTP operations for `GET`, `PUT` and `DELETE`.
Legacy command to serve a wiki over HTTP.
```
--server <port> <roottiddler> <rendertype> <servetype> <username> <password> <host> <pathprefix>
--server <port> <root-tiddler> <root-render-type> <root-serve-type> <username> <password> <host> <path-prefix> <debug-level>
```
The parameters are:
* ''port'' - port number on which to listen; non-numeric values are interpreted as a system environment variable from which the port number is extracted (defaults to "8080")
* ''roottiddler'' - the tiddler to serve at the root (defaults to "$:/core/save/all")
* ''rendertype'' - the content type to which the root tiddler should be rendered (defaults to "text/plain")
* ''servetype'' - the content type with which the root tiddler should be served (defaults to "text/html")
* ''root-tiddler'' - the tiddler to serve at the root (defaults to "$:/core/save/all")
* ''root-render-type'' - the content type to which the root tiddler should be rendered (defaults to "text/plain")
* ''rooot-serve-type'' - the content type with which the root tiddler should be served (defaults to "text/html")
* ''username'' - the default username for signing edits
* ''password'' - optional password for basic authentication
* ''host'' - optional hostname to serve from (defaults to "127.0.0.1" aka "localhost")
* ''pathprefix'' - optional prefix for paths
* ''debuglevel'' - optional debug level; set to "debug" to view request details (defaults to "none")
* ''path-prefix'' - optional prefix for paths
* ''debug-level'' - optional debug level; set to "debug" to view request details (defaults to "none")
If the password parameter is specified then the browser will prompt the user for the username and password. Note that the password is transmitted in plain text so this implementation isn't suitable for general use.
If the password parameter is specified then the browser will prompt the user for the username and password. Note that the password is transmitted in plain text so this implementation should only be used on a trusted network or over HTTPS.
For example:
@@ -37,7 +35,6 @@ The username and password can be specified as empty strings if you need to set t
To run multiple TiddlyWiki servers at the same time you'll need to put each one on a different port. It can be useful to use an environment variable to pass the port number to the Node.js process. This example references an environment variable called "MY_PORT_NUMBER":
```
--server MY_PORT_NUMBER $:/core/save/all text/plain text/html MyUserName passw0rd
```
+36
View File
@@ -94,6 +94,13 @@ Commander.prototype.executeNextCommand = function() {
if(this.verbose) {
this.streams.output.write("Executing command: " + commandName + " " + params.join(" ") + "\n");
}
// Parse named parameters if required
if(command.info.namedParameterMode) {
params = this.extractNamedParameters(params,command.info.mandatoryParameters);
if(typeof params === "string") {
return this.callback(params);
}
}
if(command.info.synchronous) {
// Synchronous command
c = new command.Command(params,this);
@@ -122,6 +129,35 @@ Commander.prototype.executeNextCommand = function() {
}
};
/*
Given an array of parameter strings `params` in name:value format, and an array of mandatory parameter names in `mandatoryParameters`, returns a hashmap of values or a string if error
*/
Commander.prototype.extractNamedParameters = function(params,mandatoryParameters) {
mandatoryParameters = mandatoryParameters || [];
var errors = [],
paramsByName = Object.create(null);
// Extract the parameters
$tw.utils.each(params,function(param) {
var index = param.indexOf("=");
if(index < 1) {
errors.push("malformed named parameter: '" + param + "'");
}
paramsByName[param.slice(0,index)] = $tw.utils.trim(param.slice(index+1));
});
// Check the mandatory parameters are present
$tw.utils.each(mandatoryParameters,function(mandatoryParameter) {
if(!$tw.utils.hop(paramsByName,mandatoryParameter)) {
errors.push("missing mandatory parameter: '" + mandatoryParameter + "'");
}
});
// Return any errors
if(errors.length > 0) {
return errors.join(" and\n");
} else {
return paramsByName;
}
};
Commander.initCommands = function(moduleType) {
moduleType = moduleType || "command";
$tw.commands = {};
+48
View File
@@ -0,0 +1,48 @@
/*\
title: $:/core/modules/commands/listen.js
type: application/javascript
module-type: command
Listen for HTTP requests and serve tiddlers
\*/
(function(){
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
var Server = require("$:/core/modules/server/server.js").Server;
exports.info = {
name: "listen",
synchronous: true,
namedParameterMode: true,
mandatoryParameters: [],
};
var Command = function(params,commander,callback) {
var self = this;
this.params = params;
this.commander = commander;
this.callback = callback;
};
Command.prototype.execute = function() {
var self = this;
if(!$tw.boot.wikiTiddlersPath) {
$tw.utils.warning("Warning: Wiki folder '" + $tw.boot.wikiPath + "' does not exist or is missing a tiddlywiki.info file");
}
// Set up server
this.server = new Server({
wiki: this.commander.wiki,
variables: self.params
});
var nodeServer = this.server.listen();
$tw.hooks.invokeHook("th-server-command-post-start",this.server,nodeServer);
return null;
};
exports.Command = Command;
})();
+19 -289
View File
@@ -3,7 +3,7 @@ title: $:/core/modules/commands/server.js
type: application/javascript
module-type: command
Serve tiddlers over http
Deprecated legacy command for serving tiddlers
\*/
(function(){
@@ -12,311 +12,41 @@ Serve tiddlers over http
/*global $tw: false */
"use strict";
if($tw.node) {
var util = require("util"),
fs = require("fs"),
url = require("url"),
path = require("path"),
http = require("http");
}
var Server = require("$:/core/modules/server/server.js").Server;
exports.info = {
name: "server",
synchronous: true
};
/*
A simple HTTP server with regexp-based routes
*/
function SimpleServer(options) {
this.routes = options.routes || [];
this.wiki = options.wiki;
this.variables = options.variables || {};
}
SimpleServer.prototype.set = function(obj) {
var self = this;
$tw.utils.each(obj,function(value,name) {
self.variables[name] = value;
});
};
SimpleServer.prototype.get = function(name) {
return this.variables[name];
};
SimpleServer.prototype.addRoute = function(route) {
this.routes.push(route);
};
SimpleServer.prototype.findMatchingRoute = function(request,state) {
var pathprefix = this.get("pathprefix") || "";
for(var t=0; t<this.routes.length; t++) {
var potentialRoute = this.routes[t],
pathRegExp = potentialRoute.path,
pathname = state.urlInfo.pathname,
match;
if(pathprefix) {
if(pathname.substr(0,pathprefix.length) === pathprefix) {
pathname = pathname.substr(pathprefix.length) || "/";
match = potentialRoute.path.exec(pathname);
} else {
match = false;
}
} else {
match = potentialRoute.path.exec(pathname);
}
if(match && request.method === potentialRoute.method) {
state.params = [];
for(var p=1; p<match.length; p++) {
state.params.push(match[p]);
}
return potentialRoute;
}
}
return null;
};
SimpleServer.prototype.checkCredentials = function(request,incomingUsername,incomingPassword) {
var header = request.headers.authorization || "",
token = header.split(/\s+/).pop() || "",
auth = $tw.utils.base64Decode(token),
parts = auth.split(/:/),
username = parts[0],
password = parts[1];
if(incomingUsername === username && incomingPassword === password) {
return "ALLOWED";
} else {
return "DENIED";
}
};
SimpleServer.prototype.requestHandler = function(request,response) {
// Compose the state object
var self = this;
var state = {};
state.wiki = self.wiki;
state.server = self;
state.urlInfo = url.parse(request.url);
// Optionally output debug info
if(self.get("debugLevel") !== "none") {
console.log("Request path:",JSON.stringify(state.urlInfo));
console.log("Request headers:",JSON.stringify(request.headers));
}
// Find the route that matches this path
var route = self.findMatchingRoute(request,state);
// Check for the username and password if we've got one
var username = self.get("username"),
password = self.get("password");
if(username && password) {
// Check they match
if(self.checkCredentials(request,username,password) !== "ALLOWED") {
var servername = state.wiki.getTiddlerText("$:/SiteTitle") || "TiddlyWiki5";
response.writeHead(401,"Authentication required",{
"WWW-Authenticate": 'Basic realm="Please provide your username and password to login to ' + servername + '"'
});
response.end();
return;
}
}
// Return a 404 if we didn't find a route
if(!route) {
response.writeHead(404);
response.end();
return;
}
// Set the encoding for the incoming request
// TODO: Presumably this would need tweaking if we supported PUTting binary tiddlers
request.setEncoding("utf8");
// Dispatch the appropriate method
switch(request.method) {
case "GET": // Intentional fall-through
case "DELETE":
route.handler(request,response,state);
break;
case "PUT":
var data = "";
request.on("data",function(chunk) {
data += chunk.toString();
});
request.on("end",function() {
state.data = data;
route.handler(request,response,state);
});
break;
}
};
SimpleServer.prototype.listen = function(port,host) {
return http.createServer(this.requestHandler.bind(this)).listen(port,host);
};
var Command = function(params,commander,callback) {
var self = this;
this.params = params;
this.commander = commander;
this.callback = callback;
// Set up server
this.server = new SimpleServer({
wiki: this.commander.wiki
});
// Add route handlers
this.server.addRoute({
method: "PUT",
path: /^\/recipes\/default\/tiddlers\/(.+)$/,
handler: function(request,response,state) {
var title = decodeURIComponent(state.params[0]),
fields = JSON.parse(state.data);
// Pull up any subfields in the `fields` object
if(fields.fields) {
$tw.utils.each(fields.fields,function(field,name) {
fields[name] = field;
});
delete fields.fields;
}
// Remove any revision field
if(fields.revision) {
delete fields.revision;
}
state.wiki.addTiddler(new $tw.Tiddler(state.wiki.getCreationFields(),fields,{title: title},state.wiki.getModificationFields()));
var changeCount = state.wiki.getChangeCount(title).toString();
response.writeHead(204, "OK",{
Etag: "\"default/" + encodeURIComponent(title) + "/" + changeCount + ":\"",
"Content-Type": "text/plain"
});
response.end();
}
});
this.server.addRoute({
method: "DELETE",
path: /^\/bags\/default\/tiddlers\/(.+)$/,
handler: function(request,response,state) {
var title = decodeURIComponent(state.params[0]);
state.wiki.deleteTiddler(title);
response.writeHead(204, "OK", {
"Content-Type": "text/plain"
});
response.end();
}
});
this.server.addRoute({
method: "GET",
path: /^\/$/,
handler: function(request,response,state) {
response.writeHead(200, {"Content-Type": state.server.get("serveType")});
var text = state.wiki.renderTiddler(state.server.get("renderType"),state.server.get("rootTiddler"));
response.end(text,"utf8");
}
});
this.server.addRoute({
method: "GET",
path: /^\/status$/,
handler: function(request,response,state) {
response.writeHead(200, {"Content-Type": "application/json"});
var text = JSON.stringify({
username: state.server.get("username"),
space: {
recipe: "default"
},
tiddlywiki_version: $tw.version
});
response.end(text,"utf8");
}
});
this.server.addRoute({
method: "GET",
path: /^\/favicon.ico$/,
handler: function(request,response,state) {
response.writeHead(200, {"Content-Type": "image/x-icon"});
var buffer = state.wiki.getTiddlerText("$:/favicon.ico","");
response.end(buffer,"base64");
}
});
this.server.addRoute({
method: "GET",
path: /^\/recipes\/default\/tiddlers.json$/,
handler: function(request,response,state) {
response.writeHead(200, {"Content-Type": "application/json"});
var tiddlers = [];
state.wiki.forEachTiddler({sortField: "title"},function(title,tiddler) {
var tiddlerFields = {};
$tw.utils.each(tiddler.fields,function(field,name) {
if(name !== "text") {
tiddlerFields[name] = tiddler.getFieldString(name);
}
});
tiddlerFields.revision = state.wiki.getChangeCount(title);
tiddlerFields.type = tiddlerFields.type || "text/vnd.tiddlywiki";
tiddlers.push(tiddlerFields);
});
var text = JSON.stringify(tiddlers);
response.end(text,"utf8");
}
});
this.server.addRoute({
method: "GET",
path: /^\/recipes\/default\/tiddlers\/(.+)$/,
handler: function(request,response,state) {
var title = decodeURIComponent(state.params[0]),
tiddler = state.wiki.getTiddler(title),
tiddlerFields = {},
knownFields = [
"bag", "created", "creator", "modified", "modifier", "permissions", "recipe", "revision", "tags", "text", "title", "type", "uri"
];
if(tiddler) {
$tw.utils.each(tiddler.fields,function(field,name) {
var value = tiddler.getFieldString(name);
if(knownFields.indexOf(name) !== -1) {
tiddlerFields[name] = value;
} else {
tiddlerFields.fields = tiddlerFields.fields || {};
tiddlerFields.fields[name] = value;
}
});
tiddlerFields.revision = state.wiki.getChangeCount(title);
tiddlerFields.type = tiddlerFields.type || "text/vnd.tiddlywiki";
response.writeHead(200, {"Content-Type": "application/json"});
response.end(JSON.stringify(tiddlerFields),"utf8");
} else {
response.writeHead(404);
response.end();
}
}
});
};
Command.prototype.execute = function() {
if(!$tw.boot.wikiTiddlersPath) {
$tw.utils.warning("Warning: Wiki folder '" + $tw.boot.wikiPath + "' does not exist or is missing a tiddlywiki.info file");
}
var port = this.params[0] || "8080",
rootTiddler = this.params[1] || "$:/core/save/all",
renderType = this.params[2] || "text/plain",
serveType = this.params[3] || "text/html",
username = this.params[4],
password = this.params[5],
host = this.params[6] || "127.0.0.1",
pathprefix = this.params[7],
debugLevel = this.params[8] || "none";
if(parseInt(port,10).toString() !== port) {
port = process.env[port] || 8080;
}
this.server.set({
rootTiddler: rootTiddler,
renderType: renderType,
serveType: serveType,
username: username,
password: password,
pathprefix: pathprefix,
debugLevel: debugLevel
// Set up server
this.server = new Server({
wiki: this.commander.wiki,
variables: {
port: this.params[0],
host: this.params[6],
"root-tiddler": this.params[1],
"root-render-type": this.params[2],
"root-serve-type": this.params[3],
username: this.params[4],
password: this.params[5],
"path-prefix": this.params[7],
"debug-level": this.params[8]
}
});
var nodeServer = this.server.listen(port,host);
$tw.utils.log("Serving on " + host + ":" + port,"brown/orange");
$tw.utils.log("(press ctrl-C to exit)","red");
// Warn if required plugins are missing
if(!$tw.wiki.getTiddler("$:/plugins/tiddlywiki/tiddlyweb") || !$tw.wiki.getTiddler("$:/plugins/tiddlywiki/filesystem")) {
$tw.utils.warning("Warning: Plugins required for client-server operation (\"tiddlywiki/filesystem\" and \"tiddlywiki/tiddlyweb\") are missing from tiddlywiki.info file");
}
$tw.hooks.invokeHook('th-server-command-post-start', this.server, nodeServer);
var nodeServer = this.server.listen();
$tw.hooks.invokeHook("th-server-command-post-start",this.server,nodeServer);
return null;
};
@@ -0,0 +1,94 @@
/*\
title: $:/core/modules/server/authenticators/basic.js
type: application/javascript
module-type: authenticator
Authenticator for WWW basic authentication
\*/
(function(){
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
if($tw.node) {
var util = require("util"),
fs = require("fs"),
url = require("url"),
path = require("path");
}
function BasicAuthenticator(server) {
this.server = server;
this.credentialsData = [];
}
/*
Returns true if the authenticator is active, false if it is inactive, or a string if there is an error
*/
BasicAuthenticator.prototype.init = function() {
// Read the credentials data
this.credentialsFilepath = this.server.get("credentials");
if(this.credentialsFilepath) {
var resolveCredentialsFilepath = path.resolve($tw.boot.wikiPath,this.credentialsFilepath);
if(fs.existsSync(resolveCredentialsFilepath) && !fs.statSync(resolveCredentialsFilepath).isDirectory()) {
var credentialsText = fs.readFileSync(resolveCredentialsFilepath,"utf8"),
credentialsData = $tw.utils.parseCsvStringWithHeader(credentialsText);
if(typeof credentialsData === "string") {
return "Error: " + credentialsData + " reading credentials from '" + resolveCredentialsFilepath + "'";
} else {
this.credentialsData = credentialsData;
}
} else {
return "Error: Unable to load user credentials from '" + credentialsFilepath + "'";
}
}
// Add the hardcoded username and password if specified
if(this.server.get("username") && this.server.get("password")) {
this.credentialsData = this.credentialsData || [];
this.credentialsData.push({
username: this.server.get("username"),
password: this.server.get("password")
});
}
return this.credentialsData.length > 0;
};
/*
Returns true if the request is authenticated and assigns the "authenticatedUsername" state variable.
Returns false if the request couldn't be authenticated having sent an appropriate response to the browser
*/
BasicAuthenticator.prototype.authenticateRequest = function(request,response,state) {
// Extract the incoming username and password from the request
var header = request.headers.authorization || "";
if(!header && state.allowAnon) {
// If there's no header and anonymous access is allowed then we don't set authenticatedUsername
return true;
}
var token = header.split(/\s+/).pop() || "",
auth = $tw.utils.base64Decode(token),
parts = auth.split(/:/),
incomingUsername = parts[0],
incomingPassword = parts[1];
// Check that at least one of the credentials matches
var matchingCredentials = this.credentialsData.find(function(credential) {
return credential.username === incomingUsername && credential.password === incomingPassword;
});
if(matchingCredentials) {
// If so, add the authenticated username to the request state
state.authenticatedUsername = incomingUsername;
return true;
} else {
// If not, return an authentication challenge
response.writeHead(401,"Authentication required",{
"WWW-Authenticate": 'Basic realm="Please provide your username and password to login to ' + state.server.servername + '"'
});
response.end();
return false;
}
};
exports.AuthenticatorClass = BasicAuthenticator;
})();
@@ -0,0 +1,47 @@
/*\
title: $:/core/modules/server/authenticators/header.js
type: application/javascript
module-type: authenticator
Authenticator for trusted header authentication
\*/
(function(){
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
function HeaderAuthenticator(server) {
this.server = server;
this.header = server.get("authenticated-user-header");
}
/*
Returns true if the authenticator is active, false if it is inactive, or a string if there is an error
*/
HeaderAuthenticator.prototype.init = function() {
return !!this.header;
};
/*
Returns true if the request is authenticated and assigns the "authenticatedUsername" state variable.
Returns false if the request couldn't be authenticated having sent an appropriate response to the browser
*/
HeaderAuthenticator.prototype.authenticateRequest = function(request,response,state) {
// Otherwise, authenticate as the username in the specified header
var username = request.headers[this.header];
if(!username && !state.allowAnon) {
response.writeHead(401,"Authorization header required to login to '" + state.server.servername + "'");
response.end();
return false;
} else {
// authenticatedUsername will be undefined for anonymous users
state.authenticatedUsername = username;
return true;
}
};
exports.AuthenticatorClass = HeaderAuthenticator;
})();
@@ -0,0 +1,28 @@
/*\
title: $:/core/modules/server/routes/delete-tiddler.js
type: application/javascript
module-type: route
DELETE /recipes/default/tiddlers/:title
\*/
(function() {
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
exports.method = "DELETE";
exports.path = /^\/bags\/default\/tiddlers\/(.+)$/;
exports.handler = function(request,response,state) {
var title = decodeURIComponent(state.params[0]);
state.wiki.deleteTiddler(title);
response.writeHead(204, "OK", {
"Content-Type": "text/plain"
});
response.end();
};
}());
+25
View File
@@ -0,0 +1,25 @@
/*\
title: $:/core/modules/server/routes/get-favicon.js
type: application/javascript
module-type: route
GET /favicon.ico
\*/
(function() {
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
exports.method = "GET";
exports.path = /^\/favicon.ico$/;
exports.handler = function(request,response,state) {
response.writeHead(200, {"Content-Type": "image/x-icon"});
var buffer = state.wiki.getTiddlerText("$:/favicon.ico","");
response.end(buffer,"base64");
};
}());
+50
View File
@@ -0,0 +1,50 @@
/*\
title: $:/core/modules/server/routes/get-file.js
type: application/javascript
module-type: route
GET /files/:filepath
\*/
(function() {
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
exports.method = "GET";
exports.path = /^\/files\/(.+)$/;
exports.handler = function(request,response,state) {
var path = require("path"),
fs = require("fs"),
util = require("util");
var filename = path.resolve($tw.boot.wikiPath,"files",decodeURIComponent(state.params[0])),
extension = path.extname(filename);
fs.readFile(filename,function(err,content) {
var status,content,type = "text/plain";
if(err) {
if(err.code === "ENOENT") {
status = 404;
content = "File '" + filename + "' not found";
} else if(err.code === "EACCES") {
status = 403;
content = "You do not have permission to access the file '" + filename + "'";
} else {
status = 500;
content = err.toString();
}
} else {
status = 200;
content = content;
type = $tw.config.fileExtensionInfo[extension] || "application/octet-stream";
}
response.writeHead(status,{
"Content-Type": type
});
response.end(content);
});
};
}());
+25
View File
@@ -0,0 +1,25 @@
/*\
title: $:/core/modules/server/routes/get-index.js
type: application/javascript
module-type: route
GET /
\*/
(function() {
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
exports.method = "GET";
exports.path = /^\/$/;
exports.handler = function(request,response,state) {
response.writeHead(200, {"Content-Type": state.server.get("root-serve-type")});
var text = state.wiki.renderTiddler(state.server.get("root-render-type"),state.server.get("root-tiddler"));
response.end(text,"utf8");
};
}());
@@ -0,0 +1,35 @@
/*\
title: $:/core/modules/server/routes/get-login-basic.js
type: application/javascript
module-type: route
GET /login-basic -- force a Basic Authentication challenge
\*/
(function() {
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
exports.method = "GET";
exports.path = /^\/login-basic$/;
exports.handler = function(request,response,state) {
if(!state.authenticatedUsername) {
// Challenge if there's no username
response.writeHead(401,{
"WWW-Authenticate": 'Basic realm="Please provide your username and password to login to ' + state.server.servername + '"'
});
response.end();
} else {
// Redirect to the root wiki if login worked
response.writeHead(302,{
Location: "/"
});
response.end();
}
};
}());
+33
View File
@@ -0,0 +1,33 @@
/*\
title: $:/core/modules/server/routes/get-status.js
type: application/javascript
module-type: route
GET /status
\*/
(function() {
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
exports.method = "GET";
exports.path = /^\/status$/;
exports.handler = function(request,response,state) {
response.writeHead(200, {"Content-Type": "application/json"});
var text = JSON.stringify({
username: state.authenticatedUsername || state.server.get("anon-username") || "",
anonymous: !state.authenticatedUsername,
read_only: !state.server.isAuthorized("writers",state.authenticatedUsername),
space: {
recipe: "default"
},
tiddlywiki_version: $tw.version
});
response.end(text,"utf8");
};
}());
@@ -0,0 +1,42 @@
/*\
title: $:/core/modules/server/routes/get-tiddler-html.js
type: application/javascript
module-type: route
GET /:title
\*/
(function() {
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
exports.method = "GET";
exports.path = /^\/([^\/]+)$/;
exports.handler = function(request,response,state) {
var title = decodeURIComponent(state.params[0]),
tiddler = state.wiki.getTiddler(title);
if(tiddler) {
var renderType,template;
// Render ordinary tiddlers as HTML, and system tiddlers in plain text
if(state.wiki.isSystemTiddler(title)) {
renderType = state.server.get("system-tiddler-render-type");
template = state.server.get("system-tiddler-template");
} else {
renderType = state.server.get("tiddler-render-type");
template = state.server.get("tiddler-template");
}
var text = state.wiki.renderTiddler(renderType,template,{parseAsInline: true, variables: {currentTiddler: title}});
// Naughty not to set a content-type, but it's the easiest way to ensure the browser will see HTML pages as HTML, and accept plain text tiddlers as CSS or JS
response.writeHead(200);
response.end(text,"utf8");
} else {
response.writeHead(404);
response.end();
}
};
}());
+46
View File
@@ -0,0 +1,46 @@
/*\
title: $:/core/modules/server/routes/get-tiddler.js
type: application/javascript
module-type: route
GET /recipes/default/tiddlers/:title
\*/
(function() {
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
exports.method = "GET";
exports.path = /^\/recipes\/default\/tiddlers\/(.+)$/;
exports.handler = function(request,response,state) {
var title = decodeURIComponent(state.params[0]),
tiddler = state.wiki.getTiddler(title),
tiddlerFields = {},
knownFields = [
"bag", "created", "creator", "modified", "modifier", "permissions", "recipe", "revision", "tags", "text", "title", "type", "uri"
];
if(tiddler) {
$tw.utils.each(tiddler.fields,function(field,name) {
var value = tiddler.getFieldString(name);
if(knownFields.indexOf(name) !== -1) {
tiddlerFields[name] = value;
} else {
tiddlerFields.fields = tiddlerFields.fields || {};
tiddlerFields.fields[name] = value;
}
});
tiddlerFields.revision = state.wiki.getChangeCount(title);
tiddlerFields.type = tiddlerFields.type || "text/vnd.tiddlywiki";
response.writeHead(200, {"Content-Type": "application/json"});
response.end(JSON.stringify(tiddlerFields),"utf8");
} else {
response.writeHead(404);
response.end();
}
};
}());
@@ -0,0 +1,37 @@
/*\
title: $:/core/modules/server/routes/get-tiddlers-json.js
type: application/javascript
module-type: route
GET /recipes/default/tiddlers/tiddlers.json
\*/
(function() {
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
exports.method = "GET";
exports.path = /^\/recipes\/default\/tiddlers.json$/;
exports.handler = function(request,response,state) {
response.writeHead(200, {"Content-Type": "application/json"});
var tiddlers = [];
state.wiki.forEachTiddler({sortField: "title"},function(title,tiddler) {
var tiddlerFields = {};
$tw.utils.each(tiddler.fields,function(field,name) {
if(name !== "text") {
tiddlerFields[name] = tiddler.getFieldString(name);
}
});
tiddlerFields.revision = state.wiki.getChangeCount(title);
tiddlerFields.type = tiddlerFields.type || "text/vnd.tiddlywiki";
tiddlers.push(tiddlerFields);
});
var text = JSON.stringify(tiddlers);
response.end(text,"utf8");
};
}());
+42
View File
@@ -0,0 +1,42 @@
/*\
title: $:/core/modules/server/routes/put-tiddler.js
type: application/javascript
module-type: route
PUT /recipes/default/tiddlers/:title
\*/
(function() {
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
exports.method = "PUT";
exports.path = /^\/recipes\/default\/tiddlers\/(.+)$/;
exports.handler = function(request,response,state) {
var title = decodeURIComponent(state.params[0]),
fields = JSON.parse(state.data);
// Pull up any subfields in the `fields` object
if(fields.fields) {
$tw.utils.each(fields.fields,function(field,name) {
fields[name] = field;
});
delete fields.fields;
}
// Remove any revision field
if(fields.revision) {
delete fields.revision;
}
state.wiki.addTiddler(new $tw.Tiddler(state.wiki.getCreationFields(),fields,{title: title},state.wiki.getModificationFields()));
var changeCount = state.wiki.getChangeCount(title).toString();
response.writeHead(204, "OK",{
Etag: "\"default/" + encodeURIComponent(title) + "/" + changeCount + ":\"",
"Content-Type": "text/plain"
});
response.end();
};
}());
+252
View File
@@ -0,0 +1,252 @@
/*\
title: $:/core/modules/server/server.js
type: application/javascript
module-type: library
Serve tiddlers over http
\*/
(function(){
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
if($tw.node) {
var util = require("util"),
fs = require("fs"),
url = require("url"),
path = require("path");
}
/*
A simple HTTP server with regexp-based routes
options: variables - optional hashmap of variables to set (a misnomer - they are really constant parameters)
routes - optional array of routes to use
wiki - reference to wiki object
*/
function Server(options) {
var self = this;
this.routes = options.routes || [];
this.authenticators = options.authenticators || [];
this.wiki = options.wiki;
this.servername = this.wiki.getTiddlerText("$:/SiteTitle") || "TiddlyWiki5";
// Initialise the variables
this.variables = $tw.utils.extend({},this.defaultVariables);
if(options.variables) {
for(var variable in options.variables) {
if(options.variables[variable]) {
this.variables[variable] = options.variables[variable];
}
}
}
$tw.utils.extend({},this.defaultVariables,options.variables);
// Initialise CSRF
this.csrfDisable = this.get("csrf-disable") === "yes";
// Initialise authorization
var authorizedUserName = (this.get("username") && this.get("password")) ? this.get("username") : "(anon)";
this.authorizationPrincipals = {
readers: (this.get("readers") || authorizedUserName).split(",").map($tw.utils.trim),
writers: (this.get("writers") || authorizedUserName).split(",").map($tw.utils.trim)
}
// Load and initialise authenticators
$tw.modules.forEachModuleOfType("authenticator", function(title,authenticatorDefinition) {
// console.log("Loading server route " + title);
self.addAuthenticator(authenticatorDefinition.AuthenticatorClass);
});
// Load route handlers
$tw.modules.forEachModuleOfType("route", function(title,routeDefinition) {
// console.log("Loading server route " + title);
self.addRoute(routeDefinition);
});
// Initialise the http vs https
this.listenOptions = null;
this.protocol = "http";
var tlsKeyFilepath = this.get("tls-key"),
tlsCertFilepath = this.get("tls-cert");
if(tlsCertFilepath && tlsKeyFilepath) {
this.listenOptions = {
key: fs.readFileSync(path.resolve($tw.boot.wikiPath,tlsKeyFilepath),"utf8"),
cert: fs.readFileSync(path.resolve($tw.boot.wikiPath,tlsCertFilepath),"utf8")
};
this.protocol = "https";
}
this.transport = require(this.protocol);
}
Server.prototype.defaultVariables = {
port: "8080",
host: "127.0.0.1",
"root-tiddler": "$:/core/save/all",
"root-render-type": "text/plain",
"root-serve-type": "text/html",
"tiddler-render-type": "text/html",
"tiddler-template": "$:/core/templates/server/static.tiddler.html",
"system-tiddler-render-type": "text/plain",
"system-tiddler-template": "$:/core/templates/wikified-tiddler",
"debug-level": "none"
};
Server.prototype.get = function(name) {
return this.variables[name];
};
Server.prototype.addRoute = function(route) {
this.routes.push(route);
};
Server.prototype.addAuthenticator = function(AuthenticatorClass) {
// Instantiate and initialise the authenticator
var authenticator = new AuthenticatorClass(this),
result = authenticator.init();
if(typeof result === "string") {
$tw.utils.error("Error: " + result);
} else if(result) {
// Only use the authenticator if it initialised successfully
this.authenticators.push(authenticator);
}
};
Server.prototype.findMatchingRoute = function(request,state) {
var pathprefix = this.get("path-prefix") || "";
for(var t=0; t<this.routes.length; t++) {
var potentialRoute = this.routes[t],
pathRegExp = potentialRoute.path,
pathname = state.urlInfo.pathname,
match;
if(pathprefix) {
if(pathname.substr(0,pathprefix.length) === pathprefix) {
pathname = pathname.substr(pathprefix.length) || "/";
match = potentialRoute.path.exec(pathname);
} else {
match = false;
}
} else {
match = potentialRoute.path.exec(pathname);
}
if(match && request.method === potentialRoute.method) {
state.params = [];
for(var p=1; p<match.length; p++) {
state.params.push(match[p]);
}
return potentialRoute;
}
}
return null;
};
Server.prototype.methodMappings = {
"GET": "readers",
"OPTIONS": "readers",
"HEAD": "readers",
"PUT": "writers",
"POST": "writers",
"DELETE": "writers"
};
/*
Check whether a given user is authorized for the specified authorizationType ("readers" or "writers"). Pass null or undefined as the username to check for anonymous access
*/
Server.prototype.isAuthorized = function(authorizationType,username) {
var principals = this.authorizationPrincipals[authorizationType] || [];
return principals.indexOf("(anon)") !== -1 || (username && (principals.indexOf("(authenticated)") !== -1 || principals.indexOf(username) !== -1));
}
Server.prototype.requestHandler = function(request,response) {
// Compose the state object
var self = this;
var state = {};
state.wiki = self.wiki;
state.server = self;
state.urlInfo = url.parse(request.url);
// Get the principals authorized to access this resource
var authorizationType = this.methodMappings[request.method] || "readers";
// Check for the CSRF header if this is a write
if(!this.csrfDisable && authorizationType === "writers" && request.headers["x-requested-with"] !== "TiddlyWiki") {
response.writeHead(403,"'X-Requested-With' header required to login to '" + this.servername + "'");
response.end();
return;
}
// Check whether anonymous access is granted
state.allowAnon = this.isAuthorized(authorizationType,null);
// Authenticate with the first active authenticator
if(this.authenticators.length > 0) {
if(!this.authenticators[0].authenticateRequest(request,response,state)) {
// Bail if we failed (the authenticator will have sent the response)
return;
}
}
// Authorize with the authenticated username
if(!this.isAuthorized(authorizationType,state.authenticatedUsername)) {
response.writeHead(401,"'" + state.authenticatedUsername + "' is not authorized to access '" + this.servername + "'");
response.end();
return;
}
// Find the route that matches this path
var route = self.findMatchingRoute(request,state);
// Optionally output debug info
if(self.get("debug-level") !== "none") {
console.log("Request path:",JSON.stringify(state.urlInfo));
console.log("Request headers:",JSON.stringify(request.headers));
console.log("authenticatedUsername:",state.authenticatedUsername);
}
// Return a 404 if we didn't find a route
if(!route) {
response.writeHead(404);
response.end();
return;
}
// Set the encoding for the incoming request
// TODO: Presumably this would need tweaking if we supported PUTting binary tiddlers
request.setEncoding("utf8");
// Dispatch the appropriate method
switch(request.method) {
case "GET": // Intentional fall-through
case "DELETE":
route.handler(request,response,state);
break;
case "PUT":
var data = "";
request.on("data",function(chunk) {
data += chunk.toString();
});
request.on("end",function() {
state.data = data;
route.handler(request,response,state);
});
break;
}
};
/*
Listen for requests
port: optional port number (falls back to value of "port" variable)
host: optional host address (falls back to value of "hist" variable)
*/
Server.prototype.listen = function(port,host) {
// Handle defaults for port and host
port = port || this.get("port");
host = host || this.get("host");
// Check for the port being a string and look it up as an environment variable
if(parseInt(port,10).toString() !== port) {
port = process.env[port] || 8080;
}
$tw.utils.log("Serving on " + this.protocol + "://" + host + ":" + port,"brown/orange");
$tw.utils.log("(press ctrl-C to exit)","red");
// Warn if required plugins are missing
if(!$tw.wiki.getTiddler("$:/plugins/tiddlywiki/tiddlyweb") || !$tw.wiki.getTiddler("$:/plugins/tiddlywiki/filesystem")) {
$tw.utils.warning("Warning: Plugins required for client-server operation (\"tiddlywiki/filesystem\" and \"tiddlywiki/tiddlyweb\") are missing from tiddlywiki.info file");
}
// Listen
var server;
if(this.listenOptions) {
server = this.transport.createServer(this.listenOptions,this.requestHandler.bind(this));
} else {
server = this.transport.createServer(this.requestHandler.bind(this));
}
return server.listen(port,host);
};
exports.Server = Server;
})();
+5 -1
View File
@@ -16,6 +16,8 @@ The syncer tracks changes to the store. If a syncadaptor is used then individual
Defaults
*/
Syncer.prototype.titleIsLoggedIn = "$:/status/IsLoggedIn";
Syncer.prototype.titleIsAnonymous = "$:/status/IsAnonymous";
Syncer.prototype.titleIsReadOnly = "$:/status/IsReadOnly";
Syncer.prototype.titleUserName = "$:/status/UserName";
Syncer.prototype.titleSyncFilter = "$:/config/SyncFilter";
Syncer.prototype.titleSavedNotification = "$:/language/Notifications/Save/Done";
@@ -169,12 +171,14 @@ Syncer.prototype.getStatus = function(callback) {
// Mark us as not logged in
this.wiki.addTiddler({title: this.titleIsLoggedIn,text: "no"});
// Get login status
this.syncadaptor.getStatus(function(err,isLoggedIn,username) {
this.syncadaptor.getStatus(function(err,isLoggedIn,username,isReadOnly,isAnonymous) {
if(err) {
self.logger.alert(err);
return;
}
// Set the various status tiddlers
self.wiki.addTiddler({title: self.titleIsReadOnly,text: isReadOnly ? "yes" : "no"});
self.wiki.addTiddler({title: self.titleIsAnonymous,text: isAnonymous ? "yes" : "no"});
self.wiki.addTiddler({title: self.titleIsLoggedIn,text: isLoggedIn ? "yes" : "no"});
if(isLoggedIn) {
self.wiki.addTiddler({title: self.titleUserName,text: username || ""});
+46
View File
@@ -0,0 +1,46 @@
/*\
title: $:/core/modules/utils/csv.js
type: application/javascript
module-type: utils
A barebones CSV parser
\*/
(function(){
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
/*
Parse a CSV string with a header row and return an array of hashmaps.
*/
exports.parseCsvStringWithHeader = function(text,options) {
options = options || {};
var separator = options.separator || ",",
rows = text.split(/\r?\n/mg).map(function(row) {
return $tw.utils.trim(row);
}).filter(function(row) {
return row !== "";
});
if(rows.length < 1) {
return "Missing header row";
}
var headings = rows[0].split(separator),
results = [];
for(var row=1; row<rows.length; row++) {
var columns = rows[row].split(separator),
columnResult = Object.create(null);
if(columns.length !== headings.length) {
return "Malformed CSV row '" + rows[row] + "'";
}
for(var column=0; column<columns.length; column++) {
var columnName = headings[column];
columnResult[columnName] = $tw.utils.trim(columns[column] || "");
}
results.push(columnResult);
}
return results;
}
})();
+3
View File
@@ -60,6 +60,9 @@ exports.httpRequest = function(options) {
if(data && !$tw.utils.hop(headers,"Content-type")) {
request.setRequestHeader("Content-type","application/x-www-form-urlencoded; charset=UTF-8");
}
if(!$tw.utils.hop(headers,"X-Requested-With")) {
request.setRequestHeader("X-Requested-With","TiddlyWiki");
}
try {
request.send(data);
} catch(e) {
@@ -0,0 +1,30 @@
title: $:/core/templates/server/static.sidebar.wikitext
\whitespace trim
<div class="tc-sidebar-scrollable" style="overflow: auto;">
<div class="tc-sidebar-header">
<h1 class="tc-site-title">
<$transclude tiddler="$:/SiteTitle"/>
</h1>
<div class="tc-site-subtitle">
<$transclude tiddler="$:/SiteSubtitle"/>
</div>
<h2>
</h2>
<div class="tc-sidebar-lists">
<$list filter={{$:/DefaultTiddlers}}>
<div class="tc-menu-list-subitem">
<$link><$text text=<<currentTiddler>>/></$link>
</div>
</$list>
</div>
<!-- Currently disabled the recent list as it is unweildy when the responsive narrow view kicks in
<h2>
{{$:/language/SideBar/Recent/Caption}}
</h2>
<div class="tc-sidebar-lists">
<$macrocall $name="timeline" format={{$:/language/RecentChanges/DateFormat}}/>
</div>
</div>
</div>
-->
@@ -0,0 +1,29 @@
title: $:/core/templates/server/static.tiddler.html
\whitespace trim
\define tv-wikilink-template() $uri_encoded$
<$importvariables filter="[[$:/core/ui/PageMacros]] [all[shadows+tiddlers]tag[$:/tags/Macro]!has[draft.of]]">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<meta name="generator" content="TiddlyWiki" />
<meta name="tiddlywiki-version" content={{$:/core/templates/version}} />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="apple-mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
<meta name="mobile-web-app-capable" content="yes"/>
<meta name="format-detection" content="telephone=no">
<link id="faviconLink" rel="shortcut icon" href="favicon.ico">
<link rel="stylesheet" href="%24%3A%2Fcore%2Ftemplates%2Fstatic.template.css">
<title><$view field="caption" format="plainwikified"><$view field="title"/></$view>: <$view tiddler="$:/core/wiki/title" format="plainwikified"/></title>
</head>
<body class="tc-body">
<$transclude tiddler="$:/core/templates/server/static.sidebar.wikitext" mode="inline"/>
<section class="tc-story-river">
<div class="tc-tiddler-frame">
<$transclude tiddler="$:/core/templates/server/static.tiddler.wikitext" mode="inline"/>
</div>
</section>
</body>
</html>
</$importvariables>
@@ -0,0 +1,23 @@
title: $:/core/templates/server/static.tiddler.wikitext
\whitespace trim
<div class="tc-tiddler-title">
<div class="tc-titlebar">
<h2><$text text=<<currentTiddler>>/></h2>
</div>
</div>
<div class="tc-subtitle">
<$link to={{!!modifier}}>
<$view field="modifier"/>
</$link> <$view field="modified" format="date" template={{$:/language/Tiddler/DateFormat}}/>
</div>
<div class="tc-tags-wrapper">
<$list filter="[all[current]tags[]sort[title]]">
<a href={{{ [<currentTiddler>encodeuricomponent[]] }}}>
<$macrocall $name="tag-pill" tag=<<currentTiddler>>/>
</a>
</$list>
</div>
<div class="tc-tiddler-body">
<$transclude mode="block"/>
</div>