Overview
The heart of
TiddlyWiki can be seen as an extensible representation transformation engine. Given the text of a tiddler and its associated MIME type, the engine can produce a rendering of the tiddler in a new MIME type. Furthermore, it can efficiently selectively update the rendering to track any changes in the tiddler or its dependents.
The most important transformations are from
text/x-tiddlywiki wikitext into
text/html or
text/plain but the engine is used throughout the system for other transformations, such as converting images for display in HTML, sanitising fragments of
JavaScript, and processing CSS.
The key feature of wikitext is the ability to include one tiddler within another (usually referred to as
transclusion). For example, one could have a tiddler called
Disclaimer that contains the boilerplate of a legal disclaimer, and then include it within lots of different tiddlers with the macro call
<<tiddler Disclaimer>>. This simple feature brings great power in terms of encapsulating and reusing content, and evolving a clean, usable implementation architecture to support it efficiently is a key objective of the
TiddlyWiki5 design.
It turns out that the transclusion capability combined with the selective refreshing mechanism provides a good foundation for building
TiddlyWiki's user interface itself. Consider, for example, the
StoryMacro in its simplest form:
<<story story:MyStoryTiddler>>
The story macro looks for a list of tiddler titles in the tiddler
MyStoryTiddler, and displays them in sequence. The subtle part is that subsequently, if
MyStoryTiddler changes, the
<<story>> macro is selectively re-rendered. So, to navigate to a new tiddler, code merely needs to add the name of the tiddler and a line break to the top of
MyStoryTiddler:
var storyTiddler = store.getTiddler("MyStoryTiddler");
store.addTiddler(new Tiddler(storyTiddler,{text: navigateTo + "\n" + storyTiddler.text}));
The mechanisms that allow all of this to work are fairly intricate. The sections below progressively build the key architectural concepts of
TiddlyWiki5 in a way that should provide a good basis for exploring the code directly.
Tiddlers
Tiddlers are an immutable dictionary of name:value pairs called fields.
The only field that is required is the
title field, but useful tiddlers also have a
text field, and some or all of the standard fields
modified,
modifier,
created,
creator,
tags and
type.
Hardcoded in the system is the knowledge that the
tags field is a string array, and that the
modified and
created fields are
JavaScript Date objects. All other fields are strings.
The
type field identifies the representation of the tiddler text with a MIME type.
WikiStore
Groups of uniquely titled tiddlers are contained in
WikiStore objects.
The
WikiStore also manages the plugin modules used for macros, and operations like serializing, deserializing, parsing and rendering tiddlers.
Each
WikiStore is connected to another shadow store that is used to provide default content. Under usual circumstances, when an attempt is made to retrieve a tiddler that doesn't exist in the store, the search continues into its shadow store (and so on, if the shadow store itself has a shadow store).
WikiStore Events
Clients can register event handlers with the
WikiStore object. Event handlers can be registered to be triggered for modifications to any tiddler in the store, or with a filter to only be invoked when a particular tiddler or set of tiddlers changes.
Whenever a change is made to a tiddler, the wikistore registers a
nexttick handler (if it hasn't already done so). The
nexttick handler looks back at all the tiddler changes, and dispatches any matching event handlers.
Parsing and Rendering
TiddlyWiki parses the content of tiddlers to build an internal tree representation that is used for several purposes:
- Rendering a tiddler to other formats (e.g. converting wikitext to HTML)
- Detecting outgoing links from a tiddler, and from them...
- ...computing incoming links to a tiddler
- Detecting tiddlers that are orphans with no incoming links
- Detecting tiddlers that are referred to but missing
The parse tree is built when needed, and then cached by the
WikiStore until the tiddler changes.
TiddlyWiki5 uses multiple parsers:
- Wikitext (
text/x-tiddlywiki) in js/WikiTextParser.js - JavaScript (
text/javascript) in js/JavaScriptParser.js - Images (
image/png and image/jpg) in js/ImageParser.js - JSON (
application/json) in js/JSONParser.js
Additional parsers are planned:
- CSS (
text/css) - Recipe (
text/x-tiddlywiki-recipe)
One global instance of each parser is instantiated in
js/App.js and registered with the main
WikiStore object.
The parsers are all used the same way:
The parse tree object exposes the following fields:
The dependencies are returned as an object like this:
{
tiddlers: {"tiddlertitle1": true, "tiddlertitle2": false},
dependentAll: false
}
The
tiddlers field is a hashmap of the title of each tiddler that is linked or included in the current one. The value is
true if the tiddler is a
'fat' dependency (ie the text is included in some way) or
false if the tiddler is a
skinny dependency.
The
dependentAll field is used to indicate that the tiddler contains a macro that scans the entire pool of tiddlers (for example the
<<list>> macro), and is potentially dependent on any of them. The effect is that the tiddler should be rerendered whenever any other tiddler changes.
Rendering
The
parseTree.compile(type) method returns a renderer object that contains a
JavaScript function that generates the new representation of the original parsed text.
The renderer is invoked as follows:
The
tiddler parameter to the
render method identifies the tiddler that is acting as the context for this rendering — for example, it provides the fields displayed by the
<<view>> macro. The
store parameter is used to resolve any references to other tiddlers.
Rerendering
When rendering to the HTML/SVG DOM in the browser,
TiddlyWiki5 also allows a previous rendering to be selectively updated in response to changes in dependent tiddlers. At the moment, only the
WikiTextRenderer supports rerendering.
The rerender method on the renderer is called as follows:
var node = document.getElementById("myNode");
var renderer = parseTree.compile("text/html");
myNode.innerHTML = renderer.render(tiddler,store);
// And then, later:
renderer.rerender(node,changes,tiddler,store,renderStep);
The parameters to
rerender() are:
| Name | Description |
|---|
| node | A reference to the DOM node containing the rendering to be rerendered |
| changes | A hashmap of {title: "created|modified|deleted"} indicating which tiddlers have changed since the original rendering |
| tiddler | The tiddler providing the rendering context |
| store | The store to use for resolving references to other tiddlers |
| renderStep | See below |
Currently, the only macro that supports rerendering is the
<<story>> macro; all other macros are rerendered by calling the ordinary
render() method again. The reason that the
<<story>> macro goes to the trouble of having a
rerender() method is so that it can be carefully selective about not disturbing tiddlers in the DOM that aren't affected by the change. If there were, for instance, a video playing in one of the open tiddlers it would be reset to the beginning if the tiddler were rerendered.
Introduction
TiddlyWiki5 is based on a 500 line boot kernel that runs on node.js or in the browser, and everything
else is plugins.
The kernel boots just enough of the
TiddlyWiki environment to allow it to load tiddlers as plugins and execute them (a barebones tiddler class, a barebones wiki store class, some utilities etc.). Plugin modules are written like
node.js modules; you can use
require() to invoke sub components and to control load order.
There are several different types of plugins: parsers, serializers, deserializers, macros etc. It goes much further than you might expect. For example, individual tiddler fields are plugins, too: there's a plugin that knows how to handle the
tags field, and another that knows how to handle the special behaviour of
the
modified and
created fields.
Some plugins have further sub-plugins: the wikitext parser, for instance, accepts rules as individual plugins.
Plugins and Modules
In
TiddlyWiki5, a plugin is a bundle of related tiddlers that are distributed together as a single unit. Plugins can include tiddlers which are
JavaScript modules.
The file
core/boot.js is a barebones
TiddlyWiki kernel that is just sufficient to load the core plugin modules and trigger a startup plugin module to load up the rest of the application.
The kernel includes:
- Eight short shared utility functions
- Three methods implementing the plugin module mechanism
- The
$tw.Tiddler class (and three field definition plugins) - The
$tw.Wiki class (and three tiddler deserialization methods) - Code for the browser to load tiddlers from the HTML DOM
- Code for the server to load tiddlers from the file system
Each module is an ordinary
node.js-style module, using the
require() function to access other modules and the
exports global to return
JavaScript values. The boot kernel smooths over the differences between
node.js and the browser, allowing the same plugin modules to execute in both environments.
In the browser,
core/boot.js is packed into a template HTML file that contains the following elements in order:
- Ordinary and shadow tiddlers, packed as HTML
<DIV> elements -
core/bootprefix.js, containing a few lines to set up the plugin environment - Plugin JavaScript modules, packed as HTML
<SCRIPT> blocks -
core/boot.js, containing the boot kernel
On the server,
core/boot.js is executed directly. It uses the
node.js local file API to load plugins directly from the file system in the
core/modules directory. The code loading is performed synchronously for brevity (and because the system is in any case inherently blocked until plugins are loaded).
The boot kernel sets up the
$tw global variable that is used to store all the state data of the system.
Core
The 'core' is the boot kernel plus the set of plugin modules that it loads. It contains plugins of the following types:
-
tiddlerfield - defines the characteristics of tiddler fields of a particular name -
tiddlerdeserializer - methods to extract tiddlers from text representations or the DOM -
startup - functions to be called by the kernel after booting -
global - members of the $tw global -
config - values to be merged over the $tw.config global -
utils - general purpose utility functions residing in $tw.utils -
tiddlermethod - additional methods for the $tw.Tiddler class -
wikimethod - additional methods for the $tw.Wiki class -
treeutils - static utility methods for parser tree nodes -
treenode - classes of parser tree nodes -
macro - macro definitions -
editor - interactive editors for different types of content -
parser - parsers for different types of content -
wikitextrule - individual rules for the wikitext parser -
command - individual commands for the $tw.Commander class
TiddlyWiki5 makes extensive use of
JavaScript inheritance:
- Tree nodes defined in
$:/core/treenodes/ all inherit from $:/core/treenodes/node.js - Macros defined in
$:/core/macros/ all inherit from $:/core/treenodes/macro.js