<h1>Welcome to <ahref='TiddlyWiki5'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>TiddlyWiki5</a></h1>Welcome to <ahref='TiddlyWiki5'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>TiddlyWiki5</a>, a reboot of <ahref='TiddlyWiki'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlyWiki</a>, the venerable, reusable non-linear personal web notebook first released in 2004. It is a complete interactive wiki that can run from a single HTML file or as a powerful <ahref='What%20is%20node.js%3F'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>node.js application</a>.<br><br>Some useful tiddlers for feature testing:<br><ul><li><ahref='ClockTiddler'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>ClockTiddler</a> showing automatic refreshing of tiddlers</li><li><ahref='ImageTests'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>ImageTests</a> showing different ways of embedding images</li><li><ahref='SampleData'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>SampleData</a> showing how JSON tiddlers are handled</li><li><ahref='VideoTests'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>VideoTests</a> showing how different online video formats can be embedded</li></ul><br>Technical documentation includes:<br><ul><li><ahref='Testing'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>Testing</a> regimen</li><li> Details of the <ahref='CommandLineInterface'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>CommandLineInterface</a></li><li> Overview of <ahref='TiddlyWikiArchitecture'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlyWikiArchitecture</a></li><ul><li><ahref='MacroInternals'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>MacroInternals</a></li></ul><li> Information about <ahref='TiddlerFiles'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlerFiles</a> and <ahref='RecipeFiles'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>RecipeFiles</a></li><li><ahref='NewWikiTextFeatures'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>NewWikiTextFeatures</a></li></ul><br><ahref='TiddlyWiki5'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>TiddlyWiki5</a> is currently in early beta, which is to say that it is useful but incomplete. You can get involved in the development on <ahref='GitHub'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>GitHub</a> and the discussions on Google Groups:<br><br><ahref='https://github.com/Jermolene/TiddlyWiki5'class='tw-tiddlylink tw-tiddlylink-external'>https://github.com/Jermolene/TiddlyWiki5</a><br><ahref='http://groups.google.com/group/TiddlyWikiDev'class='tw-tiddlylink tw-tiddlylink-external'>http://groups.google.com/group/TiddlyWikiDev</a><br><br><blockquote>Bug: this link doesn't work:<br><ahref='HelloThere'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>HelloThere</a><br></blockquote><br><h1>Usage</h1><ahref='TiddlyWiki5'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>TiddlyWiki5</a> can be used on the command line to perform an extensive set of operations based on <ahref='RecipeFiles'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>RecipeFiles</a>, <ahref='TiddlerFiles'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlerFiles</a> and <ahref='TiddlyWikiFiles'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>TiddlyWikiFiles</a>.<br><br>Usage:<br><code>
</code><br>The command line options are processed sequentially from left to right. Processing pauses during long operations, like loading a <ahref='RecipeFiles'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>recipe file</a> and all the subrecipes and <ahref='TiddlerFiles'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>tiddlers</a> that it references, and then resumes with the next command line option in sequence. The following options are available:<br><tableclass='twtable'><tbody><trclass='evenRow'><tdalign='left'><code>--recipe <filepath></code></td><tdalign='left'>Loads a specfied <code>.recipe</code> file</td></tr><trclass='oddRow'><tdalign='left'><code>--load <filepath></code></td><tdalign='left'>Load additional tiddlers from 2.x.x <ahref='TiddlyWiki'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlyWiki</a> files (<code>.html</code>), <code>.tiddler</code>, <code>.tid</code>, <code>.json</code> or other files</td></tr><trclass='evenRow'><tdalign='left'><code>--savewiki <dirpath></code></td><tdalign='left'>Saves all the loaded tiddlers as a single file <ahref='TiddlyWiki'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlyWiki</a> called <code>index.html</code> and an RSS feed called <code>index.xml</code> in a new directory of the specified name</td></tr><trclass='oddRow'><tdalign='left'><code>--savetiddler <title><filename> [<type>]</code></td><tdalign='left'>Save an individual tiddler as a specified MIME type, defaults to <code>text/html</code></td></tr><trclass='evenRow'><tdalign='left'><code>--savetiddlers <outdir></code></td><tdalign='left'>Saves all the loaded tiddlers as <code>.tid</code> files in the specified directory</td></tr><trclass='oddRow'><tdalign='left'><code>--servewiki <port></code></td><tdalign='left'>Serve the cooked <ahref='TiddlyWiki'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlyWiki</a> over HTTP at <code>/</code></td></tr><trclass='evenRow'><tdalign='left'><code>--servetiddlers <port></code></td><tdalign='left'>Serve individual tiddlers over HTTP at <code>/tiddlertitle</code></td></tr><trclass='oddRow'><tdalign='left'><code>--wikitest <dir></code></td><tdalign='left'>Run wikification tests against the tiddlers in the given directory</td></tr><trclass='evenRow'><tdalign='left'><code>--dumpstore</code></td><tdalign='left'>Dump the <ahref='TiddlyWiki'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlyWiki</a> store in JSON format</td></tr><trclass='oddRow'><tdalign='left'><code>--dumprecipe</code></td><tdalign='left'>Dump the current recipe in JSON format</td></tr><trclass='evenRow'><tdalign='left'><code>--verbose</code></td><tdalign='left'>verbose output, useful for debugging</td></tr></tbody></table><h2> Examples</h2>This example loads the tiddlers from a <ahref='TiddlyWiki'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlyWiki</a> HTML file and makes them available over HTTP:<br><code>
</code><br>This example cooks a <ahref='TiddlyWiki'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlyWiki</a> from a recipe:<br><code>
</code><br>This example ginsus a <ahref='TiddlyWiki'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlyWiki</a> into its constituent tiddlers:<br><code>
</code><br><h2> Notes</h2><code>--servewiki</code> and <code>--servertiddlers</code> are for different purposes and should not be used together. The former is for <ahref='TiddlyWiki'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlyWiki</a> core developers who want to be able to edit the <ahref='TiddlyWiki'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlyWiki</a> source files in a text editor and view the results in the browser by clicking refresh; it is slow because it reloads all the <ahref='TiddlyWiki'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlyWiki</a><ahref='JavaScript'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>JavaScript</a> files each time the page is loaded. The latter is for experimenting with the new wikification engine.<br><br><code>--wikitest</code> looks for <code>*.tid</code> files in the specified folder. It then wikifies the tiddlers to both "text/plain" and "text/html" format and checks the results against the content of the <code>*.html</code> and <code>*.txt</code> files in the same directory.<br><h1>Testing</h1><code>test.sh</code> contains a simple test script that cooks the main tiddlywiki.com recipe and compares it with the results of the old build process (ie, running cook.rb and then opening the file in a browser and performing a 'save changes' operation). It also runs a series of wikifications tests that work off the data in <code>test/wikitests/</code>.<br><br><h1>Architecture</h1><h2> Overview</h2><br>The heart of <ahref='TiddlyWiki'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlyWiki</a> can be seen as an extensible representation transformation engine. Given the text of a tiddler and its associated MIME type, the engine can produce a rendering of the tiddler in a new MIME type. Furthermore, it can efficiently selectively update the rendering to track any changes in the tiddler or its dependents.<br><br>The most important transformations are from <code>text/x-tiddlywiki</code> wikitext into <code>text/html</code> or <code>text/plain</code> but the engine is used throughout the system for other transformations, such as converting images for display in HTML, sanitising fragments of <ahref='JavaScript'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>JavaScript</a>, and processing CSS.<br><br>The key feature of wikitext is the ability to include one tiddler within another (usually referred to as <em>transclusion</em>). For example, one could have a tiddler called <em>Disclaimer</em> that contains the boilerplate of a legal disclaimer, and then include it within lots of different tiddlers with the macro call <code><<tiddler Disclaimer>></code>. This simple feature brings great power in terms of encapsulating and reusing content, and evolving a clean, usable implementation architecture to support it efficiently is a key objective of the <ahref='TiddlyWiki5'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>TiddlyWiki5</a> design.<br><br>It turns out that the transclusion capability combined with the selective refreshing mechanism provides a good foundation for building <ahref='TiddlyWiki'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlyWiki</a>'s user interface itself. Consider, for example, the <ahref='StoryMacro'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>StoryMacro</a> in its simplest form:<br><pre><<story story:MyStoryTiddler>>
</pre>The story macro looks for a list of tiddler titles in the tiddler <code>MyStoryTiddler</code>, and displays them in sequence. The subtle part is that subsequently, if <code>MyStoryTiddler</code> changes, the <code><<story>></code> macro is selectively re-rendered. So, to navigate to a new tiddler, code merely needs to add the name of the tiddler and a line break to the top of <code>MyStoryTiddler</code>:<br><pre>var storyTiddler = store.getTiddler("MyStoryTiddler");
</pre>The mechanisms that allow all of this to work are fairly intricate. The sections below progressively build the key architectural concepts of <ahref='TiddlyWiki5'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>TiddlyWiki5</a> in a way that should provide a good basis for exploring the code directly.<br><h2> Tiddlers</h2>Tiddlers are an immutable dictionary of name:value pairs called fields.<br><br>The only field that is required is the <code>title</code> field, but useful tiddlers also have a <code>text</code> field, and some or all of the standard fields <code>modified</code>, <code>modifier</code>, <code>created</code>, <code>creator</code>, <code>tags</code> and <code>type</code>.<br><br>Hardcoded in the system is the knowledge that the <code>tags</code> field is a string array, and that the <code>modified</code> and <code>created</code> fields are <ahref='JavaScript'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>JavaScript</a><code>Date</code> objects. All other fields are strings.<br><br>The <code>type</code> field identifies the representation of the tiddler text with a MIME type.<br><h2> WikiStore</h2>Groups of uniquely titled tiddlers are contained in <ahref='WikiStore'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>WikiStore</a> objects.<br><br>The <ahref='WikiStore'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>WikiStore</a> also manages the plugin modules used for macros, and operations like serializing, deserializing, parsing and rendering tiddlers.<br><br>Each <ahref='WikiStore'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>WikiStore</a> is connected to another shadow store that is used to provide default content. Under usual circumstances, when an attempt is made to retrieve a tiddler that doesn't exist in the store, the search continues into its shadow store (and so on, if the shadow store itself has a shadow store).<br><h2> WikiStore Events</h2>Clients can register event handlers with the <ahref='WikiStore'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>WikiStore</a> object. Event handlers can be registered to be triggered for modifications to any tiddler in the store, or with a filter to only be invoked when a particular tiddler or set of tiddlers changes.<br><br>Whenever a change is made to a tiddler, the wikistore registers a <code>nexttick</code> handler (if it hasn't already done so). The <code>nexttick</code> handler looks back at all the tiddler changes, and dispatches any matching event handlers. <br><h2> Parsing and Compiling</h2><ahref='TiddlyWiki'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlyWiki</a> parses the content of tiddlers to build an internal tree representation that is used for several purposes:<br><ul><li> Rendering a tiddler to other formats (e.g. converting wikitext to HTML)</li><li> Detecting outgoing links from a tiddler, and from them...</li><li> ...computing incoming links to a tiddler</li><li> Detecting tiddlers that are orphans with no incoming links</li><li> Detecting tiddlers that are referred to but missing</li></ul>The parse tree is built when needed, and then cached by the <ahref='WikiStore'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>WikiStore</a> until the tiddler changes.<br><br><ahref='TiddlyWiki5'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>TiddlyWiki5</a> uses multiple parsers:<br><ul><li> Wikitext (<code>text/x-tiddlywiki</code>) in <code>js/WikiTextParser.js</code></li><li><ahref='JavaScript'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>JavaScript</a> (<code>text/javascript</code>) in <code>js/JavaScriptParser.js</code></li><li> Images (<code>image/png</code> and <code>image/jpg</code>) in <code>js/ImageParser.js</code></li><li> JSON (<code>application/json</code>) in <code>js/JSONParser.js</code></li></ul>Additional parsers are planned:<br><ul><li> CSS (<code>text/css</code>)</li><li> Recipe (<code>text/x-tiddlywiki-recipe</code
</pre>The parse tree object exposes the following fields:<br><pre>var renderer = parseTree.compile(type); // Compiles the parse tree into a renderer for the specified MIME type
console.log(parseTree.toString(type)); // Returns a readable string representation of the parse tree (either `text/html` or `text/plain`)
var dependencies = parseTree.dependencies; // Gets the dependencies of the parse tree (see below)
</pre>The dependencies are returned as an object like this:<br><pre>{
</pre>The <code>link</code> and <code>include</code> fields are hashmaps of the title of each tiddler that is linked or included in the current one. For the tiddler to be subsequently rendered correctly, the linked tiddlers must be present, at least in skinny form, and the included tiddlers must be fully loaded.<br><br>The <code>dependentAll</code> field is used to indicate that the tiddler contains a macro that scans the entire pool of tiddlers (for example the <code><<list>></code> macro), and is potentially dependent on any of them. The effect is that the tiddler should be rerendered whenever any other tiddler changes.<br><h2> Rendering</h2>The <code>parseTree.compile(type)</code> method returns a renderer object that contains a <ahref='JavaScript'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>JavaScript</a> function that generates the new representation of the original parsed text.<br><br>The renderer is invoked as follows:<br><pre>var renderer = parseTree.compile("text/html");
</pre>The <code>tiddler</code> parameter to the <code>render</code> method identifies the tiddler that is acting as the context for this rendering — for example, it provides the fields displayed by the <code><<view>></code> macro. The <code>store</code> parameter is used to resolve any references to other tiddlers.<br><h2> Rerendering</h2>When rendering to the HTML/SVG DOM in the browser, <ahref='TiddlyWiki5'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>TiddlyWiki5</a> also allows a previous rendering to be selectively updated in response to changes in dependent tiddlers. At the moment, only the <ahref='WikiTextRenderer'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>WikiTextRenderer</a> supports rerendering.<br><br>The rerender method on the renderer is called as follows:<br><pre>var node = document.getElementById("myNode");
</pre>The parameters to <code>rerender()</code> are:<br><tableclass='twtable'><tbody><trclass='evenRow'><thalign='left'>Name</th><thalign='left'>Description</th></tr><trclass='oddRow'><tdalign='left'>node</td><tdalign='left'>A reference to the DOM node containing the rendering to be rerendered</td></tr><trclass='evenRow'><tdalign='left'>changes</td><tdalign='left'>A hashmap of <code>{title: "created|modified|deleted"}</code> indicating which tiddlers have changed since the original rendering</td></tr><trclass='oddRow'><tdalign='left'>tiddler</td><tdalign='left'>The tiddler providing the rendering context</td></tr><trclass='evenRow'><tdalign='left'>store</td><tdalign='left'>The store to use for resolving references to other tiddlers</td></tr><trclass='oddRow'><tdalign='left'>renderStep</td><tdalign='left'>See below</td></tr></tbody></table>Currently, the only macro that supports rerendering is the <code><<story>></code> macro; all other macros are rerendered by calling the ordinary <code>render()</code> method again. The reason that the <code><<story>></code> macro goes to the trouble of having a <code>rerender()</code> method is so that it can be carefully selective about not disturbing tiddlers in the DOM that aren't affected by the change. If there were, for instance, a video playing in one of the open tiddlers it would be reset to the beginning if the tiddler were rerendered.<br><br><hr><em>The remaining text is from an earlier draft, and is in the process of being updated</em><br><br>When the text of a tiddler is requested in a different format than its native type, <ahref='TiddlyWiki5'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>TiddlyWiki5</a> compiles a <ahref='JavaScript'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>JavaScript</a> function that generates the new format from the text of the tiddler.<br><br>So, a simple tiddler in <code>application/x-tiddlywiki</code> format might read:<br><pre>Hello World
</pre><br>The function can also include calls to the store to incorporate the values of other tiddlers. Consider this tiddler, called <code>HelloThere</code>:<br><pre>Hello <<tiddler Who>>
</pre>And this one called <code>Who</code>:<br><pre>World
</pre>The function to generate <code>HelloThere</code> in <code>text/html</code> might be:<br><pre>function() {
</pre><br>Now, the return value of this function can be cached until a tiddler in the dependency chain changes. The function itself can be cached until the tiddler itself changes, or a macro that it uses changes.<br><br>The dependency chain is calculated when a tiddler is parsed. Every tiddler that is directly referenced is accumulated (until the point at which it is concluded that it is simpler to mark the tiddler as being dependent on any other tiddler changing).<br><br>Evaluated macro parameters are parsed and can be checked for safeness, and then included in the compiled code. For example,<br><pre>Hello <<echo {{2+2}}>>
</pre><br>The compilation process has several steps:<br><ul><li> First, the parse tree is used to generate a <ahref='JavaScript'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>JavaScript</a> tree</li><li> The <ahref='JavaScript'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>JavaScript</a> tree is scanned to determine the tiddlers on which this one depends</li><li> Finally, executable <ahref='JavaScript'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>JavaScript</a> text is generated by walking the <ahref='JavaScript'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>JavaScript</a> tree</li></ul><br><h1>Planned <ahref='WikiText'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>WikiText</a> Features</h1>It is proposed to extend the existing <ahref='TiddlyWiki'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-resolves'>TiddlyWiki</a> wikitext syntax with the following extensions<br><br><ol><li> Addition of <code>**bold**</code> character formatting</li><li> Addition of <code>`backtick for code`</code> character formatting</li><li> Addition of <ahref='WikiCreole-style'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>WikiCreole-style</a> forced line break, e.g. <code>force\\linebreak</code></li><li> Addition of <ahref='WikiCreole-style'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>WikiCreole-style</a> headings, e.g. <code>==Heading</code></li><li> Addition of <ahref='WikiCreole-style'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>WikiCreole-style</a> headings in tables, e.g. <code>|=|=table|=header|</code></li><li> Addition of white-listed HTML tags intermixed with wikitext</li><li> Addition of <ahref='WikiCreole-style'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>WikiCreole-style</a> pretty links, e.g. <code>[[description -> link]]</code></li><li> Addition of multiline macros, e.g.</li></ol><pre><<myMacro
</pre><br><br><em>This <code>readme</code> file was automatically generated by <ahref='TiddlyWiki5'class='tw-tiddlylink tw-tiddlylink-internal tw-tiddlylink-missing'>TiddlyWiki5</a></em><br>
