1
0
mirror of https://github.com/Jermolene/TiddlyWiki5 synced 2024-11-29 21:09:56 +00:00
TiddlyWiki5/editions/dev/tiddlers/new/Data Storage in Single File TiddlyWiki.tid

100 lines
8.1 KiB
Plaintext
Raw Normal View History

Add support for JSON-formatted tiddler store, and make it the default (#5708) * Add support for JSON-formatted tiddler store, and make it the default The change to `getTiddlersAsJson()` is to allow experimentation * Move JSON tiddlers into their own store area, and fix support for encrypted tiddlers Also add a dummy old-style store area for backwards compatibility The current arrangement is that JSON tiddlers will always override old-style tiddlers. * Use the deserialiser mechanism to decode the content * Refactor $:/core/modules/deserializers.js before we start extending it Cleaning up the helper function names and ordering * Drop support for the "systemArea" div It was only used in really old v5.0.x * Update deserializer to support JSON store format and add some tests * Life UI restrictions on characters in fieldnames * Add another test case * Correct mis-merge * Remove toLowerCase() methods applied to fieldnames * Insert line breaks in output of getTiddlersAsJson (#5786) Rather than have the entire store on one line, insert a line break after each tiddler. * Refactor #5786 for backwards compatibility * Only read .tiddlywiki-tiddler-store blocks from script tags Prompted by @simonbaird's comment here: https://github.com/Jermolene/TiddlyWiki5/pull/5708#discussion_r648833367 * Clean up escaping of unsafe script characters It seems that escaping `<` is sufficient * Add docs from @saqimtiaz Thanks @saqimtiaz * Docs tweaks * Remove excess whitespace Thanks @simonbaird * Fix templates for lazy loading * Remove obsolete item from release note * Clean up whitespace * Docs for the jsontiddler widget * Fix whitespace Fixes #5840 * Comments * Fix newlines in JSON store area * Remove obsolete docs change Co-authored-by: Simon Baird <simon.baird@gmail.com>
2021-07-14 08:15:30 +00:00
created: 20210525165258247
modified: 20210614134705320
tags: Data-Storage doc [[Data Storage]]
title: Data Storage in Single File TiddlyWiki
The single file version of ~TiddlyWiki saves the tiddlers within the HTML file.
Version 5.2.0 of ~TiddlyWiki introduced a new format for how tiddlers are saved within the HTML file.
---
!! Up to and including ~TiddlyWiki v5.1.23
Tiddlers are saved within the HTML file in one of two `<div>` tags depending on whether the TiddlyWiki is configured to encrypt its content or not.
!!! Without encryption
If the ~TiddlyWiki is not encrypted the data is stored in a `<div>` tag with an `id` attribute of "storeArea".
```html
<div id="storeArea" style="display:none;">
```
Within the store area `<div>`, each tiddler is stored in its own div.
//Tiddler DIV//
Each tiddler `<div>` has an attribute corresponding to each of the tiddler's fields, except the text field which is saved within a `pre` tag within the `<div>`. Note that all attributes and therefore field names must be lowercase letters, digits or the characters `-` (dash), `_` (underscore) and `.` (period). The only attribute that is required is `title`, all other attributes are optional. All attribute values and the text field within the `pre` tag are HTML encoded.
Example:
```html
<div id="storeArea" style="display:none;">
<div created="20140611153703343" modified="20140611153734589" tags="testTag" testfield="testvalue" title="TestTiddler" type="text/plain">
<pre>testText</pre>
</div>
</div>
```
!!! With encryption
If the ~TiddlyWiki is configured to encrypt its content, the tiddlers are stored as as an encrypted JSON string in a `<pre>` tag with the `id` attribute "encryptedStoreArea".
```html
<pre id="encryptedStoreArea" type="text/plain" style="display:none;">
```
~TiddlyWiki uses the [[Stanford JavaScript Crypto Library|http://bitwiseshiftleft.github.io/sjcl/]].
---
!! From ~TiddlyWiki v5.2.0
From v5.2.0 onwards, ~TiddlyWiki introduces a new JSON based format for the tiddler store area for unencrypted content, while retaining support for the older store area formats described above for backwards compatibility. The store area format remains unchanged for encrypted content.
!!! Without encryption
By default, all tiddlers are now stored as an array of JSON objects inside a `<script>` tag with the `class` attribute "tiddlywiki-tiddler-store" and the `type` "application/json". For better readability, every tiddler object begins on a new line. There are no longer any restrictions on characters that are allowed in tiddler field names. However, `<script>` tag store areas must encode the `<` character to `\u003c`.
```html
<script class="tiddlywiki-tiddler-store" type="application/json">[
{"title":"XLSX Utilities Edition","created":"20161023202301847","modified":"20161023202301847","tags":"Editions","type":"text/vnd.tiddlywiki","text":"The ''XLSX Utilities'' edition of TiddlyWiki contains tools to work with `.XLSX` spreadsheets generated by applications like Microsoft Excel and Google Sheets. It can be used in the browser or under Node.js.\n\nhttps://tiddlywiki.com/editions/xlsx-utils/\r\n"},
{"text":"In accordance with the [[Philosophy of Tiddlers]], documentation tiddlers are typically short and interlinked.\n\nWhen a tiddler seems as if it needs to contain subheadings, this is often a sign that it should in fact be split into several tiddlers. But it is reasonable for a [[reference tiddler|Reference Tiddlers]] to consist of an untitled introductory section followed by a titled section of details.\n\nConsistency of terminology is essential if the reader is not to become confused. Consistent typography and punctuation lend a professional quality to the documentation. Macros can improve the consistency and maintainability of the text.\n\nUse numbered lists for step-by-step instructions, and bullet points for lists whose order is arbitrary. Use a definition list in preference to a bulleted list if each bulleted item would begin with a term and a colon. If at all possible, avoid burdening the reader with a nested list.\n\nUse a table when information naturally falls into three or more columns, and also for lists of parameters, attributes, etc in [[reference tiddlers|Reference Tiddlers]].\n\nAvoid periods at the end of list items, headings and table cell text.\n\nThe documentation describes the current reality of ~TiddlyWiki. Avoid discussing future aspirations.\n","title":"Tiddler Structure","tags":"[[Improving TiddlyWiki Documentation]]","modified":"20210207124737959","created":"20150110183300000"}
]</script>
```
To retain compatibility with external tools that might insert tiddlers by directly manipulating the ~TiddlyWiki HTML file, the older format `<div>` store area is still present in the HTML file immediately after the new `<script>` tag store area:
```html
<script class="tiddlywiki-tiddler-store" type="application/json">[
{"title":"XLSX Utilities Edition","created":"20161023202301847","modified":"20161023202301847","tags":"Editions","type":"text/vnd.tiddlywiki","text":"The ''XLSX Utilities'' edition of TiddlyWiki contains tools to work with `.XLSX` spreadsheets generated by applications like Microsoft Excel and Google Sheets. It can be used in the browser or under Node.js.\n\nhttps://tiddlywiki.com/editions/xlsx-utils/\r\n"},
{"text":"In accordance with the [[Philosophy of Tiddlers]], documentation tiddlers are typically short and interlinked.\n\nWhen a tiddler seems as if it needs to contain subheadings, this is often a sign that it should in fact be split into several tiddlers. But it is reasonable for a [[reference tiddler|Reference Tiddlers]] to consist of an untitled introductory section followed by a titled section of details.\n\nConsistency of terminology is essential if the reader is not to become confused. Consistent typography and punctuation lend a professional quality to the documentation. Macros can improve the consistency and maintainability of the text.\n\nUse numbered lists for step-by-step instructions, and bullet points for lists whose order is arbitrary. Use a definition list in preference to a bulleted list if each bulleted item would begin with a term and a colon. If at all possible, avoid burdening the reader with a nested list.\n\nUse a table when information naturally falls into three or more columns, and also for lists of parameters, attributes, etc in [[reference tiddlers|Reference Tiddlers]].\n\nAvoid periods at the end of list items, headings and table cell text.\n\nThe documentation describes the current reality of ~TiddlyWiki. Avoid discussing future aspirations.\n","title":"Tiddler Structure","tags":"[[Improving TiddlyWiki Documentation]]","modified":"20210207124737959","created":"20150110183300000"}
]</script><div id="storeArea" style="display:none;"></div>
```
Any tiddlers in the older format `<div>` store area are also loaded before tiddlers from the new `<script>` store area.
!!! Multiple store areas and precedence
Tiddlers from the new `<script>` tag store areas are loaded in the order of their store areas in the HTML file. Therefore if the same tiddler exists in two different `<script>` tag store areas, the tiddler from the later store area takes precedence. Note however that tiddlers from `<script>` tag store areas always take precedence over tiddlers from the older format `<div>` store area. Therefore a tiddler from the older `<div>` store area can never overwrite a tiddler from a `<script>` tag store area.
2021-10-03 10:40:27 +00:00
Note that all `<script>` tags with the `class` attribute "tiddlywiki-tiddler-store" that precede the `$:/boot.js` module in the HTML file will have their content parsed as JSON and loaded as tiddlers.
This allows external tools to easily insert tiddlers into an HTML file by prepending an additional `<script>` tag(s) at the very start of the HTML file:
Add support for JSON-formatted tiddler store, and make it the default (#5708) * Add support for JSON-formatted tiddler store, and make it the default The change to `getTiddlersAsJson()` is to allow experimentation * Move JSON tiddlers into their own store area, and fix support for encrypted tiddlers Also add a dummy old-style store area for backwards compatibility The current arrangement is that JSON tiddlers will always override old-style tiddlers. * Use the deserialiser mechanism to decode the content * Refactor $:/core/modules/deserializers.js before we start extending it Cleaning up the helper function names and ordering * Drop support for the "systemArea" div It was only used in really old v5.0.x * Update deserializer to support JSON store format and add some tests * Life UI restrictions on characters in fieldnames * Add another test case * Correct mis-merge * Remove toLowerCase() methods applied to fieldnames * Insert line breaks in output of getTiddlersAsJson (#5786) Rather than have the entire store on one line, insert a line break after each tiddler. * Refactor #5786 for backwards compatibility * Only read .tiddlywiki-tiddler-store blocks from script tags Prompted by @simonbaird's comment here: https://github.com/Jermolene/TiddlyWiki5/pull/5708#discussion_r648833367 * Clean up escaping of unsafe script characters It seems that escaping `<` is sufficient * Add docs from @saqimtiaz Thanks @saqimtiaz * Docs tweaks * Remove excess whitespace Thanks @simonbaird * Fix templates for lazy loading * Remove obsolete item from release note * Clean up whitespace * Docs for the jsontiddler widget * Fix whitespace Fixes #5840 * Comments * Fix newlines in JSON store area * Remove obsolete docs change Co-authored-by: Simon Baird <simon.baird@gmail.com>
2021-07-14 08:15:30 +00:00
```html
<script class="tiddlywiki-tiddler-store" type="application/json">[
{"created":"20210525212411223","text":"This is some test text","tags":"[[test tag]] [[another tag]]","title":"My new tiddler to insert","modified":"20210525212430577"}
]</script>
2021-10-03 10:40:27 +00:00
<!doctype html>
...
Add support for JSON-formatted tiddler store, and make it the default (#5708) * Add support for JSON-formatted tiddler store, and make it the default The change to `getTiddlersAsJson()` is to allow experimentation * Move JSON tiddlers into their own store area, and fix support for encrypted tiddlers Also add a dummy old-style store area for backwards compatibility The current arrangement is that JSON tiddlers will always override old-style tiddlers. * Use the deserialiser mechanism to decode the content * Refactor $:/core/modules/deserializers.js before we start extending it Cleaning up the helper function names and ordering * Drop support for the "systemArea" div It was only used in really old v5.0.x * Update deserializer to support JSON store format and add some tests * Life UI restrictions on characters in fieldnames * Add another test case * Correct mis-merge * Remove toLowerCase() methods applied to fieldnames * Insert line breaks in output of getTiddlersAsJson (#5786) Rather than have the entire store on one line, insert a line break after each tiddler. * Refactor #5786 for backwards compatibility * Only read .tiddlywiki-tiddler-store blocks from script tags Prompted by @simonbaird's comment here: https://github.com/Jermolene/TiddlyWiki5/pull/5708#discussion_r648833367 * Clean up escaping of unsafe script characters It seems that escaping `<` is sufficient * Add docs from @saqimtiaz Thanks @saqimtiaz * Docs tweaks * Remove excess whitespace Thanks @simonbaird * Fix templates for lazy loading * Remove obsolete item from release note * Clean up whitespace * Docs for the jsontiddler widget * Fix whitespace Fixes #5840 * Comments * Fix newlines in JSON store area * Remove obsolete docs change Co-authored-by: Simon Baird <simon.baird@gmail.com>
2021-07-14 08:15:30 +00:00
```
2021-10-03 10:40:27 +00:00
Although invalid HTML, all known browsers will silently move the script tag to a valid position within the
Add support for JSON-formatted tiddler store, and make it the default (#5708) * Add support for JSON-formatted tiddler store, and make it the default The change to `getTiddlersAsJson()` is to allow experimentation * Move JSON tiddlers into their own store area, and fix support for encrypted tiddlers Also add a dummy old-style store area for backwards compatibility The current arrangement is that JSON tiddlers will always override old-style tiddlers. * Use the deserialiser mechanism to decode the content * Refactor $:/core/modules/deserializers.js before we start extending it Cleaning up the helper function names and ordering * Drop support for the "systemArea" div It was only used in really old v5.0.x * Update deserializer to support JSON store format and add some tests * Life UI restrictions on characters in fieldnames * Add another test case * Correct mis-merge * Remove toLowerCase() methods applied to fieldnames * Insert line breaks in output of getTiddlersAsJson (#5786) Rather than have the entire store on one line, insert a line break after each tiddler. * Refactor #5786 for backwards compatibility * Only read .tiddlywiki-tiddler-store blocks from script tags Prompted by @simonbaird's comment here: https://github.com/Jermolene/TiddlyWiki5/pull/5708#discussion_r648833367 * Clean up escaping of unsafe script characters It seems that escaping `<` is sufficient * Add docs from @saqimtiaz Thanks @saqimtiaz * Docs tweaks * Remove excess whitespace Thanks @simonbaird * Fix templates for lazy loading * Remove obsolete item from release note * Clean up whitespace * Docs for the jsontiddler widget * Fix whitespace Fixes #5840 * Comments * Fix newlines in JSON store area * Remove obsolete docs change Co-authored-by: Simon Baird <simon.baird@gmail.com>
2021-07-14 08:15:30 +00:00
Additional topics:
* [[Extracting tiddlers from a single file TiddlyWiki]]