* Add support for JSON-formatted tiddler store, and make it the default
The change to `getTiddlersAsJson()` is to allow experimentation
* Move JSON tiddlers into their own store area, and fix support for encrypted tiddlers
Also add a dummy old-style store area for backwards compatibility
The current arrangement is that JSON tiddlers will always override old-style tiddlers.
* Use the deserialiser mechanism to decode the content
* Refactor $:/core/modules/deserializers.js before we start extending it
Cleaning up the helper function names and ordering
* Drop support for the "systemArea" div
It was only used in really old v5.0.x
* Update deserializer to support JSON store format and add some tests
* Life UI restrictions on characters in fieldnames
* Add another test case
* Correct mis-merge
* Remove toLowerCase() methods applied to fieldnames
* Insert line breaks in output of getTiddlersAsJson (#5786)
Rather than have the entire store on one line, insert a line break
after each tiddler.
* Refactor #5786 for backwards compatibility
* Only read .tiddlywiki-tiddler-store blocks from script tags
Prompted by @simonbaird's comment here: https://github.com/Jermolene/TiddlyWiki5/pull/5708#discussion_r648833367
* Clean up escaping of unsafe script characters
It seems that escaping `<` is sufficient
* Add docs from @saqimtiaz
Thanks @saqimtiaz
* Docs tweaks
* Remove excess whitespace
Thanks @simonbaird
* Fix templates for lazy loading
* Remove obsolete item from release note
* Clean up whitespace
* Docs for the jsontiddler widget
* Fix whitespace
Fixes#5840
* Comments
* Fix newlines in JSON store area
* Remove obsolete docs change
Co-authored-by: Simon Baird <simon.baird@gmail.com>
* Add :sort filter run prefix, docs and tests. Also extended .utils.makeCompareFunction with a flag for caseSensitivity.
* Documentation updates
* Move case sensitivity handling entirely to utils method so it is reusable
* double quotes are no longer escaped in html bodies
* changed tiddlyweb's html-div-tiddler; documentation
French version still needs a translation though
Default with no suffix is pre-5.1.23 behavior, escaping all Unicode
characters for maximum compatibility (avoids encoding issues). New
"rawunicode" suffix allows passing through Unicode characters U+0080
and up unchanged, for cases where you know your tools are handling
encoding correctly and you want less verbose escaping.
* Add suffix and parameter to trim operator
Fixes#4809
* Unit tests for new trim operator parameters
* Mention trim operator in 5.1.23 release notes
* Address review comments
* Move regex escaping into utils.js trim functions
This way the trimPrefix and trimSuffix functions from utils.js are safe
to call without regex-escaping their parameters, which should make them
easier to use from other parts of the Javascript code.
* First commit
* Add throttling of saves
Now we refuse to save a tiddler more often than once per second.
* Wait for a timeout before trying again after an error
* Modest optimisations of isDirty() method
* Synchronise system tiddlers and deletions from the server
Fixes two long-standing issues:
* Changes to system tiddlers are not synchronised from the server to the browser
* Deletions of tiddlers on the server are not propagated to browser clients
* Make sure we update the dirty status even if there isn't a task to perform
* Replace save-wiki button with popup sync menu
* Remove the "Server" control panel tab
We don't need it with the enhanced sync dropdown
* Add indentation to the save-wiki button
* Fix spacing in dropdown menu items
* Switch between cloud icons according to dirty status
* Add a menu item to copy syncer logs to the clipboard
* Improve animated icon
* Remove indentation from save-wiki button
@pmario the annoying thing is that using `\trim whitespace` trims significant whitespace too, so it means we have to use <$text text=" "/> when we need a space that won't be trimmed. For the moment, I've removed the indentation but will keep thinking about it.
* Further icon, UI and copy text tweaks
Move the icons and styles from the core into the TiddlyWeb plugin
* Clean up PR diff
* Tweak animation durations
* Break the actions from the syncer dropdown into separate tiddlers
@pmario I think this makes things a bit easier to follow
* Refactor syncadaptor creation and logging
The goal is for the syncadaptor to be able to log to the same logger as the syncer, so that the "copy syncer logs to clipboard" data is more useful.
* Don't transition the dirty indicator container colour, just the SVG's colour
* Only trigger a sync for changes to tiddlers we're interested in
Otherwise it is triggered by the creation of the alert tiddlers used to display errors.
* Restore deleting local tiddlers removed from the server
(I had commented it out for some testing and accidentally commited it).
* Guard against missing adaptor info
* We still need to trigger a timeout when there was no task to process
* Avoid repeatedly polling for changes
Instead we only trigger a timeout call at if there is a pending task (ie a tiddler that has changed but isn't yet old enough to save).
* Lazy loading: include skinny versions of lazily loaded tiddlers in the index.html
* Introduce _is_skinny field for indicating that a tiddler is subject to lazy loading
* Remove savetrail plugin from prerelease
It doesn't yet work with the new syncer
* Make the savetrail plugin work again
* Clear outstanding alerts when synchronisation is restored
* Logger: only remove alerts from the same component
Missed off 9f5c0de07
* Make the saving throttle interval configurable (#4385)
After switching Bob to use the core syncer the throttle interval makes saving feel very sluggish compared to the message queue setup that I had before.
The editing lock that I use to prevent conflicts with multiple users doesn't go away until the save is completed, and with the 1 second delay it means that if you edit a tiddler and save it than you have to wait one second before you can edit it again.
* Tweaks to appearance of alerts
* Exclude temp tiddlers from offline snapshots
Otherwise alerts will persist
* Tweak appearance of status line in dropdown
* Update release note
* Web server: Don't include full path in error messages
Fixes#3724
* In change event handler check for deletions
* Disable the official plugin library when the tiddlyweb plugin is loaded
* Hide error details from browser for /files/ route
See https://github.com/Jermolene/TiddlyWiki5/issues/3724#issuecomment-565702492 -- thanks @pmario
* Revert all the changes to the relationship between the syncer and the syncadaptor
Previously we had some major rearrangements to make it possible for the syncadaptor to route it's logging to the logger used by the syncer. The motivation is so that the "copy logs to clipboard" button is more useful.
On reflection, changing the interface this drastically is undesirable from a backwards compatibility perspective, so I'm going to investigate other ways to achieve the logger sharing
* Make the tiddlyweb adaptor use the syncer's logger
So that both are availavble when copying the syncer logs to the clipboard
* Update release note
* Support setting port=0 to get an OS assigned port
Quite useful
* Update code comment
* UI: Use "Get latest changes from server" instead of "Refresh"
* Add getUpdatedTiddlers() method to syncadaptor API
See https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-573579495
* Refactor revision handling within the syncer
Thanks @pmario
* Fix typo in tiddlywebadaptor
* Improve presentation of errors
See https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-573695267
* Add docs for getTiddlerRevision()
* Remove unused error animation
* Update comment for GET /recipes/default/tiddlers/tiddlers.json
* Optimise SVG cloud image
* Add optional list of allowed filters for get all tiddlers route
An attempt to address @Arlen22's concern here:
https://github.com/Jermolene/TiddlyWiki5/pull/4373#pullrequestreview-342146190
* Fix network error alert text translatability
* Fix error code and logging for GET /recipes/default/tiddlers/tiddlers.json
Thanks @Arlen22
* Flip GET /recipes/default/tiddlers/tiddlers.json allowed filter handling to be secure by default
* Validate updates received from getUpdatedTiddlers()
* Add syncer method to force loading of a tiddler from the server
* Remove the release note update to remove the merge conflict
* Fix crash when there's no config section in the tiddlywiki.info file
* Use config tiddler title to check filter query (merge into fix-syncer) (#4478)
* Use config tiddler title to check filter query
* Create config-tiddlers-filter.tid
* Add config switch to enable all filters on GET /recipes/default/tiddlers/tiddlers.json
And update docs
* Fix bug when deleting a tiddler with a shadow
Reported by @kookma at https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-604027528
Co-authored-by: jed <inmysocks@fastmail.com>
Co-authored-by: Arlen22 <arlenbee@gmail.com>
Make good use of "$:/core/modules/utils/base64-utf8/base64-utf8.module.js"
* Add a new base64Encode()
* Both of base64Encode and base64Decode work for Nodejs and Browsers
* changes for global keyboardshortcuts
* add keyboard.js startup module
* remove not existing "th-opening-window" hook
* correct title
* use utils.addEventListeners
* define platform lookup-names on startup
* use the startup-lookup-names array
* use the platform-specific lookupNames only
* Update keyboard.js
* move initializations to the constructor
* move initializations to the constructor
* rename hasAnyTiddlerChanged
* don't explicitely create new RegExp
* use $tw.utils.hopArray
* match strings, no regex
* remove hopArray, move to boot.js
* add $tw.utils.hopArray to boot.js
* style update
* style updates
* move more to keyboardManager module
this could probably be moved to rootwidget.js
* move more to keyboardManager module
* add event listener for shortcuts in new windows
* prevent error when opening window is blocked
* add keydown listener on document in startup.js
* delete startup/keyboard.js
* add missing this.shortcutTiddlers
* Update keyboard.js
* Update boot.js
* add exports.hopArray to utils.js
* minor codingstyle tweak
* change how lookupnames get pushed to array
* Update windows.js
* re-add shortcuts-listener for new windows
I removed this before which I think was because I misunderstood what exactly should go to a separate PR
* fixed the "0 is not a number bug" in listops and x-listops
* Fixed one comment
* "default" is not a good name for a variable
* Following code styles.
Moving getInt to utils.
* Removing unwanted spaces introduced by me
* Add ability to now macro to return same UTC string used in the modified and created fields.
* Revert "Add ability to now macro to return same UTC string used in the modified and created fields."
This reverts commit 7b6ad7db4a.
* Mods to formatDateString to adjust displayed date to UTC for [UTC]
format. Corresponding doc changes, and optimization for special
case.
The code here had got a bit broken by some PRs that I should have
checked more carefully. I’ve done a major refactoring which will
hopefully make it easier to understand, and fixes a number of problems:
* Problem with eg .md tiddlers not being deleted correctly
* Problem with Windows path separators not being usable within
$:/config/FileSystemPaths on Windows
* Problem with filename clashes not being detected correctly when
saving to a different directory via $:/config/FileSystemPaths
* Enables slashes within tiddler titles to be mapped into folders
* Enables plain text files like .md and .css to be saved with .meta
files instead of as .tid files (see #2558)
* No longer replaces spaces with underscores
As this is such a major update, I’d be grateful if Node.js users could
give it a careful run through — in particular, you’ll need to try
creating new tiddlers of various types and ensure that the expected
files are created.
We were using `String.prototype.replace()` without addressing the
wrinkle that dollar signs in the replacement string have special
handling. This caused problems in situations where the replacement
string is derived from user input and contains dollar signs.
Fixes#2517
* Save binary tiddlers with meta file
The filesystemadaptor plugin was a little simplistic in its
understanding of a binary file. It was using the typeInfo dictionary to
choose what tiddler types were binary (and hence needed a meta file when
saving).
I looked as if it was trying to be smart by looking for the hasMetaFile
*OR* had the encoding of base64. Unfortunately the typeInfo only defined
image/jpeg and so any other base64 encoded tiddler was assumed to be of
type text/vnd.tiddlywiki.
The net effect was only JPG images got a meta file and everything else
were saved as .tid files with base64 encoding. It all still worked but
made working with binary data in a Git repo a bit daunting.
There is enough information in the $tw.config.contentTypeInfo to
determine if a tiddler type is encoded with base64 or not. A better list
is available from boot/boot.js who registers all the types thorough the
registerFileType and marks then with base64 were appropriate.
This commit uses the typeInfo dictionary first for any filesystem
specific overrides, then the contentTypeInfo, and finally defaults to
the typeInfo["text/vnd.tiddlywiki"]. It also eliminates the now
unnecessary override for image/jpeg.
I think this might have been the original intent from commit 10b192e7.
From my limited testing all files described in boot/boot.js (lines
1832-1856) with an encoding of base64 now save as the original binary
and a meta file. Meaning that when you start the node server and then
drag-n-drop a binary file (i.e. image/png) it will PUT to the server
and then save it on the filesystem as-is allowing the file to be managed
as a binary file and not a text file. (Binary diffs are better and
GitHub supports them as well).
* Prevent duplicate file extensions
A side effects of using the $tw.config.contentFileInfo in the previous
commit is that it will always append a file extension to the tiddler
title when saving. In most cases this is the correct course of action.
However, sometimes that title is already a proper filename with an
extension (for example importing 'foobar.png' would save a file named
'foobar.png.png') which seemed silly.
This commit simply checks to make sure the title does not already end
with the file extension before appending it to the filename. A little
convenience really.
Since IE apparently doesn't have the String endsWith method I took the
liberty to add a helper method to $tw.utils trying to follow the other
polyfill patterns. I figured this was more generic and readable then
attempting to use a one-off solution inline. I got the polyfill code
from MDN.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/endsWith#Polyfill
Is strEndsWith the best method name?
Entities such as `👷` were broken because
`String.fromCharCode()` is not fully Unicode aware. The fix is to use
`String.fromCodePoint()` where available.
Noted by @ericshulman
Jeremy Ruston