Fix syncer to handler errors properly (#4373)

* First commit * Add throttling of saves Now we refuse to save a tiddler more often than once per second. * Wait for a timeout before trying again after an error * Modest optimisations of isDirty() method * Synchronise system tiddlers and deletions from the server Fixes two long-standing issues: * Changes to system tiddlers are not synchronised from the server to the browser * Deletions of tiddlers on the server are not propagated to browser clients * Make sure we update the dirty status even if there isn't a task to perform * Replace save-wiki button with popup sync menu * Remove the "Server" control panel tab We don't need it with the enhanced sync dropdown * Add indentation to the save-wiki button * Fix spacing in dropdown menu items * Switch between cloud icons according to dirty status * Add a menu item to copy syncer logs to the clipboard * Improve animated icon * Remove indentation from save-wiki button @pmario the annoying thing is that using `\trim whitespace` trims significant whitespace too, so it means we have to use <$text text=" "/> when we need a space that won't be trimmed. For the moment, I've removed the indentation but will keep thinking about it. * Further icon, UI and copy text tweaks Move the icons and styles from the core into the TiddlyWeb plugin * Clean up PR diff * Tweak animation durations * Break the actions from the syncer dropdown into separate tiddlers @pmario I think this makes things a bit easier to follow * Refactor syncadaptor creation and logging The goal is for the syncadaptor to be able to log to the same logger as the syncer, so that the "copy syncer logs to clipboard" data is more useful. * Don't transition the dirty indicator container colour, just the SVG's colour * Only trigger a sync for changes to tiddlers we're interested in Otherwise it is triggered by the creation of the alert tiddlers used to display errors. * Restore deleting local tiddlers removed from the server (I had commented it out for some testing and accidentally commited it). * Guard against missing adaptor info * We still need to trigger a timeout when there was no task to process * Avoid repeatedly polling for changes Instead we only trigger a timeout call at if there is a pending task (ie a tiddler that has changed but isn't yet old enough to save). * Lazy loading: include skinny versions of lazily loaded tiddlers in the index.html * Introduce _is_skinny field for indicating that a tiddler is subject to lazy loading * Remove savetrail plugin from prerelease It doesn't yet work with the new syncer * Make the savetrail plugin work again * Clear outstanding alerts when synchronisation is restored * Logger: only remove alerts from the same component Missed off 9f5c0de07 * Make the saving throttle interval configurable (#4385) After switching Bob to use the core syncer the throttle interval makes saving feel very sluggish compared to the message queue setup that I had before. The editing lock that I use to prevent conflicts with multiple users doesn't go away until the save is completed, and with the 1 second delay it means that if you edit a tiddler and save it than you have to wait one second before you can edit it again. * Tweaks to appearance of alerts * Exclude temp tiddlers from offline snapshots Otherwise alerts will persist * Tweak appearance of status line in dropdown * Update release note * Web server: Don't include full path in error messages Fixes #3724 * In change event handler check for deletions * Disable the official plugin library when the tiddlyweb plugin is loaded * Hide error details from browser for /files/ route See https://github.com/Jermolene/TiddlyWiki5/issues/3724#issuecomment-565702492 -- thanks @pmario * Revert all the changes to the relationship between the syncer and the syncadaptor Previously we had some major rearrangements to make it possible for the syncadaptor to route it's logging to the logger used by the syncer. The motivation is so that the "copy logs to clipboard" button is more useful. On reflection, changing the interface this drastically is undesirable from a backwards compatibility perspective, so I'm going to investigate other ways to achieve the logger sharing * Make the tiddlyweb adaptor use the syncer's logger So that both are availavble when copying the syncer logs to the clipboard * Update release note * Support setting port=0 to get an OS assigned port Quite useful * Update code comment * UI: Use "Get latest changes from server" instead of "Refresh" * Add getUpdatedTiddlers() method to syncadaptor API See https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-573579495 * Refactor revision handling within the syncer Thanks @pmario * Fix typo in tiddlywebadaptor * Improve presentation of errors See https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-573695267 * Add docs for getTiddlerRevision() * Remove unused error animation * Update comment for GET /recipes/default/tiddlers/tiddlers.json * Optimise SVG cloud image * Add optional list of allowed filters for get all tiddlers route An attempt to address @Arlen22's concern here: https://github.com/Jermolene/TiddlyWiki5/pull/4373#pullrequestreview-342146190 * Fix network error alert text translatability * Fix error code and logging for GET /recipes/default/tiddlers/tiddlers.json Thanks @Arlen22 * Flip GET /recipes/default/tiddlers/tiddlers.json allowed filter handling to be secure by default * Validate updates received from getUpdatedTiddlers() * Add syncer method to force loading of a tiddler from the server * Remove the release note update to remove the merge conflict * Fix crash when there's no config section in the tiddlywiki.info file * Use config tiddler title to check filter query (merge into fix-syncer) (#4478) * Use config tiddler title to check filter query * Create config-tiddlers-filter.tid * Add config switch to enable all filters on GET /recipes/default/tiddlers/tiddlers.json And update docs * Fix bug when deleting a tiddler with a shadow Reported by @kookma at https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-604027528 Co-authored-by: jed <inmysocks@fastmail.com> Co-authored-by: Arlen22 <arlenbee@gmail.com>
2026-09-20 08:40:42 +00:00 · 2020-03-30 15:24:05 +01:00
parent 15b8545d72
commit b95723a022
41 changed files with 722 additions and 324 deletions
@@ -1,20 +0,0 @@
-title: $:/plugins/tiddlywiki/tiddlyweb/ServerControlPanel
-caption: Server
-tags: $:/tags/ControlPanel
-
-<$reveal state="$:/status/IsLoggedIn" type="nomatch" text="yes">
-Log in to ~TiddlyWeb: <$button message="tm-login">Login</$button>
-</$reveal>
-<$reveal state="$:/status/IsLoggedIn" type="match" text="yes">
-Logged in as {{$:/status/UserName}} <$button message="tm-logout">Logout</$button>
-</$reveal>
-
----
-
-Host configuration: <$edit-text tiddler="$:/config/tiddlyweb/host" tag="input" default=""/>
-
-<blockquote>//for example, `$protocol$//$host$/folder`, where `$protocol$` is replaced by the protocol (typically `http` or `https`), and `$host$` by the host name//</blockquote>
-
----
-
-<$button message="tm-server-refresh">Refresh</$button> to fetch changes from the server immediately
@@ -0,0 +1,2 @@
+title: $:/config/Server/ExternalFilters/[all[tiddlers]] -[[$:/isEncrypted]] -[prefix[$:/temp/]] -[prefix[$:/status/]]
+text: yes
@@ -0,0 +1,3 @@
+title: $:/config/OfficialPluginLibrary
+
+(This core tiddler is overridden by the tiddlyweb plugin to prevent users from installing official plugins via control panel. Instead they should be installed by editing tiddlywiki.info in the root of the wiki folder)
@@ -0,0 +1,7 @@
+title: $:/core/templates/css-tiddler
+
+<!--
+
+This template is used for saving CSS tiddlers as a style tag with data attributes representing the tiddler fields. This version includes the tiddler changecount as the field `revision`.
+
+-->`<style`<$fields template=' data-tiddler-$name$="$encoded_value$"'></$fields>` data-tiddler-revision="`<<changecount>>`" data-tiddler-bag="default" type="text/css">`<$view field="text" format="text" />`</style>`
@@ -0,0 +1,9 @@
+title: $:/core/templates/html-div-skinny-tiddler
+
+<!--
+
+This template is a variant of the tiddlyweb plugin's overridden version of $:/core/templates/html-div-tiddler used for saving skinny tiddlers (with no text field)
+
+-->`<div`<$fields exclude='text revision bag' template=' $name$="$encoded_value$"'></$fields>` revision="`<<changecount>>`" bag="default" _is_skinny="">
+<pre></pre>
+</div>`
@@ -0,0 +1,4 @@
+title: $:/plugins/tiddlywiki/tiddlyweb/icon/cloud
+tags: $:/tags/Image
+
+<svg class="tc-image-cloud tc-image-button" width="22pt" height="22pt" viewBox="0 0 128 128"><g><path d="M24 103C10.745 103 0 92.255 0 79c0-9.697 5.75-18.05 14.027-21.836A24.787 24.787 0 0114 56c0-13.255 10.745-24 24-24 1.373 0 2.718.115 4.028.337C48.628 24.2 58.707 19 70 19c19.882 0 36 16.118 36 36v.082c12.319 1.016 22 11.336 22 23.918 0 12.239-9.16 22.337-20.999 23.814L107 103H24z"/><path class="tc-image-cloud-idle" d="M57.929 84.698a6 6 0 01-8.485 0L35.302 70.556a6 6 0 118.485-8.485l9.9 9.9L81.97 43.686a6 6 0 018.485 8.486L57.929 84.698z"/><path class="tc-image-cloud-progress tc-animate-rotate-slow" d="M44.8 40a3.6 3.6 0 100 7.2h2.06A23.922 23.922 0 0040 64c0 13.122 10.531 23.785 23.603 23.997L64 88l.001-7.2c-9.171 0-16.626-7.348-16.798-16.477L47.2 64c0-5.165 2.331-9.786 5.999-12.868L53.2 55.6a3.6 3.6 0 107.2 0v-12a3.6 3.6 0 00-3.6-3.6h-12zM64 40v7.2c9.278 0 16.8 7.522 16.8 16.8 0 5.166-2.332 9.787-6 12.869V72.4a3.6 3.6 0 10-7.2 0v12a3.6 3.6 0 003.6 3.6h12a3.6 3.6 0 100-7.2l-2.062.001A23.922 23.922 0 0088 64c0-13.255-10.745-24-24-24z"/></g></svg>
@@ -0,0 +1,7 @@
+title: $:/core/templates/javascript-tiddler
+
+<!--
+
+This template is used for saving JavaScript tiddlers as a script tag with data attributes representing the tiddler fields. This version includes the tiddler changecount as the field `revision`.
+
+-->`<script`<$fields template=' data-tiddler-$name$="$encoded_value$"'></$fields>` data-tiddler-revision="`<<changecount>>`" data-tiddler-bag="default" type="text/javascript">`<$view field="text" format="text" />`</script>`
@@ -2,6 +2,6 @@ title: $:/plugins/tiddlywiki/tiddlyweb/save/offline

 \import [[$:/core/ui/PageMacros]] [all[shadows+tiddlers]tag[$:/tags/Macro]!has[draft.of]]
 \define saveTiddlerFilter()
-[is[tiddler]] -[[$:/boot/boot.css]] -[[$:/HistoryList]] -[type[application/javascript]library[yes]] -[[$:/boot/boot.js]] -[[$:/boot/bootprefix.js]] -[[$:/plugins/tiddlywiki/filesystem]] -[[$:/plugins/tiddlywiki/tiddlyweb]] +[sort[title]] $(publishFilter)$
+[is[tiddler]] -[[$:/boot/boot.css]] -[[$:/HistoryList]] -[type[application/javascript]library[yes]] -[[$:/boot/boot.js]] -[[$:/boot/bootprefix.js]] -[[$:/plugins/tiddlywiki/filesystem]] -[[$:/plugins/tiddlywiki/tiddlyweb]] -[prefix[$:/temp/]] +[sort[title]] $(publishFilter)$
 \end
 {{$:/core/templates/tiddlywiki5.html}}
@@ -0,0 +1,25 @@
+title: $:/core/ui/Buttons/save-wiki
+tags: $:/tags/PageControls
+caption: {{$:/plugins/tiddlywiki/tiddlyweb/icon/cloud}} Server status
+description: Status of synchronisation with server
+
+\define config-title()
+$:/config/PageControlButtons/Visibility/$(listItem)$
+\end
+<$button popup=<<qualify "$:/state/popup/save-wiki">> tooltip="Status of synchronisation with server" aria-label="Server status" class=<<tv-config-toolbar-class>> selectedClass="tc-selected">
+<span class="tc-dirty-indicator">
+<$list filter="[<tv-config-toolbar-icons>match[yes]]">
+{{$:/plugins/tiddlywiki/tiddlyweb/icon/cloud}}
+</$list>
+<$list filter="[<tv-config-toolbar-text>match[yes]]">
+<span class="tc-btn-text"><$text text="Server status"/></span>
+</$list>
+</span>
+</$button>
+<$reveal state=<<qualify "$:/state/popup/save-wiki">> type="popup" position="below" animate="yes">
+<div class="tc-drop-down">
+<$list filter="[all[shadows+tiddlers]tag[$:/tags/SyncerDropdown]!has[draft.of]]" variable="listItem">
+<$transclude tiddler=<<listItem>>/>
+</$list>
+</div>
+</$reveal>
@@ -0,0 +1,40 @@
+title: $:/plugins/tiddlywiki/tiddlyweb/styles
+tags: [[$:/tags/Stylesheet]]
+
+\rules only filteredtranscludeinline transcludeinline macrodef macrocallinline macrocallblock
+
+body.tc-dirty span.tc-dirty-indicator svg {
+	transition: fill 250ms ease-in-out;
+}
+
+body .tc-image-cloud-idle {
+	fill: <<colour background>>;
+	transition: opacity 250ms ease-in-out;
+	opacity: 1;	
+}
+
+body.tc-dirty .tc-image-cloud-idle {
+	opacity: 0;
+}
+
+body .tc-image-cloud-progress {
+	transition: opacity 250ms ease-in-out;
+	transform-origin: 50% 50%;
+	transform: rotate(359deg);
+	animation: animation-rotate-slow 2s infinite linear;
+	fill: <<colour background>>;
+	opacity: 0;
+}
+
+body.tc-dirty .tc-image-cloud-progress {
+	opacity: 1;
+}
+
+@keyframes animation-rotate-slow {
+	from {
+		transform: rotate(0deg);
+	}
+	to {
+		transform: scale(359deg);
+	}
+}
@@ -0,0 +1,6 @@
+title: $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/copy-logs
+tags: $:/tags/SyncerDropdown
+
+<$button message="tm-copy-syncer-logs-to-clipboard" class="tc-btn-invisible">
+{{$:/core/images/copy-clipboard}} Copy syncer logs to clipboard
+</$button>
@@ -0,0 +1,9 @@
+title: $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/login-status
+tags: $:/tags/SyncerDropdown
+
+<$reveal state="$:/status/IsLoggedIn" type="match" text="yes">
+<div class="tc-drop-down-info">
+You are logged in<$reveal state="$:/status/UserName" type="nomatch" text="" default=""> as <strong><$text text={{$:/status/UserName}}/></strong></$reveal><$reveal state="$:/status/IsReadOnly" type="match" text="yes" default="no"> (read-only)</$reveal>
+</div>
+<hr/>
+</$reveal>
@@ -0,0 +1,8 @@
+title: $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/login
+tags: $:/tags/SyncerDropdown
+
+<$reveal state="$:/status/IsLoggedIn" type="nomatch" text="yes">
+<$button message="tm-login" class="tc-btn-invisible">
+{{$:/core/images/unlocked-padlock}} Login
+</$button>
+</$reveal>
@@ -0,0 +1,8 @@
+title: $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/logout
+tags: $:/tags/SyncerDropdown
+
+<$reveal state="$:/status/IsLoggedIn" type="match" text="yes">
+<$button message="tm-logout" class="tc-btn-invisible">
+{{$:/core/images/cancel-button}} Logout
+</$button>
+</$reveal>
@@ -0,0 +1,9 @@
+title: $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/refresh
+tags: $:/tags/SyncerDropdown
+
+<$reveal state="$:/status/IsLoggedIn" type="match" text="yes">
+<$button tooltip="Get latest changes from the server" aria-label="Refresh from server" class="tc-btn-invisible">
+<$action-sendmessage $message="tm-server-refresh"/>
+{{$:/core/images/refresh-button}} <span clas]s="tc-btn-text"><$text text="Get latest changes from the server"/></span>
+</$button>
+</$reveal>
@@ -0,0 +1,9 @@
+title: $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/save-snapshot
+tags: $:/tags/SyncerDropdown
+
+<$button class="tc-btn-invisible">
+<$wikify name="site-title" text={{$:/config/SaveWikiButton/Filename}}>
+<$action-sendmessage $message="tm-download-file" $param={{$:/config/SaveWikiButton/Template}} filename=<<site-title>>/>
+</$wikify>
+{{$:/core/images/download-button}} Save snapshot for offline use
+</$button>
@@ -0,0 +1,2 @@
+title: $:/tags/SyncerDropdown
+list: $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/login-status $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/login $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/refresh $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/logout $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/save-snapshot $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/copy-logs
@@ -27,6 +27,12 @@ function TiddlyWebAdaptor(options) {

 TiddlyWebAdaptor.prototype.name = "tiddlyweb";

+TiddlyWebAdaptor.prototype.supportsLazyLoading = true;
+
+TiddlyWebAdaptor.prototype.setLoggerSaveBuffer = function(loggerForSaving) {
+	this.logger.setSaveBuffer(loggerForSaving);
+};
+
 TiddlyWebAdaptor.prototype.isReady = function() {
 	return this.hasStatus;
 };
@@ -50,6 +56,11 @@ TiddlyWebAdaptor.prototype.getTiddlerInfo = function(tiddler) {
 	};
 };

+TiddlyWebAdaptor.prototype.getTiddlerRevision = function(title) {
+	var tiddler = this.wiki.getTiddler(title);
+	return tiddler.fields.revision;
+};
+
 /*
 Get the current status of the TiddlyWeb connection
 */
@@ -147,6 +158,9 @@ TiddlyWebAdaptor.prototype.getSkinnyTiddlers = function(callback) {
 	var self = this;
 	$tw.utils.httpRequest({
 		url: this.host + "recipes/" + this.recipe + "/tiddlers.json",
+		data: {
+			filter: "[all[tiddlers]] -[[$:/isEncrypted]] -[prefix[$:/temp/]] -[prefix[$:/status/]]"
+		},
 		callback: function(err,data) {
 			// Check for errors
 			if(err) {
@@ -220,7 +234,7 @@ TiddlyWebAdaptor.prototype.deleteTiddler = function(title,callback,options) {
 		return callback(null);
 	}
 	// If we don't have a bag it means that the tiddler hasn't been seen by the server, so we don't need to delete it
-	var bag = options.tiddlerInfo.adaptorInfo.bag;
+	var bag = options.tiddlerInfo.adaptorInfo && options.tiddlerInfo.adaptorInfo.bag;
 	if(!bag) {
 		return callback(null);
 	}