Fix syncer to handler errors properly (#4373)

* First commit * Add throttling of saves Now we refuse to save a tiddler more often than once per second. * Wait for a timeout before trying again after an error * Modest optimisations of isDirty() method * Synchronise system tiddlers and deletions from the server Fixes two long-standing issues: * Changes to system tiddlers are not synchronised from the server to the browser * Deletions of tiddlers on the server are not propagated to browser clients * Make sure we update the dirty status even if there isn't a task to perform * Replace save-wiki button with popup sync menu * Remove the "Server" control panel tab We don't need it with the enhanced sync dropdown * Add indentation to the save-wiki button * Fix spacing in dropdown menu items * Switch between cloud icons according to dirty status * Add a menu item to copy syncer logs to the clipboard * Improve animated icon * Remove indentation from save-wiki button @pmario the annoying thing is that using `\trim whitespace` trims significant whitespace too, so it means we have to use <$text text=" "/> when we need a space that won't be trimmed. For the moment, I've removed the indentation but will keep thinking about it. * Further icon, UI and copy text tweaks Move the icons and styles from the core into the TiddlyWeb plugin * Clean up PR diff * Tweak animation durations * Break the actions from the syncer dropdown into separate tiddlers @pmario I think this makes things a bit easier to follow * Refactor syncadaptor creation and logging The goal is for the syncadaptor to be able to log to the same logger as the syncer, so that the "copy syncer logs to clipboard" data is more useful. * Don't transition the dirty indicator container colour, just the SVG's colour * Only trigger a sync for changes to tiddlers we're interested in Otherwise it is triggered by the creation of the alert tiddlers used to display errors. * Restore deleting local tiddlers removed from the server (I had commented it out for some testing and accidentally commited it). * Guard against missing adaptor info * We still need to trigger a timeout when there was no task to process * Avoid repeatedly polling for changes Instead we only trigger a timeout call at if there is a pending task (ie a tiddler that has changed but isn't yet old enough to save). * Lazy loading: include skinny versions of lazily loaded tiddlers in the index.html * Introduce _is_skinny field for indicating that a tiddler is subject to lazy loading * Remove savetrail plugin from prerelease It doesn't yet work with the new syncer * Make the savetrail plugin work again * Clear outstanding alerts when synchronisation is restored * Logger: only remove alerts from the same component Missed off 9f5c0de07 * Make the saving throttle interval configurable (#4385) After switching Bob to use the core syncer the throttle interval makes saving feel very sluggish compared to the message queue setup that I had before. The editing lock that I use to prevent conflicts with multiple users doesn't go away until the save is completed, and with the 1 second delay it means that if you edit a tiddler and save it than you have to wait one second before you can edit it again. * Tweaks to appearance of alerts * Exclude temp tiddlers from offline snapshots Otherwise alerts will persist * Tweak appearance of status line in dropdown * Update release note * Web server: Don't include full path in error messages Fixes #3724 * In change event handler check for deletions * Disable the official plugin library when the tiddlyweb plugin is loaded * Hide error details from browser for /files/ route See https://github.com/Jermolene/TiddlyWiki5/issues/3724#issuecomment-565702492 -- thanks @pmario * Revert all the changes to the relationship between the syncer and the syncadaptor Previously we had some major rearrangements to make it possible for the syncadaptor to route it's logging to the logger used by the syncer. The motivation is so that the "copy logs to clipboard" button is more useful. On reflection, changing the interface this drastically is undesirable from a backwards compatibility perspective, so I'm going to investigate other ways to achieve the logger sharing * Make the tiddlyweb adaptor use the syncer's logger So that both are availavble when copying the syncer logs to the clipboard * Update release note * Support setting port=0 to get an OS assigned port Quite useful * Update code comment * UI: Use "Get latest changes from server" instead of "Refresh" * Add getUpdatedTiddlers() method to syncadaptor API See https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-573579495 * Refactor revision handling within the syncer Thanks @pmario * Fix typo in tiddlywebadaptor * Improve presentation of errors See https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-573695267 * Add docs for getTiddlerRevision() * Remove unused error animation * Update comment for GET /recipes/default/tiddlers/tiddlers.json * Optimise SVG cloud image * Add optional list of allowed filters for get all tiddlers route An attempt to address @Arlen22's concern here: https://github.com/Jermolene/TiddlyWiki5/pull/4373#pullrequestreview-342146190 * Fix network error alert text translatability * Fix error code and logging for GET /recipes/default/tiddlers/tiddlers.json Thanks @Arlen22 * Flip GET /recipes/default/tiddlers/tiddlers.json allowed filter handling to be secure by default * Validate updates received from getUpdatedTiddlers() * Add syncer method to force loading of a tiddler from the server * Remove the release note update to remove the merge conflict * Fix crash when there's no config section in the tiddlywiki.info file * Use config tiddler title to check filter query (merge into fix-syncer) (#4478) * Use config tiddler title to check filter query * Create config-tiddlers-filter.tid * Add config switch to enable all filters on GET /recipes/default/tiddlers/tiddlers.json And update docs * Fix bug when deleting a tiddler with a shadow Reported by @kookma at https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-604027528 Co-authored-by: jed <inmysocks@fastmail.com> Co-authored-by: Arlen22 <arlenbee@gmail.com>
2025-10-28 22:27:39 +00:00 · 2020-03-30 15:24:05 +01:00
parent 15b8545d72
commit b95723a022
41 changed files with 722 additions and 324 deletions
--- a/core/modules/syncer.js
+++ b/core/modules/syncer.js
@@ -3,7 +3,7 @@ title: $:/core/modules/syncer.js
 type: application/javascript
 module-type: global

-The syncer tracks changes to the store. If a syncadaptor is used then individual tiddlers are synchronised through it. If there is no syncadaptor then the entire wiki is saved via saver modules.
+The syncer tracks changes to the store and synchronises them to a remote data store represented as a "sync adaptor"

 \*/
 (function(){
@@ -23,8 +23,10 @@ Syncer.prototype.titleSyncFilter = "$:/config/SyncFilter";
 Syncer.prototype.titleSyncPollingInterval = "$:/config/SyncPollingInterval";
 Syncer.prototype.titleSyncDisableLazyLoading = "$:/config/SyncDisableLazyLoading";
 Syncer.prototype.titleSavedNotification = "$:/language/Notifications/Save/Done";
+Syncer.prototype.titleSyncThrottleInterval = "$:/config/SyncThrottleInterval";
 Syncer.prototype.taskTimerInterval = 1 * 1000; // Interval for sync timer
 Syncer.prototype.throttleInterval = 1 * 1000; // Defer saving tiddlers if they've changed in the last 1s...
+Syncer.prototype.errorRetryInterval = 5 * 1000; // Interval to retry after an error
 Syncer.prototype.fallbackInterval = 10 * 1000; // Unless the task is older than 10s
 Syncer.prototype.pollTimerInterval = 60 * 1000; // Interval for polling for changes from the adaptor

@@ -36,6 +38,7 @@ wiki: wiki to be synced
 function Syncer(options) {
 	var self = this;
 	this.wiki = options.wiki;
+	// Save parameters
 	this.syncadaptor = options.syncadaptor;
 	this.disableUI = !!options.disableUI;
 	this.titleIsLoggedIn = options.titleIsLoggedIn || this.titleIsLoggedIn;
@@ -43,27 +46,54 @@ function Syncer(options) {
 	this.titleSyncFilter = options.titleSyncFilter || this.titleSyncFilter;
 	this.titleSavedNotification = options.titleSavedNotification || this.titleSavedNotification;
 	this.taskTimerInterval = options.taskTimerInterval || this.taskTimerInterval;
-	this.throttleInterval = options.throttleInterval || this.throttleInterval;
+	this.throttleInterval = options.throttleInterval || parseInt(this.wiki.getTiddlerText(this.titleSyncThrottleInterval,""),10) || this.throttleInterval;
+	this.errorRetryInterval = options.errorRetryInterval || this.errorRetryInterval;
 	this.fallbackInterval = options.fallbackInterval || this.fallbackInterval;
 	this.pollTimerInterval = options.pollTimerInterval || parseInt(this.wiki.getTiddlerText(this.titleSyncPollingInterval,""),10) || this.pollTimerInterval;
 	this.logging = "logging" in options ? options.logging : true;
 	// Make a logger
 	this.logger = new $tw.utils.Logger("syncer" + ($tw.browser ? "-browser" : "") + ($tw.node ? "-server" : "")  + (this.syncadaptor.name ? ("-" + this.syncadaptor.name) : ""),{
-			colour: "cyan",
-			enable: this.logging
-		});
+		colour: "cyan",
+		enable: this.logging,
+		saveHistory: true
+	});
+	if(this.syncadaptor.setLoggerSaveBuffer) {
+		this.syncadaptor.setLoggerSaveBuffer(this.logger);
+	}
 	// Compile the dirty tiddler filter
 	this.filterFn = this.wiki.compileFilter(this.wiki.getTiddlerText(this.titleSyncFilter));
 	// Record information for known tiddlers
 	this.readTiddlerInfo();
-	// Tasks are {type: "load"/"save"/"delete", title:, queueTime:, lastModificationTime:}
-	this.taskQueue = {}; // Hashmap of tasks yet to be performed
-	this.taskInProgress = {}; // Hash of tasks in progress
+	this.titlesToBeLoaded = {}; // Hashmap of titles of tiddlers that need loading from the server
+	this.titlesHaveBeenLazyLoaded = {}; // Hashmap of titles of tiddlers that have already been lazily loaded from the server
+	// Timers
 	this.taskTimerId = null; // Timer for task dispatch
 	this.pollTimerId = null; // Timer for polling server
+	// Number of outstanding requests
+	this.numTasksInProgress = 0;
 	// Listen out for changes to tiddlers
 	this.wiki.addEventListener("change",function(changes) {
-		self.syncToServer(changes);
+		// Filter the changes to just include ones that are being synced
+		var filteredChanges = self.getSyncedTiddlers(function(callback) {
+			$tw.utils.each(changes,function(change,title) {
+				var tiddler = self.wiki.tiddlerExists(title) && self.wiki.getTiddler(title);
+				callback(tiddler,title);
+			});
+		});
+		if(filteredChanges.length > 0) {
+			self.processTaskQueue();
+		} else {
+			// Look for deletions of tiddlers we're already syncing	
+			var outstandingDeletion = false
+			$tw.utils.each(changes,function(change,title,object) {
+				if(change.deleted && $tw.utils.hop(self.tiddlerInfo,title)) {
+					outstandingDeletion = true;
+				}
+			});
+			if(outstandingDeletion) {
+				self.processTaskQueue();
+			}
+		}
 	});
 	// Browser event handlers
 	if($tw.browser && !this.disableUI) {
@@ -86,6 +116,9 @@ function Syncer(options) {
 		$tw.rootWidget.addEventListener("tm-server-refresh",function() {
 			self.handleRefreshEvent();
 		});
+		$tw.rootWidget.addEventListener("tm-copy-syncer-logs-to-clipboard",function() {
+			$tw.utils.copyToClipboard($tw.utils.getSystemInfo() + "\n\nLog:\n" + self.logger.getBuffer());
+		});
 	}
 	// Listen out for lazyLoad events
 	if(!this.disableUI && $tw.wiki.getTiddlerText(this.titleSyncDisableLazyLoading) !== "yes") {
@@ -100,45 +133,83 @@ function Syncer(options) {
 	});
 }

+/*
+Show a generic network error alert
+*/
+Syncer.prototype.showErrorAlert = function() {
+console.log($tw.language.getString("Error/NetworkErrorAlert"))
+	this.logger.alert($tw.language.getString("Error/NetworkErrorAlert"));
+};
+
+/*
+Return an array of the tiddler titles that are subjected to syncing
+*/
+Syncer.prototype.getSyncedTiddlers = function(source) {
+	return this.filterFn.call(this.wiki,source);
+};
+
+/*
+Return an array of the tiddler titles that are subjected to syncing
+*/
+Syncer.prototype.getTiddlerRevision = function(title) {
+	if(this.syncadaptor && this.syncadaptor.getTiddlerRevision) {
+		return this.syncadaptor.getTiddlerRevision(title);
+	} else {
+		return this.wiki.getTiddler(title).fields.revision;	
+	} 
+};
+
 /*
 Read (or re-read) the latest tiddler info from the store
 */
 Syncer.prototype.readTiddlerInfo = function() {
 	// Hashmap by title of {revision:,changeCount:,adaptorInfo:}
+	// "revision" is the revision of the tiddler last seen on the server, and "changecount" is the corresponding local changecount
 	this.tiddlerInfo = {};
 	// Record information for known tiddlers
 	var self = this,
-		tiddlers = this.filterFn.call(this.wiki);
+		tiddlers = this.getSyncedTiddlers();
 	$tw.utils.each(tiddlers,function(title) {
-		var tiddler = self.wiki.getTiddler(title);
+		var tiddler = self.wiki.tiddlerExists(title) && self.wiki.getTiddler(title);
 		self.tiddlerInfo[title] = {
-			revision: tiddler.fields.revision,
+			revision: self.getTiddlerRevision(title),
 			adaptorInfo: self.syncadaptor && self.syncadaptor.getTiddlerInfo(tiddler),
-			changeCount: self.wiki.getChangeCount(title),
-			hasBeenLazyLoaded: false
+			changeCount: self.wiki.getChangeCount(title)
 		};
 	});
 };

-/*
-Create an tiddlerInfo structure if it doesn't already exist
-*/
-Syncer.prototype.createTiddlerInfo = function(title) {
-	if(!$tw.utils.hop(this.tiddlerInfo,title)) {
-		this.tiddlerInfo[title] = {
-			revision: null,
-			adaptorInfo: {},
-			changeCount: -1,
-			hasBeenLazyLoaded: false
-		};
-	}
-};
-
 /*
 Checks whether the wiki is dirty (ie the window shouldn't be closed)
 */
 Syncer.prototype.isDirty = function() {
-	return (this.numTasksInQueue() > 0) || (this.numTasksInProgress() > 0);
+	this.logger.log("Checking dirty status");
+	// Check tiddlers that are in the store and included in the filter function
+	var titles = this.getSyncedTiddlers();
+	for(var index=0; index<titles.length; index++) {
+		var title = titles[index],
+			tiddlerInfo = this.tiddlerInfo[title];
+		if(this.wiki.tiddlerExists(title)) {
+			if(tiddlerInfo) {
+				// If the tiddler is known on the server and has been modified locally then it needs to be saved to the server
+				if($tw.wiki.getChangeCount(title) > tiddlerInfo.changeCount) {
+					return true;
+				}
+			} else {
+				// If the tiddler isn't known on the server then it needs to be saved to the server
+				return true;
+			}
+		}
+	}
+	// Check tiddlers that are known from the server but not currently in the store
+	titles = Object.keys(this.tiddlerInfo);
+	for(index=0; index<titles.length; index++) {
+		if(!this.wiki.tiddlerExists(titles[index])) {
+			// There must be a pending delete
+			return true;
+		}
+	}
+	return false;
 };

 /*
@@ -146,23 +217,26 @@ Update the document body with the class "tc-dirty" if the wiki has unsaved/unsyn
 */
 Syncer.prototype.updateDirtyStatus = function() {
 	if($tw.browser && !this.disableUI) {
-		$tw.utils.toggleClass(document.body,"tc-dirty",this.isDirty());
+		var dirty = this.isDirty();
+		$tw.utils.toggleClass(document.body,"tc-dirty",dirty);
+		if(!dirty) {
+			this.logger.clearAlerts();
+		}
 	}
 };

 /*
 Save an incoming tiddler in the store, and updates the associated tiddlerInfo
 */
-Syncer.prototype.storeTiddler = function(tiddlerFields,hasBeenLazyLoaded) {
+Syncer.prototype.storeTiddler = function(tiddlerFields) {
 	// Save the tiddler
 	var tiddler = new $tw.Tiddler(tiddlerFields);
 	this.wiki.addTiddler(tiddler);
 	// Save the tiddler revision and changeCount details
 	this.tiddlerInfo[tiddlerFields.title] = {
-		revision: tiddlerFields.revision,
+		revision: this.getTiddlerRevision(tiddlerFields.title),
 		adaptorInfo: this.syncadaptor.getTiddlerInfo(tiddler),
-		changeCount: this.wiki.getChangeCount(tiddlerFields.title),
-		hasBeenLazyLoaded: hasBeenLazyLoaded !== undefined ? hasBeenLazyLoaded : true
+		changeCount: this.wiki.getChangeCount(tiddlerFields.title)
 	};
 };

@@ -176,14 +250,14 @@ Syncer.prototype.getStatus = function(callback) {
 		this.syncadaptor.getStatus(function(err,isLoggedIn,username,isReadOnly,isAnonymous) {
 			if(err) {
 				self.logger.alert(err);
-				return;
-			}
-			// Set the various status tiddlers
-			self.wiki.addTiddler({title: self.titleIsReadOnly,text: isReadOnly ? "yes" : "no"});
-			self.wiki.addTiddler({title: self.titleIsAnonymous,text: isAnonymous ? "yes" : "no"});
-			self.wiki.addTiddler({title: self.titleIsLoggedIn,text: isLoggedIn ? "yes" : "no"});
-			if(isLoggedIn) {
-				self.wiki.addTiddler({title: self.titleUserName,text: username || ""});
+			} else {
+				// Set the various status tiddlers
+				self.wiki.addTiddler({title: self.titleIsReadOnly,text: isReadOnly ? "yes" : "no"});
+				self.wiki.addTiddler({title: self.titleIsAnonymous,text: isAnonymous ? "yes" : "no"});
+				self.wiki.addTiddler({title: self.titleIsLoggedIn,text: isLoggedIn ? "yes" : "no"});
+				if(isLoggedIn) {
+					self.wiki.addTiddler({title: self.titleUserName,text: username || ""});
+				}
 			}
 			// Invoke the callback
 			if(callback) {
@@ -199,91 +273,112 @@ Syncer.prototype.getStatus = function(callback) {
 Synchronise from the server by reading the skinny tiddler list and queuing up loads for any tiddlers that we don't already have up to date
 */
 Syncer.prototype.syncFromServer = function() {
-	if(this.syncadaptor && this.syncadaptor.getSkinnyTiddlers) {
-		this.logger.log("Retrieving skinny tiddler list");
-		var self = this;
-		if(this.pollTimerId) {
-			clearTimeout(this.pollTimerId);
-			this.pollTimerId = null;
-		}
-		this.syncadaptor.getSkinnyTiddlers(function(err,tiddlers) {
-			// Trigger the next sync
+	var self = this,
+		cancelNextSync = function() {
+			if(self.pollTimerId) {
+				clearTimeout(self.pollTimerId);
+				self.pollTimerId = null;
+			}
+		},
+		triggerNextSync = function() {
 			self.pollTimerId = setTimeout(function() {
 				self.pollTimerId = null;
 				self.syncFromServer.call(self);
 			},self.pollTimerInterval);
-			// Check for errors
+		};
+	if(this.syncadaptor && this.syncadaptor.getUpdatedTiddlers) {
+		this.logger.log("Retrieving updated tiddler list");
+		cancelNextSync();
+		this.syncadaptor.getUpdatedTiddlers(self,function(err,updates) {
+			triggerNextSync();
 			if(err) {
-				self.logger.alert($tw.language.getString("Error/RetrievingSkinny") + ":",err);
+				self.showErrorAlert();
+				self.logger.log($tw.language.getString("Error/RetrievingSkinny") + ":",err);
 				return;
 			}
+			if(updates) {
+				$tw.utils.each(updates.modifications,function(title) {
+					self.titlesToBeLoaded[title] = true;
+				});
+				$tw.utils.each(updates.deletions,function(title) {
+					delete self.tiddlerInfo[title];
+					self.logger.log("Deleting tiddler missing from server:",title);
+					self.wiki.deleteTiddler(title);
+				});
+				if(updates.modifications.length > 0 || updates.deletions.length > 0) {
+					self.processTaskQueue();
+				}				
+			}
+		});
+	} else if(this.syncadaptor && this.syncadaptor.getSkinnyTiddlers) {
+		this.logger.log("Retrieving skinny tiddler list");
+		cancelNextSync();
+		this.syncadaptor.getSkinnyTiddlers(function(err,tiddlers) {
+			triggerNextSync();
+			// Check for errors
+			if(err) {
+				self.showErrorAlert();
+				self.logger.log($tw.language.getString("Error/RetrievingSkinny") + ":",err);
+				return;
+			}
+			// Keep track of which tiddlers we already know about have been reported this time
+			var previousTitles = Object.keys(self.tiddlerInfo);
 			// Process each incoming tiddler
 			for(var t=0; t<tiddlers.length; t++) {
 				// Get the incoming tiddler fields, and the existing tiddler
 				var tiddlerFields = tiddlers[t],
 					incomingRevision = tiddlerFields.revision + "",
-					tiddler = self.wiki.getTiddler(tiddlerFields.title),
+					tiddler = self.wiki.tiddlerExists(tiddlerFields.title) && self.wiki.getTiddler(tiddlerFields.title),
 					tiddlerInfo = self.tiddlerInfo[tiddlerFields.title],
-					currRevision = tiddlerInfo ? tiddlerInfo.revision : null;
+					currRevision = tiddlerInfo ? tiddlerInfo.revision : null,
+					indexInPreviousTitles = previousTitles.indexOf(tiddlerFields.title);
+				if(indexInPreviousTitles !== -1) {
+					previousTitles.splice(indexInPreviousTitles,1);
+				}
 				// Ignore the incoming tiddler if it's the same as the revision we've already got
 				if(currRevision !== incomingRevision) {
-					// Do a full load if we've already got a fat version of the tiddler
-					if(tiddler && tiddler.fields.text !== undefined) {
-						// Do a full load of this tiddler
-						self.enqueueSyncTask({
-							type: "load",
-							title: tiddlerFields.title
-						});
-					} else {
-						// Load the skinny version of the tiddler
-						self.storeTiddler(tiddlerFields,false);
+					// Only load the skinny version if we don't already have a fat version of the tiddler
+					if(!tiddler || tiddler.fields.text === undefined) {
+						self.storeTiddler(tiddlerFields);
 					}
+					// Do a full load of this tiddler
+					self.titlesToBeLoaded[tiddlerFields.title] = true;
 				}
 			}
+			// Delete any tiddlers that were previously reported but missing this time
+			$tw.utils.each(previousTitles,function(title) {
+				delete self.tiddlerInfo[title];
+				self.logger.log("Deleting tiddler missing from server:",title);
+				self.wiki.deleteTiddler(title);
+			});
+			self.processTaskQueue();
 		});
 	}
 };

 /*
-Synchronise a set of changes to the server
+Force load a tiddler from the server
 */
-Syncer.prototype.syncToServer = function(changes) {
-	var self = this,
-		now = Date.now(),
-		filteredChanges = this.filterFn.call(this.wiki,function(callback) {
-			$tw.utils.each(changes,function(change,title) {
-				var tiddler = self.wiki.getTiddler(title);
-				callback(tiddler,title);
-			});
-		});
-	$tw.utils.each(changes,function(change,title,object) {
-		// Process the change if it is a deletion of a tiddler we're already syncing, or is on the filtered change list
-		if((change.deleted && $tw.utils.hop(self.tiddlerInfo,title)) || filteredChanges.indexOf(title) !== -1) {
-			// Queue a task to sync this tiddler
-			self.enqueueSyncTask({
-				type: change.deleted ? "delete" : "save",
-				title: title
-			});
-		}
-	});
+Syncer.prototype.enqueueLoadTiddler = function(title) {
+	this.titlesToBeLoaded[title] = true;
+	this.processTaskQueue();
 };

 /*
 Lazily load a skinny tiddler if we can
 */
 Syncer.prototype.handleLazyLoadEvent = function(title) {
+	// Ignore if the syncadaptor doesn't handle it
+	if(!this.syncadaptor.supportsLazyLoading) {
+		return;
+	}
 	// Don't lazy load the same tiddler twice
-	var info = this.tiddlerInfo[title];
-	if(!info || !info.hasBeenLazyLoaded) {
+	if(!this.titlesHaveBeenLazyLoaded[title]) {
 		// Don't lazy load if the tiddler isn't included in the sync filter
-		if(this.filterFn.call(this.wiki).indexOf(title) !== -1) {
-			this.createTiddlerInfo(title);
-			this.tiddlerInfo[title].hasBeenLazyLoaded = true;
-			// Queue up a sync task to load this tiddler
-			this.enqueueSyncTask({
-				type: "load",
-				title: title
-			});
+		if(this.getSyncedTiddlers().indexOf(title) !== -1) {
+			// Mark the tiddler as needing loading, and having already been lazily loaded
+			this.titlesToBeLoaded[title] = true;
+			this.titlesHaveBeenLazyLoaded[title] = true;
 		}
 	}
 };
@@ -294,7 +389,7 @@ Dispay a password prompt and allow the user to login
 Syncer.prototype.handleLoginEvent = function() {
 	var self = this;
 	this.getStatus(function(err,isLoggedIn,username) {
-		if(!isLoggedIn) {
+		if(!err && !isLoggedIn) {
 			$tw.passwordPrompt.createPrompt({
 				serviceName: $tw.language.getString("LoginToTiddlySpace"),
 				callback: function(data) {
@@ -324,7 +419,7 @@ Syncer.prototype.login = function(username,password,callback) {
 			}
 			self.getStatus(function(err,isLoggedIn,username) {
 				if(callback) {
-					callback(null,isLoggedIn);
+					callback(err,isLoggedIn);
 				}
 			});
 		});
@@ -358,189 +453,180 @@ Syncer.prototype.handleRefreshEvent = function() {
 };

 /*
-Queue up a sync task. If there is already a pending task for the tiddler, just update the last modification time
+Process the next task
 */
-Syncer.prototype.enqueueSyncTask = function(task) {
-	var self = this,
-		now = Date.now();
-	// Set the timestamps on this task
-	task.queueTime = now;
-	task.lastModificationTime = now;
-	// Fill in some tiddlerInfo if the tiddler is one we haven't seen before
-	this.createTiddlerInfo(task.title);
-	// Bail if this is a save and the tiddler is already at the changeCount that the server has
-	if(task.type === "save" && this.wiki.getChangeCount(task.title) <= this.tiddlerInfo[task.title].changeCount) {
-		return;
-	}
-	// Check if this tiddler is already in the queue
-	if($tw.utils.hop(this.taskQueue,task.title)) {
-		// this.logger.log("Re-queueing up sync task with type:",task.type,"title:",task.title);
-		var existingTask = this.taskQueue[task.title];
-		// If so, just update the last modification time
-		existingTask.lastModificationTime = task.lastModificationTime;
-		// If the new task is a save then we upgrade the existing task to a save. Thus a pending load is turned into a save if the tiddler changes locally in the meantime. But a pending save is not modified to become a load
-		if(task.type === "save" || task.type === "delete") {
-			existingTask.type = task.type;
+Syncer.prototype.processTaskQueue = function() {
+	var self = this;
+	// Only process a task if the sync adaptor is fully initialised and we're not already performing
+	// a task. If we are already performing a task then we'll dispatch the next one when it completes
+	if((!this.syncadaptor.isReady || this.syncadaptor.isReady()) && this.numTasksInProgress === 0) {
+		// Choose the next task to perform
+		var task = this.chooseNextTask();
+		// Perform the task if we had one
+		if(typeof task === "object" && task !== null) {
+			this.numTasksInProgress += 1;
+			task.run(function(err) {
+				self.numTasksInProgress -= 1;
+				if(err) {
+					self.showErrorAlert();
+					self.logger.log("Sync error while processing " + task.type + " of '" + task.title + "':\n" + err);
+					self.updateDirtyStatus();
+					self.triggerTimeout(self.errorRetryInterval);
+				} else {
+					self.updateDirtyStatus();
+					// Process the next task
+					self.processTaskQueue.call(self);					
+				}
+			});
+		} else {
+			// No task is ready so update the status
+			this.updateDirtyStatus();
+			// And trigger a timeout if there is a pending task
+			if(task === true) {
+				this.triggerTimeout();				
+			}
 		}
 	} else {
-		// this.logger.log("Queuing up sync task with type:",task.type,"title:",task.title);
-		// If it is not in the queue, insert it
-		this.taskQueue[task.title] = task;
-		this.updateDirtyStatus();
+		this.updateDirtyStatus();		
 	}
-	// Process the queue
-	$tw.utils.nextTick(function() {self.processTaskQueue.call(self);});
 };

-/*
-Return the number of tasks in progress
-*/
-Syncer.prototype.numTasksInProgress = function() {
-	return $tw.utils.count(this.taskInProgress);
-};
-
-/*
-Return the number of tasks in the queue
-*/
-Syncer.prototype.numTasksInQueue = function() {
-	return $tw.utils.count(this.taskQueue);
-};
-
-/*
-Trigger a timeout if one isn't already outstanding
-*/
-Syncer.prototype.triggerTimeout = function() {
+Syncer.prototype.triggerTimeout = function(interval) {
 	var self = this;
 	if(!this.taskTimerId) {
 		this.taskTimerId = setTimeout(function() {
 			self.taskTimerId = null;
 			self.processTaskQueue.call(self);
-		},self.taskTimerInterval);
+		},interval || self.taskTimerInterval);
 	}
 };

 /*
-Process the task queue, performing the next task if appropriate
-*/
-Syncer.prototype.processTaskQueue = function() {
-	var self = this;
-	// Only process a task if the sync adaptor is fully initialised and we're not already performing a task. If we are already performing a task then we'll dispatch the next one when it completes
-	if((!this.syncadaptor.isReady || this.syncadaptor.isReady()) && this.numTasksInProgress() === 0) {
-		// Choose the next task to perform
-		var task = this.chooseNextTask();
-		// Perform the task if we had one
-		if(task) {
-			// Remove the task from the queue and add it to the in progress list
-			delete this.taskQueue[task.title];
-			this.taskInProgress[task.title] = task;
-			this.updateDirtyStatus();
-			// Dispatch the task
-			this.dispatchTask(task,function(err) {
-				if(err) {
-					self.logger.alert("Sync error while processing '" + task.title + "':\n" + err);
-				}
-				// Mark that this task is no longer in progress
-				delete self.taskInProgress[task.title];
-				self.updateDirtyStatus();
-				// Process the next task
-				self.processTaskQueue.call(self);
-			});
-		} else {
-			// Make sure we've set a time if there wasn't a task to perform, but we've still got tasks in the queue
-			if(this.numTasksInQueue() > 0) {
-				this.triggerTimeout();
-			}
-		}
-	}
-};
+Choose the next sync task. We prioritise saves, then deletes, then loads from the server

-/*
-Choose the next applicable task
+Returns either a task object, null if there's no upcoming tasks, or the boolean true if there are pending tasks that aren't yet due
 */
 Syncer.prototype.chooseNextTask = function() {
-	var self = this,
-		candidateTask = null,
-		now = Date.now();
-	// Select the best candidate task
-	$tw.utils.each(this.taskQueue,function(task,title) {
-		// Exclude the task if there's one of the same name in progress
-		if($tw.utils.hop(self.taskInProgress,title)) {
-			return;
+	var thresholdLastSaved = (new Date()) - this.throttleInterval,
+		havePending = null;
+	// First we look for tiddlers that have been modified locally and need saving back to the server
+	var titles = this.getSyncedTiddlers();
+	for(var index=0; index<titles.length; index++) {
+		var title = titles[index],
+			tiddler = this.wiki.tiddlerExists(title) && this.wiki.getTiddler(title),
+			tiddlerInfo = this.tiddlerInfo[title];
+		if(tiddler) {
+			// If the tiddler is not known on the server, or has been modified locally no more recently than the threshold then it needs to be saved to the server
+			var hasChanged = !tiddlerInfo || $tw.wiki.getChangeCount(title) > tiddlerInfo.changeCount,
+				isReadyToSave = !tiddlerInfo || !tiddlerInfo.timestampLastSaved || tiddlerInfo.timestampLastSaved < thresholdLastSaved;
+			if(hasChanged) {
+				if(isReadyToSave) {
+					return new SaveTiddlerTask(this,title); 					
+				} else {
+					havePending = true;
+				}
+			}
 		}
-		// Exclude the task if it is a save and the tiddler has been modified recently, but not hit the fallback time
-		if(task.type === "save" && (now - task.lastModificationTime) < self.throttleInterval &&
-			(now - task.queueTime) < self.fallbackInterval) {
-			return;
+	}
+	// Second, we check tiddlers that are known from the server but not currently in the store, and so need deleting on the server
+	titles = Object.keys(this.tiddlerInfo);
+	for(index=0; index<titles.length; index++) {
+		title = titles[index];
+		tiddlerInfo = this.tiddlerInfo[title];
+		tiddler = this.wiki.tiddlerExists(title) && this.wiki.getTiddler(title);
+		if(!tiddler) {
+			return new DeleteTiddlerTask(this,title);
 		}
-		// Exclude the task if it is newer than the current best candidate
-		if(candidateTask && candidateTask.queueTime < task.queueTime) {
-			return;
-		}
-		// Now this is our best candidate
-		candidateTask = task;
-	});
-	return candidateTask;
+	}
+	// Check for tiddlers that need loading
+	title = Object.keys(this.titlesToBeLoaded)[0];
+	if(title) {
+		delete this.titlesToBeLoaded[title];
+		return new LoadTiddlerTask(this,title);
+	}
+	// No tasks are ready
+	return havePending;
 };

-/*
-Dispatch a task and invoke the callback
-*/
-Syncer.prototype.dispatchTask = function(task,callback) {
-	var self = this;
-	if(task.type === "save") {
-		var changeCount = this.wiki.getChangeCount(task.title),
-			tiddler = this.wiki.getTiddler(task.title);
-		this.logger.log("Dispatching 'save' task:",task.title);
-		if(tiddler) {
-			this.syncadaptor.saveTiddler(tiddler,function(err,adaptorInfo,revision) {
-				if(err) {
-					return callback(err);
-				}
-				// Adjust the info stored about this tiddler
-				self.tiddlerInfo[task.title] = {
-					changeCount: changeCount,
-					adaptorInfo: adaptorInfo,
-					revision: revision
-				};
-				// Invoke the callback
-				callback(null);
-			},{
-				tiddlerInfo: self.tiddlerInfo[task.title]
-			});
-		} else {
-			this.logger.log(" Not Dispatching 'save' task:",task.title,"tiddler does not exist");
-			return callback(null);
-		}
-	} else if(task.type === "load") {
-		// Load the tiddler
-		this.logger.log("Dispatching 'load' task:",task.title);
-		this.syncadaptor.loadTiddler(task.title,function(err,tiddlerFields) {
+function SaveTiddlerTask(syncer,title) {
+	this.syncer = syncer;
+	this.title = title;
+	this.type = "save";
+}
+
+SaveTiddlerTask.prototype.run = function(callback) {
+	var self = this,
+		changeCount = this.syncer.wiki.getChangeCount(this.title),
+		tiddler = this.syncer.wiki.tiddlerExists(this.title) && this.syncer.wiki.getTiddler(this.title);
+	this.syncer.logger.log("Dispatching 'save' task:",this.title);
+	if(tiddler) {
+		this.syncer.syncadaptor.saveTiddler(tiddler,function(err,adaptorInfo,revision) {
+			// If there's an error, exit without changing any internal state
 			if(err) {
 				return callback(err);
 			}
-			// Store the tiddler
-			if(tiddlerFields) {
-				self.storeTiddler(tiddlerFields,true);
-			}
+			// Adjust the info stored about this tiddler
+			self.syncer.tiddlerInfo[self.title] = {
+				changeCount: changeCount,
+				adaptorInfo: adaptorInfo,
+				revision: revision,
+				timestampLastSaved: new Date()
+			};
 			// Invoke the callback
 			callback(null);
 		});
-	} else if(task.type === "delete") {
-		// Delete the tiddler
-		this.logger.log("Dispatching 'delete' task:",task.title);
-		this.syncadaptor.deleteTiddler(task.title,function(err) {
-			if(err) {
-				return callback(err);
-			}
-			delete self.tiddlerInfo[task.title];
-			// Invoke the callback
-			callback(null);
-		},{
-			tiddlerInfo: self.tiddlerInfo[task.title]
-		});
+	} else {
+		this.syncer.logger.log(" Not Dispatching 'save' task:",this.title,"tiddler does not exist");
+		$tw.utils.nextTick(callback(null));
 	}
 };

+function DeleteTiddlerTask(syncer,title) {
+	this.syncer = syncer;
+	this.title = title;
+	this.type = "delete";
+}
+
+DeleteTiddlerTask.prototype.run = function(callback) {
+	var self = this;
+	this.syncer.logger.log("Dispatching 'delete' task:",this.title);
+	this.syncer.syncadaptor.deleteTiddler(this.title,function(err) {
+		// If there's an error, exit without changing any internal state
+		if(err) {
+			return callback(err);
+		}
+		// Remove the info stored about this tiddler
+		delete self.syncer.tiddlerInfo[self.title];
+		// Invoke the callback
+		callback(null);
+	},{
+		tiddlerInfo: self.syncer.tiddlerInfo[this.title]
+	});
+};
+
+function LoadTiddlerTask(syncer,title) {
+	this.syncer = syncer;
+	this.title = title;
+	this.type = "load";
+}
+
+LoadTiddlerTask.prototype.run = function(callback) {
+	var self = this;
+	this.syncer.logger.log("Dispatching 'load' task:",this.title);
+	this.syncer.syncadaptor.loadTiddler(this.title,function(err,tiddlerFields) {
+		// If there's an error, exit without changing any internal state
+		if(err) {
+			return callback(err);
+		}
+		// Update the info stored about this tiddler
+		if(tiddlerFields) {
+			self.syncer.storeTiddler(tiddlerFields);
+		}
+		// Invoke the callback
+		callback(null);
+	});
+};
+
 exports.Syncer = Syncer;

 })();