1
0
mirror of https://github.com/Jermolene/TiddlyWiki5 synced 2024-06-18 11:29:55 +00:00

Fix syncer to handler errors properly (#4373)

* First commit

* Add throttling of saves

Now we refuse to save a tiddler more often than once per second.

* Wait for a timeout before trying again after an error

* Modest optimisations of isDirty() method

* Synchronise system tiddlers and deletions from the server

Fixes two long-standing issues:

* Changes to system tiddlers are not synchronised from the server to the browser
* Deletions of tiddlers on the server are not propagated to browser clients

* Make sure we update the dirty status even if there isn't a task to perform

* Replace save-wiki button with popup sync menu

* Remove the "Server" control panel tab

We don't need it with the enhanced sync dropdown

* Add indentation to the save-wiki button

* Fix spacing in dropdown menu items

* Switch between cloud icons according to dirty status

* Add a menu item to copy syncer logs to the clipboard

* Improve animated icon

* Remove indentation from save-wiki button

@pmario the annoying thing is that using `\trim whitespace` trims significant whitespace too, so it means we have to use <$text text=" "/> when we need a space that won't be trimmed. For the moment, I've removed the indentation but will keep thinking about it.

* Further icon, UI and copy text tweaks

Move the icons and styles from the core into the TiddlyWeb plugin

* Clean up PR diff

* Tweak animation durations

* Break the actions from the syncer dropdown into separate tiddlers

@pmario I think this makes things a bit easier to follow

* Refactor syncadaptor creation and logging

The goal is for the syncadaptor to be able to log to the same logger as the syncer, so that the "copy syncer logs to clipboard" data is more useful.

* Don't transition the dirty indicator container colour, just the SVG's colour

* Only trigger a sync for changes to tiddlers we're interested in

Otherwise it is triggered by the creation of the alert tiddlers used to display errors.

* Restore deleting local tiddlers removed from the server

(I had commented it out for some testing and accidentally commited it).

* Guard against missing adaptor info

* We still need to trigger a timeout when there was no task to process

* Avoid repeatedly polling for changes

Instead we only trigger a timeout call at if there is a pending task (ie a tiddler that has changed but isn't yet old enough to save).

* Lazy loading: include skinny versions of lazily loaded tiddlers in the index.html

* Introduce _is_skinny field for indicating that a tiddler is subject to lazy loading

* Remove savetrail plugin from prerelease

It doesn't yet work with the new syncer

* Make the savetrail plugin work again

* Clear outstanding alerts when synchronisation is restored

* Logger: only remove alerts from the same component

Missed off 9f5c0de07

* Make the saving throttle interval configurable (#4385)

After switching Bob to use the core syncer the throttle interval makes saving feel very sluggish compared to the message queue setup that I had before.
The editing lock that I use to prevent conflicts with multiple users doesn't go away until the save is completed, and with the 1 second delay it means that if you edit a tiddler and save it than you have to wait one second before you can edit it again.

* Tweaks to appearance of alerts

* Exclude temp tiddlers from offline snapshots

Otherwise alerts will persist

* Tweak appearance of status line in dropdown

* Update release note

* Web server: Don't include full path in error messages

Fixes #3724

* In change event handler check for deletions

* Disable the official plugin library when the tiddlyweb plugin is loaded

* Hide error details from browser for /files/ route

See https://github.com/Jermolene/TiddlyWiki5/issues/3724#issuecomment-565702492 -- thanks @pmario

* Revert all the changes to the relationship between the syncer and the syncadaptor

Previously we had some major rearrangements to make it possible for the syncadaptor to route it's logging to the logger used by the syncer. The motivation is so that the "copy logs to clipboard" button is more useful.

On reflection, changing the interface this drastically is undesirable from a backwards compatibility perspective, so I'm going to investigate other ways to achieve the logger sharing

* Make the tiddlyweb adaptor use the syncer's logger

So that both are availavble when copying the syncer logs to the clipboard

* Update release note

* Support setting port=0 to get an OS assigned port

Quite useful

* Update code comment

* UI: Use "Get latest changes from server" instead of "Refresh"

* Add getUpdatedTiddlers() method to syncadaptor API

See https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-573579495

* Refactor revision handling within the syncer

Thanks @pmario

* Fix typo in tiddlywebadaptor

* Improve presentation of errors

See https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-573695267

* Add docs for getTiddlerRevision()

* Remove unused error animation

* Update comment for GET /recipes/default/tiddlers/tiddlers.json

* Optimise SVG cloud image

* Add optional list of allowed filters for get all tiddlers route

An attempt to address @Arlen22's concern here:

https://github.com/Jermolene/TiddlyWiki5/pull/4373#pullrequestreview-342146190

* Fix network error alert text translatability

* Fix error code and logging for GET /recipes/default/tiddlers/tiddlers.json

Thanks @Arlen22

* Flip GET /recipes/default/tiddlers/tiddlers.json allowed filter handling to be secure by default

* Validate updates received from getUpdatedTiddlers()

* Add syncer method to force loading of a tiddler from the server

* Remove the release note update to remove the merge conflict

* Fix crash when there's no config section in the tiddlywiki.info file

* Use config tiddler title to check filter query (merge into fix-syncer) (#4478)

* Use config tiddler title to check filter query

* Create config-tiddlers-filter.tid

* Add config switch to enable all filters on GET /recipes/default/tiddlers/tiddlers.json

And update docs

* Fix bug when deleting a tiddler with a shadow

Reported by @kookma at https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-604027528

Co-authored-by: jed <inmysocks@fastmail.com>
Co-authored-by: Arlen22 <arlenbee@gmail.com>
This commit is contained in:
Jeremy Ruston 2020-03-30 15:24:05 +01:00 committed by GitHub
parent 15b8545d72
commit b95723a022
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
41 changed files with 722 additions and 324 deletions

View File

@ -35,3 +35,4 @@ title: The unique name of a tiddler
toc-link: Suppresses the tiddler's link in a Table of Contents tree if set to: ''no''
type: The content type of a tiddler
version: Version information for a plugin
_is_skinny: If present, indicates that the tiddler text field must be loaded from the server

View File

@ -28,6 +28,7 @@ Error/Filter: Filter error
Error/FilterSyntax: Syntax error in filter expression
Error/IsFilterOperator: Filter Error: Unknown operand for the 'is' filter operator
Error/LoadingPluginLibrary: Error loading plugin library
Error/NetworkErrorAlert: `<h2>''Network Error''</h2>It looks like the connection to the server has been lost. This may indicate a problem with your network connection. Please attempt to restore network connectivity before continuing.<br><br>''Any unsaved changes will be automatically synchronised when connectivity is restored''.`
Error/RecursiveTransclusion: Recursive transclusion error in transclude widget
Error/RetrievingSkinny: Error retrieving skinny tiddler list
Error/SavingToTWEdit: Error saving to TWEdit

View File

@ -19,22 +19,16 @@ exports.path = /^\/files\/(.+)$/;
exports.handler = function(request,response,state) {
var path = require("path"),
fs = require("fs"),
util = require("util");
var filename = path.resolve($tw.boot.wikiPath,"files",decodeURIComponent(state.params[0])),
util = require("util"),
suppliedFilename = decodeURIComponent(state.params[0]),
filename = path.resolve($tw.boot.wikiPath,"files",suppliedFilename),
extension = path.extname(filename);
fs.readFile(filename,function(err,content) {
var status,content,type = "text/plain";
if(err) {
if(err.code === "ENOENT") {
status = 404;
content = "File '" + filename + "' not found";
} else if(err.code === "EACCES") {
status = 403;
content = "You do not have permission to access the file '" + filename + "'";
} else {
status = 500;
content = err.toString();
}
console.log("Error accessing file " + filename + ": " + err.toString());
status = 404;
content = "File '" + suppliedFilename + "' not found";
} else {
status = 200;
content = content;

View File

@ -3,7 +3,7 @@ title: $:/core/modules/server/routes/get-tiddlers-json.js
type: application/javascript
module-type: route
GET /recipes/default/tiddlers/tiddlers.json
GET /recipes/default/tiddlers/tiddlers.json?filter=<filter>
\*/
(function() {
@ -12,23 +12,34 @@ GET /recipes/default/tiddlers/tiddlers.json
/*global $tw: false */
"use strict";
var DEFAULT_FILTER = "[all[tiddlers]!is[system]sort[title]]";
exports.method = "GET";
exports.path = /^\/recipes\/default\/tiddlers.json$/;
exports.handler = function(request,response,state) {
var filter = state.queryParameters.filter || DEFAULT_FILTER;
if($tw.wiki.getTiddlerText("$:/config/Server/AllowAllExternalFilters") !== "yes") {
if($tw.wiki.getTiddlerText("$:/config/Server/ExternalFilters/" + filter) !== "yes") {
console.log("Blocked attempt to GET /recipes/default/tiddlers/tiddlers.json with filter: " + filter);
response.writeHead(403);
response.end();
return;
}
}
var excludeFields = (state.queryParameters.exclude || "text").split(","),
titles = state.wiki.filterTiddlers(filter);
response.writeHead(200, {"Content-Type": "application/json"});
var tiddlers = [];
state.wiki.forEachTiddler({sortField: "title"},function(title,tiddler) {
var tiddlerFields = {};
$tw.utils.each(tiddler.fields,function(field,name) {
if(name !== "text") {
tiddlerFields[name] = tiddler.getFieldString(name);
}
});
tiddlerFields.revision = state.wiki.getChangeCount(title);
tiddlerFields.type = tiddlerFields.type || "text/vnd.tiddlywiki";
tiddlers.push(tiddlerFields);
$tw.utils.each(titles,function(title) {
var tiddler = state.wiki.getTiddler(title);
if(tiddler) {
var tiddlerFields = tiddler.getFieldStrings({exclude: excludeFields});
tiddlerFields.revision = state.wiki.getChangeCount(title);
tiddlerFields.type = tiddlerFields.type || "text/vnd.tiddlywiki";
tiddlers.push(tiddlerFields);
}
});
var text = JSON.stringify(tiddlers);
response.end(text,"utf8");

View File

@ -16,7 +16,8 @@ if($tw.node) {
var util = require("util"),
fs = require("fs"),
url = require("url"),
path = require("path");
path = require("path"),
querystring = require("querystring");
}
/*
@ -162,6 +163,7 @@ Server.prototype.requestHandler = function(request,response) {
state.wiki = self.wiki;
state.server = self;
state.urlInfo = url.parse(request.url);
state.queryParameters = querystring.parse(state.urlInfo.query);
// Get the principals authorized to access this resource
var authorizationType = this.methodMappings[request.method] || "readers";
// Check for the CSRF header if this is a write
@ -236,6 +238,7 @@ host: optional host address (falls back to value of "host" variable)
prefix: optional prefix (falls back to value of "path-prefix" variable)
*/
Server.prototype.listen = function(port,host,prefix) {
var self = this;
// Handle defaults for port and host
port = port || this.get("port");
host = host || this.get("host");
@ -244,19 +247,24 @@ Server.prototype.listen = function(port,host,prefix) {
if(parseInt(port,10).toString() !== port) {
port = process.env[port] || 8080;
}
$tw.utils.log("Serving on " + this.protocol + "://" + host + ":" + port + prefix,"brown/orange");
$tw.utils.log("(press ctrl-C to exit)","red");
// Warn if required plugins are missing
if(!$tw.wiki.getTiddler("$:/plugins/tiddlywiki/tiddlyweb") || !$tw.wiki.getTiddler("$:/plugins/tiddlywiki/filesystem")) {
$tw.utils.warning("Warning: Plugins required for client-server operation (\"tiddlywiki/filesystem\" and \"tiddlywiki/tiddlyweb\") are missing from tiddlywiki.info file");
}
// Listen
// Create the server
var server;
if(this.listenOptions) {
server = this.transport.createServer(this.listenOptions,this.requestHandler.bind(this));
} else {
server = this.transport.createServer(this.requestHandler.bind(this));
}
// Display the port number after we've started listening (the port number might have been specified as zero, in which case we will get an assigned port)
server.on("listening",function() {
var address = server.address();
$tw.utils.log("Serving on " + self.protocol + "://" + address.address + ":" + address.port + prefix,"brown/orange");
$tw.utils.log("(press ctrl-C to exit)","red");
});
// Listen
return server.listen(port,host);
};

View File

@ -128,7 +128,7 @@ exports.startup = function() {
// Set up the syncer object if we've got a syncadaptor
if($tw.syncadaptor) {
$tw.syncer = new $tw.Syncer({wiki: $tw.wiki, syncadaptor: $tw.syncadaptor});
}
}
// Setup the saver handler
$tw.saverHandler = new $tw.SaverHandler({
wiki: $tw.wiki,

View File

@ -3,7 +3,7 @@ title: $:/core/modules/syncer.js
type: application/javascript
module-type: global
The syncer tracks changes to the store. If a syncadaptor is used then individual tiddlers are synchronised through it. If there is no syncadaptor then the entire wiki is saved via saver modules.
The syncer tracks changes to the store and synchronises them to a remote data store represented as a "sync adaptor"
\*/
(function(){
@ -23,8 +23,10 @@ Syncer.prototype.titleSyncFilter = "$:/config/SyncFilter";
Syncer.prototype.titleSyncPollingInterval = "$:/config/SyncPollingInterval";
Syncer.prototype.titleSyncDisableLazyLoading = "$:/config/SyncDisableLazyLoading";
Syncer.prototype.titleSavedNotification = "$:/language/Notifications/Save/Done";
Syncer.prototype.titleSyncThrottleInterval = "$:/config/SyncThrottleInterval";
Syncer.prototype.taskTimerInterval = 1 * 1000; // Interval for sync timer
Syncer.prototype.throttleInterval = 1 * 1000; // Defer saving tiddlers if they've changed in the last 1s...
Syncer.prototype.errorRetryInterval = 5 * 1000; // Interval to retry after an error
Syncer.prototype.fallbackInterval = 10 * 1000; // Unless the task is older than 10s
Syncer.prototype.pollTimerInterval = 60 * 1000; // Interval for polling for changes from the adaptor
@ -36,6 +38,7 @@ wiki: wiki to be synced
function Syncer(options) {
var self = this;
this.wiki = options.wiki;
// Save parameters
this.syncadaptor = options.syncadaptor;
this.disableUI = !!options.disableUI;
this.titleIsLoggedIn = options.titleIsLoggedIn || this.titleIsLoggedIn;
@ -43,27 +46,54 @@ function Syncer(options) {
this.titleSyncFilter = options.titleSyncFilter || this.titleSyncFilter;
this.titleSavedNotification = options.titleSavedNotification || this.titleSavedNotification;
this.taskTimerInterval = options.taskTimerInterval || this.taskTimerInterval;
this.throttleInterval = options.throttleInterval || this.throttleInterval;
this.throttleInterval = options.throttleInterval || parseInt(this.wiki.getTiddlerText(this.titleSyncThrottleInterval,""),10) || this.throttleInterval;
this.errorRetryInterval = options.errorRetryInterval || this.errorRetryInterval;
this.fallbackInterval = options.fallbackInterval || this.fallbackInterval;
this.pollTimerInterval = options.pollTimerInterval || parseInt(this.wiki.getTiddlerText(this.titleSyncPollingInterval,""),10) || this.pollTimerInterval;
this.logging = "logging" in options ? options.logging : true;
// Make a logger
this.logger = new $tw.utils.Logger("syncer" + ($tw.browser ? "-browser" : "") + ($tw.node ? "-server" : "") + (this.syncadaptor.name ? ("-" + this.syncadaptor.name) : ""),{
colour: "cyan",
enable: this.logging
});
colour: "cyan",
enable: this.logging,
saveHistory: true
});
if(this.syncadaptor.setLoggerSaveBuffer) {
this.syncadaptor.setLoggerSaveBuffer(this.logger);
}
// Compile the dirty tiddler filter
this.filterFn = this.wiki.compileFilter(this.wiki.getTiddlerText(this.titleSyncFilter));
// Record information for known tiddlers
this.readTiddlerInfo();
// Tasks are {type: "load"/"save"/"delete", title:, queueTime:, lastModificationTime:}
this.taskQueue = {}; // Hashmap of tasks yet to be performed
this.taskInProgress = {}; // Hash of tasks in progress
this.titlesToBeLoaded = {}; // Hashmap of titles of tiddlers that need loading from the server
this.titlesHaveBeenLazyLoaded = {}; // Hashmap of titles of tiddlers that have already been lazily loaded from the server
// Timers
this.taskTimerId = null; // Timer for task dispatch
this.pollTimerId = null; // Timer for polling server
// Number of outstanding requests
this.numTasksInProgress = 0;
// Listen out for changes to tiddlers
this.wiki.addEventListener("change",function(changes) {
self.syncToServer(changes);
// Filter the changes to just include ones that are being synced
var filteredChanges = self.getSyncedTiddlers(function(callback) {
$tw.utils.each(changes,function(change,title) {
var tiddler = self.wiki.tiddlerExists(title) && self.wiki.getTiddler(title);
callback(tiddler,title);
});
});
if(filteredChanges.length > 0) {
self.processTaskQueue();
} else {
// Look for deletions of tiddlers we're already syncing
var outstandingDeletion = false
$tw.utils.each(changes,function(change,title,object) {
if(change.deleted && $tw.utils.hop(self.tiddlerInfo,title)) {
outstandingDeletion = true;
}
});
if(outstandingDeletion) {
self.processTaskQueue();
}
}
});
// Browser event handlers
if($tw.browser && !this.disableUI) {
@ -86,6 +116,9 @@ function Syncer(options) {
$tw.rootWidget.addEventListener("tm-server-refresh",function() {
self.handleRefreshEvent();
});
$tw.rootWidget.addEventListener("tm-copy-syncer-logs-to-clipboard",function() {
$tw.utils.copyToClipboard($tw.utils.getSystemInfo() + "\n\nLog:\n" + self.logger.getBuffer());
});
}
// Listen out for lazyLoad events
if(!this.disableUI && $tw.wiki.getTiddlerText(this.titleSyncDisableLazyLoading) !== "yes") {
@ -100,45 +133,83 @@ function Syncer(options) {
});
}
/*
Show a generic network error alert
*/
Syncer.prototype.showErrorAlert = function() {
console.log($tw.language.getString("Error/NetworkErrorAlert"))
this.logger.alert($tw.language.getString("Error/NetworkErrorAlert"));
};
/*
Return an array of the tiddler titles that are subjected to syncing
*/
Syncer.prototype.getSyncedTiddlers = function(source) {
return this.filterFn.call(this.wiki,source);
};
/*
Return an array of the tiddler titles that are subjected to syncing
*/
Syncer.prototype.getTiddlerRevision = function(title) {
if(this.syncadaptor && this.syncadaptor.getTiddlerRevision) {
return this.syncadaptor.getTiddlerRevision(title);
} else {
return this.wiki.getTiddler(title).fields.revision;
}
};
/*
Read (or re-read) the latest tiddler info from the store
*/
Syncer.prototype.readTiddlerInfo = function() {
// Hashmap by title of {revision:,changeCount:,adaptorInfo:}
// "revision" is the revision of the tiddler last seen on the server, and "changecount" is the corresponding local changecount
this.tiddlerInfo = {};
// Record information for known tiddlers
var self = this,
tiddlers = this.filterFn.call(this.wiki);
tiddlers = this.getSyncedTiddlers();
$tw.utils.each(tiddlers,function(title) {
var tiddler = self.wiki.getTiddler(title);
var tiddler = self.wiki.tiddlerExists(title) && self.wiki.getTiddler(title);
self.tiddlerInfo[title] = {
revision: tiddler.fields.revision,
revision: self.getTiddlerRevision(title),
adaptorInfo: self.syncadaptor && self.syncadaptor.getTiddlerInfo(tiddler),
changeCount: self.wiki.getChangeCount(title),
hasBeenLazyLoaded: false
changeCount: self.wiki.getChangeCount(title)
};
});
};
/*
Create an tiddlerInfo structure if it doesn't already exist
*/
Syncer.prototype.createTiddlerInfo = function(title) {
if(!$tw.utils.hop(this.tiddlerInfo,title)) {
this.tiddlerInfo[title] = {
revision: null,
adaptorInfo: {},
changeCount: -1,
hasBeenLazyLoaded: false
};
}
};
/*
Checks whether the wiki is dirty (ie the window shouldn't be closed)
*/
Syncer.prototype.isDirty = function() {
return (this.numTasksInQueue() > 0) || (this.numTasksInProgress() > 0);
this.logger.log("Checking dirty status");
// Check tiddlers that are in the store and included in the filter function
var titles = this.getSyncedTiddlers();
for(var index=0; index<titles.length; index++) {
var title = titles[index],
tiddlerInfo = this.tiddlerInfo[title];
if(this.wiki.tiddlerExists(title)) {
if(tiddlerInfo) {
// If the tiddler is known on the server and has been modified locally then it needs to be saved to the server
if($tw.wiki.getChangeCount(title) > tiddlerInfo.changeCount) {
return true;
}
} else {
// If the tiddler isn't known on the server then it needs to be saved to the server
return true;
}
}
}
// Check tiddlers that are known from the server but not currently in the store
titles = Object.keys(this.tiddlerInfo);
for(index=0; index<titles.length; index++) {
if(!this.wiki.tiddlerExists(titles[index])) {
// There must be a pending delete
return true;
}
}
return false;
};
/*
@ -146,23 +217,26 @@ Update the document body with the class "tc-dirty" if the wiki has unsaved/unsyn
*/
Syncer.prototype.updateDirtyStatus = function() {
if($tw.browser && !this.disableUI) {
$tw.utils.toggleClass(document.body,"tc-dirty",this.isDirty());
var dirty = this.isDirty();
$tw.utils.toggleClass(document.body,"tc-dirty",dirty);
if(!dirty) {
this.logger.clearAlerts();
}
}
};
/*
Save an incoming tiddler in the store, and updates the associated tiddlerInfo
*/
Syncer.prototype.storeTiddler = function(tiddlerFields,hasBeenLazyLoaded) {
Syncer.prototype.storeTiddler = function(tiddlerFields) {
// Save the tiddler
var tiddler = new $tw.Tiddler(tiddlerFields);
this.wiki.addTiddler(tiddler);
// Save the tiddler revision and changeCount details
this.tiddlerInfo[tiddlerFields.title] = {
revision: tiddlerFields.revision,
revision: this.getTiddlerRevision(tiddlerFields.title),
adaptorInfo: this.syncadaptor.getTiddlerInfo(tiddler),
changeCount: this.wiki.getChangeCount(tiddlerFields.title),
hasBeenLazyLoaded: hasBeenLazyLoaded !== undefined ? hasBeenLazyLoaded : true
changeCount: this.wiki.getChangeCount(tiddlerFields.title)
};
};
@ -176,14 +250,14 @@ Syncer.prototype.getStatus = function(callback) {
this.syncadaptor.getStatus(function(err,isLoggedIn,username,isReadOnly,isAnonymous) {
if(err) {
self.logger.alert(err);
return;
}
// Set the various status tiddlers
self.wiki.addTiddler({title: self.titleIsReadOnly,text: isReadOnly ? "yes" : "no"});
self.wiki.addTiddler({title: self.titleIsAnonymous,text: isAnonymous ? "yes" : "no"});
self.wiki.addTiddler({title: self.titleIsLoggedIn,text: isLoggedIn ? "yes" : "no"});
if(isLoggedIn) {
self.wiki.addTiddler({title: self.titleUserName,text: username || ""});
} else {
// Set the various status tiddlers
self.wiki.addTiddler({title: self.titleIsReadOnly,text: isReadOnly ? "yes" : "no"});
self.wiki.addTiddler({title: self.titleIsAnonymous,text: isAnonymous ? "yes" : "no"});
self.wiki.addTiddler({title: self.titleIsLoggedIn,text: isLoggedIn ? "yes" : "no"});
if(isLoggedIn) {
self.wiki.addTiddler({title: self.titleUserName,text: username || ""});
}
}
// Invoke the callback
if(callback) {
@ -199,91 +273,112 @@ Syncer.prototype.getStatus = function(callback) {
Synchronise from the server by reading the skinny tiddler list and queuing up loads for any tiddlers that we don't already have up to date
*/
Syncer.prototype.syncFromServer = function() {
if(this.syncadaptor && this.syncadaptor.getSkinnyTiddlers) {
this.logger.log("Retrieving skinny tiddler list");
var self = this;
if(this.pollTimerId) {
clearTimeout(this.pollTimerId);
this.pollTimerId = null;
}
this.syncadaptor.getSkinnyTiddlers(function(err,tiddlers) {
// Trigger the next sync
var self = this,
cancelNextSync = function() {
if(self.pollTimerId) {
clearTimeout(self.pollTimerId);
self.pollTimerId = null;
}
},
triggerNextSync = function() {
self.pollTimerId = setTimeout(function() {
self.pollTimerId = null;
self.syncFromServer.call(self);
},self.pollTimerInterval);
// Check for errors
};
if(this.syncadaptor && this.syncadaptor.getUpdatedTiddlers) {
this.logger.log("Retrieving updated tiddler list");
cancelNextSync();
this.syncadaptor.getUpdatedTiddlers(self,function(err,updates) {
triggerNextSync();
if(err) {
self.logger.alert($tw.language.getString("Error/RetrievingSkinny") + ":",err);
self.showErrorAlert();
self.logger.log($tw.language.getString("Error/RetrievingSkinny") + ":",err);
return;
}
if(updates) {
$tw.utils.each(updates.modifications,function(title) {
self.titlesToBeLoaded[title] = true;
});
$tw.utils.each(updates.deletions,function(title) {
delete self.tiddlerInfo[title];
self.logger.log("Deleting tiddler missing from server:",title);
self.wiki.deleteTiddler(title);
});
if(updates.modifications.length > 0 || updates.deletions.length > 0) {
self.processTaskQueue();
}
}
});
} else if(this.syncadaptor && this.syncadaptor.getSkinnyTiddlers) {
this.logger.log("Retrieving skinny tiddler list");
cancelNextSync();
this.syncadaptor.getSkinnyTiddlers(function(err,tiddlers) {
triggerNextSync();
// Check for errors
if(err) {
self.showErrorAlert();
self.logger.log($tw.language.getString("Error/RetrievingSkinny") + ":",err);
return;
}
// Keep track of which tiddlers we already know about have been reported this time
var previousTitles = Object.keys(self.tiddlerInfo);
// Process each incoming tiddler
for(var t=0; t<tiddlers.length; t++) {
// Get the incoming tiddler fields, and the existing tiddler
var tiddlerFields = tiddlers[t],
incomingRevision = tiddlerFields.revision + "",
tiddler = self.wiki.getTiddler(tiddlerFields.title),
tiddler = self.wiki.tiddlerExists(tiddlerFields.title) && self.wiki.getTiddler(tiddlerFields.title),
tiddlerInfo = self.tiddlerInfo[tiddlerFields.title],
currRevision = tiddlerInfo ? tiddlerInfo.revision : null;
currRevision = tiddlerInfo ? tiddlerInfo.revision : null,
indexInPreviousTitles = previousTitles.indexOf(tiddlerFields.title);
if(indexInPreviousTitles !== -1) {
previousTitles.splice(indexInPreviousTitles,1);
}
// Ignore the incoming tiddler if it's the same as the revision we've already got
if(currRevision !== incomingRevision) {
// Do a full load if we've already got a fat version of the tiddler
if(tiddler && tiddler.fields.text !== undefined) {
// Do a full load of this tiddler
self.enqueueSyncTask({
type: "load",
title: tiddlerFields.title
});
} else {
// Load the skinny version of the tiddler
self.storeTiddler(tiddlerFields,false);
// Only load the skinny version if we don't already have a fat version of the tiddler
if(!tiddler || tiddler.fields.text === undefined) {
self.storeTiddler(tiddlerFields);
}
// Do a full load of this tiddler
self.titlesToBeLoaded[tiddlerFields.title] = true;
}
}
// Delete any tiddlers that were previously reported but missing this time
$tw.utils.each(previousTitles,function(title) {
delete self.tiddlerInfo[title];
self.logger.log("Deleting tiddler missing from server:",title);
self.wiki.deleteTiddler(title);
});
self.processTaskQueue();
});
}
};
/*
Synchronise a set of changes to the server
Force load a tiddler from the server
*/
Syncer.prototype.syncToServer = function(changes) {
var self = this,
now = Date.now(),
filteredChanges = this.filterFn.call(this.wiki,function(callback) {
$tw.utils.each(changes,function(change,title) {
var tiddler = self.wiki.getTiddler(title);
callback(tiddler,title);
});
});
$tw.utils.each(changes,function(change,title,object) {
// Process the change if it is a deletion of a tiddler we're already syncing, or is on the filtered change list
if((change.deleted && $tw.utils.hop(self.tiddlerInfo,title)) || filteredChanges.indexOf(title) !== -1) {
// Queue a task to sync this tiddler
self.enqueueSyncTask({
type: change.deleted ? "delete" : "save",
title: title
});
}
});
Syncer.prototype.enqueueLoadTiddler = function(title) {
this.titlesToBeLoaded[title] = true;
this.processTaskQueue();
};
/*
Lazily load a skinny tiddler if we can
*/
Syncer.prototype.handleLazyLoadEvent = function(title) {
// Ignore if the syncadaptor doesn't handle it
if(!this.syncadaptor.supportsLazyLoading) {
return;
}
// Don't lazy load the same tiddler twice
var info = this.tiddlerInfo[title];
if(!info || !info.hasBeenLazyLoaded) {
if(!this.titlesHaveBeenLazyLoaded[title]) {
// Don't lazy load if the tiddler isn't included in the sync filter
if(this.filterFn.call(this.wiki).indexOf(title) !== -1) {
this.createTiddlerInfo(title);
this.tiddlerInfo[title].hasBeenLazyLoaded = true;
// Queue up a sync task to load this tiddler
this.enqueueSyncTask({
type: "load",
title: title
});
if(this.getSyncedTiddlers().indexOf(title) !== -1) {
// Mark the tiddler as needing loading, and having already been lazily loaded
this.titlesToBeLoaded[title] = true;
this.titlesHaveBeenLazyLoaded[title] = true;
}
}
};
@ -294,7 +389,7 @@ Dispay a password prompt and allow the user to login
Syncer.prototype.handleLoginEvent = function() {
var self = this;
this.getStatus(function(err,isLoggedIn,username) {
if(!isLoggedIn) {
if(!err && !isLoggedIn) {
$tw.passwordPrompt.createPrompt({
serviceName: $tw.language.getString("LoginToTiddlySpace"),
callback: function(data) {
@ -324,7 +419,7 @@ Syncer.prototype.login = function(username,password,callback) {
}
self.getStatus(function(err,isLoggedIn,username) {
if(callback) {
callback(null,isLoggedIn);
callback(err,isLoggedIn);
}
});
});
@ -358,189 +453,180 @@ Syncer.prototype.handleRefreshEvent = function() {
};
/*
Queue up a sync task. If there is already a pending task for the tiddler, just update the last modification time
Process the next task
*/
Syncer.prototype.enqueueSyncTask = function(task) {
var self = this,
now = Date.now();
// Set the timestamps on this task
task.queueTime = now;
task.lastModificationTime = now;
// Fill in some tiddlerInfo if the tiddler is one we haven't seen before
this.createTiddlerInfo(task.title);
// Bail if this is a save and the tiddler is already at the changeCount that the server has
if(task.type === "save" && this.wiki.getChangeCount(task.title) <= this.tiddlerInfo[task.title].changeCount) {
return;
}
// Check if this tiddler is already in the queue
if($tw.utils.hop(this.taskQueue,task.title)) {
// this.logger.log("Re-queueing up sync task with type:",task.type,"title:",task.title);
var existingTask = this.taskQueue[task.title];
// If so, just update the last modification time
existingTask.lastModificationTime = task.lastModificationTime;
// If the new task is a save then we upgrade the existing task to a save. Thus a pending load is turned into a save if the tiddler changes locally in the meantime. But a pending save is not modified to become a load
if(task.type === "save" || task.type === "delete") {
existingTask.type = task.type;
Syncer.prototype.processTaskQueue = function() {
var self = this;
// Only process a task if the sync adaptor is fully initialised and we're not already performing
// a task. If we are already performing a task then we'll dispatch the next one when it completes
if((!this.syncadaptor.isReady || this.syncadaptor.isReady()) && this.numTasksInProgress === 0) {
// Choose the next task to perform
var task = this.chooseNextTask();
// Perform the task if we had one
if(typeof task === "object" && task !== null) {
this.numTasksInProgress += 1;
task.run(function(err) {
self.numTasksInProgress -= 1;
if(err) {
self.showErrorAlert();
self.logger.log("Sync error while processing " + task.type + " of '" + task.title + "':\n" + err);
self.updateDirtyStatus();
self.triggerTimeout(self.errorRetryInterval);
} else {
self.updateDirtyStatus();
// Process the next task
self.processTaskQueue.call(self);
}
});
} else {
// No task is ready so update the status
this.updateDirtyStatus();
// And trigger a timeout if there is a pending task
if(task === true) {
this.triggerTimeout();
}
}
} else {
// this.logger.log("Queuing up sync task with type:",task.type,"title:",task.title);
// If it is not in the queue, insert it
this.taskQueue[task.title] = task;
this.updateDirtyStatus();
this.updateDirtyStatus();
}
// Process the queue
$tw.utils.nextTick(function() {self.processTaskQueue.call(self);});
};
/*
Return the number of tasks in progress
*/
Syncer.prototype.numTasksInProgress = function() {
return $tw.utils.count(this.taskInProgress);
};
/*
Return the number of tasks in the queue
*/
Syncer.prototype.numTasksInQueue = function() {
return $tw.utils.count(this.taskQueue);
};
/*
Trigger a timeout if one isn't already outstanding
*/
Syncer.prototype.triggerTimeout = function() {
Syncer.prototype.triggerTimeout = function(interval) {
var self = this;
if(!this.taskTimerId) {
this.taskTimerId = setTimeout(function() {
self.taskTimerId = null;
self.processTaskQueue.call(self);
},self.taskTimerInterval);
},interval || self.taskTimerInterval);
}
};
/*
Process the task queue, performing the next task if appropriate
*/
Syncer.prototype.processTaskQueue = function() {
var self = this;
// Only process a task if the sync adaptor is fully initialised and we're not already performing a task. If we are already performing a task then we'll dispatch the next one when it completes
if((!this.syncadaptor.isReady || this.syncadaptor.isReady()) && this.numTasksInProgress() === 0) {
// Choose the next task to perform
var task = this.chooseNextTask();
// Perform the task if we had one
if(task) {
// Remove the task from the queue and add it to the in progress list
delete this.taskQueue[task.title];
this.taskInProgress[task.title] = task;
this.updateDirtyStatus();
// Dispatch the task
this.dispatchTask(task,function(err) {
if(err) {
self.logger.alert("Sync error while processing '" + task.title + "':\n" + err);
}
// Mark that this task is no longer in progress
delete self.taskInProgress[task.title];
self.updateDirtyStatus();
// Process the next task
self.processTaskQueue.call(self);
});
} else {
// Make sure we've set a time if there wasn't a task to perform, but we've still got tasks in the queue
if(this.numTasksInQueue() > 0) {
this.triggerTimeout();
}
}
}
};
Choose the next sync task. We prioritise saves, then deletes, then loads from the server
/*
Choose the next applicable task
Returns either a task object, null if there's no upcoming tasks, or the boolean true if there are pending tasks that aren't yet due
*/
Syncer.prototype.chooseNextTask = function() {
var self = this,
candidateTask = null,
now = Date.now();
// Select the best candidate task
$tw.utils.each(this.taskQueue,function(task,title) {
// Exclude the task if there's one of the same name in progress
if($tw.utils.hop(self.taskInProgress,title)) {
return;
var thresholdLastSaved = (new Date()) - this.throttleInterval,
havePending = null;
// First we look for tiddlers that have been modified locally and need saving back to the server
var titles = this.getSyncedTiddlers();
for(var index=0; index<titles.length; index++) {
var title = titles[index],
tiddler = this.wiki.tiddlerExists(title) && this.wiki.getTiddler(title),
tiddlerInfo = this.tiddlerInfo[title];
if(tiddler) {
// If the tiddler is not known on the server, or has been modified locally no more recently than the threshold then it needs to be saved to the server
var hasChanged = !tiddlerInfo || $tw.wiki.getChangeCount(title) > tiddlerInfo.changeCount,
isReadyToSave = !tiddlerInfo || !tiddlerInfo.timestampLastSaved || tiddlerInfo.timestampLastSaved < thresholdLastSaved;
if(hasChanged) {
if(isReadyToSave) {
return new SaveTiddlerTask(this,title);
} else {
havePending = true;
}
}
}
// Exclude the task if it is a save and the tiddler has been modified recently, but not hit the fallback time
if(task.type === "save" && (now - task.lastModificationTime) < self.throttleInterval &&
(now - task.queueTime) < self.fallbackInterval) {
return;
}
// Second, we check tiddlers that are known from the server but not currently in the store, and so need deleting on the server
titles = Object.keys(this.tiddlerInfo);
for(index=0; index<titles.length; index++) {
title = titles[index];
tiddlerInfo = this.tiddlerInfo[title];
tiddler = this.wiki.tiddlerExists(title) && this.wiki.getTiddler(title);
if(!tiddler) {
return new DeleteTiddlerTask(this,title);
}
// Exclude the task if it is newer than the current best candidate
if(candidateTask && candidateTask.queueTime < task.queueTime) {
return;
}
// Now this is our best candidate
candidateTask = task;
});
return candidateTask;
}
// Check for tiddlers that need loading
title = Object.keys(this.titlesToBeLoaded)[0];
if(title) {
delete this.titlesToBeLoaded[title];
return new LoadTiddlerTask(this,title);
}
// No tasks are ready
return havePending;
};
/*
Dispatch a task and invoke the callback
*/
Syncer.prototype.dispatchTask = function(task,callback) {
var self = this;
if(task.type === "save") {
var changeCount = this.wiki.getChangeCount(task.title),
tiddler = this.wiki.getTiddler(task.title);
this.logger.log("Dispatching 'save' task:",task.title);
if(tiddler) {
this.syncadaptor.saveTiddler(tiddler,function(err,adaptorInfo,revision) {
if(err) {
return callback(err);
}
// Adjust the info stored about this tiddler
self.tiddlerInfo[task.title] = {
changeCount: changeCount,
adaptorInfo: adaptorInfo,
revision: revision
};
// Invoke the callback
callback(null);
},{
tiddlerInfo: self.tiddlerInfo[task.title]
});
} else {
this.logger.log(" Not Dispatching 'save' task:",task.title,"tiddler does not exist");
return callback(null);
}
} else if(task.type === "load") {
// Load the tiddler
this.logger.log("Dispatching 'load' task:",task.title);
this.syncadaptor.loadTiddler(task.title,function(err,tiddlerFields) {
function SaveTiddlerTask(syncer,title) {
this.syncer = syncer;
this.title = title;
this.type = "save";
}
SaveTiddlerTask.prototype.run = function(callback) {
var self = this,
changeCount = this.syncer.wiki.getChangeCount(this.title),
tiddler = this.syncer.wiki.tiddlerExists(this.title) && this.syncer.wiki.getTiddler(this.title);
this.syncer.logger.log("Dispatching 'save' task:",this.title);
if(tiddler) {
this.syncer.syncadaptor.saveTiddler(tiddler,function(err,adaptorInfo,revision) {
// If there's an error, exit without changing any internal state
if(err) {
return callback(err);
}
// Store the tiddler
if(tiddlerFields) {
self.storeTiddler(tiddlerFields,true);
}
// Adjust the info stored about this tiddler
self.syncer.tiddlerInfo[self.title] = {
changeCount: changeCount,
adaptorInfo: adaptorInfo,
revision: revision,
timestampLastSaved: new Date()
};
// Invoke the callback
callback(null);
});
} else if(task.type === "delete") {
// Delete the tiddler
this.logger.log("Dispatching 'delete' task:",task.title);
this.syncadaptor.deleteTiddler(task.title,function(err) {
if(err) {
return callback(err);
}
delete self.tiddlerInfo[task.title];
// Invoke the callback
callback(null);
},{
tiddlerInfo: self.tiddlerInfo[task.title]
});
} else {
this.syncer.logger.log(" Not Dispatching 'save' task:",this.title,"tiddler does not exist");
$tw.utils.nextTick(callback(null));
}
};
function DeleteTiddlerTask(syncer,title) {
this.syncer = syncer;
this.title = title;
this.type = "delete";
}
DeleteTiddlerTask.prototype.run = function(callback) {
var self = this;
this.syncer.logger.log("Dispatching 'delete' task:",this.title);
this.syncer.syncadaptor.deleteTiddler(this.title,function(err) {
// If there's an error, exit without changing any internal state
if(err) {
return callback(err);
}
// Remove the info stored about this tiddler
delete self.syncer.tiddlerInfo[self.title];
// Invoke the callback
callback(null);
},{
tiddlerInfo: self.syncer.tiddlerInfo[this.title]
});
};
function LoadTiddlerTask(syncer,title) {
this.syncer = syncer;
this.title = title;
this.type = "load";
}
LoadTiddlerTask.prototype.run = function(callback) {
var self = this;
this.syncer.logger.log("Dispatching 'load' task:",this.title);
this.syncer.syncadaptor.loadTiddler(this.title,function(err,tiddlerFields) {
// If there's an error, exit without changing any internal state
if(err) {
return callback(err);
}
// Update the info stored about this tiddler
if(tiddlerFields) {
self.syncer.storeTiddler(tiddlerFields);
}
// Invoke the callback
callback(null);
});
};
exports.Syncer = Syncer;
})();

View File

@ -22,6 +22,7 @@ A quick and dirty HTTP function; to be refactored later. Options are:
*/
exports.httpRequest = function(options) {
var type = options.type || "GET",
url = options.url,
headers = options.headers || {accept: "application/json"},
returnProp = options.returnProp || "responseText",
request = new XMLHttpRequest(),
@ -36,7 +37,11 @@ exports.httpRequest = function(options) {
$tw.utils.each(options.data,function(dataItem,dataItemTitle) {
results.push(dataItemTitle + "=" + encodeURIComponent(dataItem));
});
data = results.join("&");
if(type === "GET" || type === "HEAD") {
url += "?" + results.join("&");
} else {
data = results.join("&");
}
}
}
// Set up the state change handler
@ -52,7 +57,7 @@ exports.httpRequest = function(options) {
}
};
// Make the request
request.open(type,options.url,true);
request.open(type,url,true);
if(headers) {
$tw.utils.each(headers,function(header,headerTitle,object) {
request.setRequestHeader(headerTitle,header);

View File

@ -22,15 +22,42 @@ function Logger(componentName,options) {
this.componentName = componentName || "";
this.colour = options.colour || "white";
this.enable = "enable" in options ? options.enable : true;
this.save = "save" in options ? options.save : true;
this.saveLimit = options.saveLimit || 100 * 1024;
this.saveBufferLogger = this;
this.buffer = "";
this.alertCount = 0;
}
Logger.prototype.setSaveBuffer = function(logger) {
this.saveBufferLogger = logger;
};
/*
Log a message
*/
Logger.prototype.log = function(/* args */) {
if(this.enable && console !== undefined && console.log !== undefined) {
return Function.apply.call(console.log, console, [$tw.utils.terminalColour(this.colour),this.componentName + ":"].concat(Array.prototype.slice.call(arguments,0)).concat($tw.utils.terminalColour()));
}
var self = this;
if(this.enable) {
if(this.saveBufferLogger.save) {
this.saveBufferLogger.buffer += $tw.utils.formatDateString(new Date(),"YYYY MM DD 0hh:0mm:0ss.0XXX") + ":";
$tw.utils.each(Array.prototype.slice.call(arguments,0),function(arg,index) {
self.saveBufferLogger.buffer += " " + arg;
});
this.saveBufferLogger.buffer += "\n";
this.saveBufferLogger.buffer = this.saveBufferLogger.buffer.slice(-this.saveBufferLogger.saveLimit);
}
if(console !== undefined && console.log !== undefined) {
return Function.apply.call(console.log, console, [$tw.utils.terminalColour(this.colour),this.componentName + ":"].concat(Array.prototype.slice.call(arguments,0)).concat($tw.utils.terminalColour()));
}
}
};
/*
Read the message buffer
*/
Logger.prototype.getBuffer = function() {
return this.saveBufferLogger.buffer;
};
/*
@ -70,6 +97,7 @@ Logger.prototype.alert = function(/* args */) {
component: this.componentName
};
existingCount = 0;
this.alertCount += 1;
}
alertFields.modified = new Date();
if(++existingCount > 1) {
@ -87,6 +115,22 @@ Logger.prototype.alert = function(/* args */) {
}
};
/*
Clear outstanding alerts
*/
Logger.prototype.clearAlerts = function() {
var self = this;
if($tw.browser && this.alertCount > 0) {
$tw.utils.each($tw.wiki.getTiddlersWithTag(ALERT_TAG),function(title) {
var tiddler = $tw.wiki.getTiddler(title);
if(tiddler.fields.component === self.componentName) {
$tw.wiki.deleteTiddler(title);
}
});
this.alertCount = 0;
}
};
exports.Logger = Logger;
})();

View File

@ -783,4 +783,22 @@ exports.strEndsWith = function(str,ending,position) {
}
};
/*
Return system information useful for debugging
*/
exports.getSystemInfo = function(str,ending,position) {
var results = [],
save = function(desc,value) {
results.push(desc + ": " + value);
};
if($tw.browser) {
save("User Agent",navigator.userAgent);
save("Online Status",window.navigator.onLine);
}
if($tw.node) {
save("Node Version",process.version);
}
return results.join("\n");
};
})();

View File

@ -1246,9 +1246,9 @@ exports.getTiddlerText = function(title,defaultText) {
if(!tiddler) {
return defaultText;
}
if(tiddler.fields.text !== undefined) {
if(!tiddler.hasField("_is_skinny")) {
// Just return the text if we've got it
return tiddler.fields.text;
return tiddler.fields.text || "";
} else {
// Tell any listeners about the need to lazily load this tiddler
this.dispatchEvent("lazyLoad",title);

View File

@ -0,0 +1,9 @@
title: $:/core/templates/html-div-skinny-tiddler
<!--
This template is a variant of $:/core/templates/html-div-tiddler used for saving skinny tiddlers (with no text field)
-->`<div`<$fields template=' $name$="$encoded_value$"'></$fields>`>
<pre></pre>
</div>`

View File

@ -3,4 +3,7 @@ title: $:/core/save/lazy-all
\define saveTiddlerFilter()
[is[system]] -[prefix[$:/state/popup/]] -[[$:/HistoryList]] -[[$:/boot/boot.css]] -[type[application/javascript]library[yes]] -[[$:/boot/boot.js]] -[[$:/boot/bootprefix.js]] +[sort[title]]
\end
\define skinnySaveTiddlerFilter()
[!is[system]]
\end
{{$:/core/templates/tiddlywiki5.html}}

View File

@ -3,4 +3,7 @@ title: $:/core/save/lazy-images
\define saveTiddlerFilter()
[is[tiddler]] -[prefix[$:/state/popup/]] -[[$:/HistoryList]] -[[$:/boot/boot.css]] -[type[application/javascript]library[yes]] -[[$:/boot/boot.js]] -[[$:/boot/bootprefix.js]] -[!is[system]is[image]] +[sort[title]]
\end
\define skinnySaveTiddlerFilter()
[is[image]]
\end
{{$:/core/templates/tiddlywiki5.html}}

View File

@ -3,6 +3,7 @@ title: $:/core/templates/store.area.template.html
<$reveal type="nomatch" state="$:/isEncrypted" text="yes">
`<div id="storeArea" style="display:none;">`
<$list filter=<<saveTiddlerFilter>> template="$:/core/templates/html-div-tiddler"/>
<$list filter={{{ [<skinnySaveTiddlerFilter>] }}} template="$:/core/templates/html-div-skinny-tiddler"/>
`</div>`
</$reveal>
<$reveal type="match" state="$:/isEncrypted" text="yes">

View File

@ -2,10 +2,12 @@ title: $:/core/ui/AlertTemplate
<div class="tc-alert">
<div class="tc-alert-toolbar">
<$button class="tc-btn-invisible"><$action-deletetiddler $tiddler=<<currentTiddler>>/>{{$:/core/images/delete-button}}</$button>
<$button class="tc-btn-invisible"><$action-deletetiddler $tiddler=<<currentTiddler>>/>{{$:/core/images/cancel-button}}</$button>
</div>
<div class="tc-alert-subtitle">
<$view field="component"/> - <$view field="modified" format="date" template="0hh:0mm:0ss DD MM YYYY"/> <$reveal type="nomatch" state="!!count" text=""><span class="tc-alert-highlight">({{$:/language/Count}}: <$view field="count"/>)</span></$reveal>
<$wikify name="format" text=<<lingo Tiddler/DateFormat>>>
<$view field="component"/> - <$view field="modified" format="date" template=<<format>>/> <$reveal type="nomatch" state="!!count" text=""><span class="tc-alert-highlight">({{$:/language/Count}}: <$view field="count"/>)</span></$reveal>
</$wikify>
</div>
<div class="tc-alert-body">

View File

@ -0,0 +1,2 @@
title: $:/config/Server/ExternalFilters/[all[tiddlers]!is[system]sort[title]]
text: yes

View File

@ -1,5 +1,5 @@
created: 20130825162100000
modified: 20140814094907624
modified: 20200113094126878
tags: dev moduletypes
title: SyncAdaptorModules
type: text/vnd.tiddlywiki
@ -14,6 +14,8 @@ SyncAdaptorModules encapsulate storage mechanisms that can be used by the SyncMe
SyncAdaptorModules are represented as JavaScript tiddlers with the field `module-type` set to `syncadaptor`.
See [[this pull request|https://github.com/Jermolene/TiddlyWiki5/pull/4373]] for background on the evolution of this API.
! Exports
The following properties should be exposed via the `exports` object:
@ -47,12 +49,21 @@ Gets the supplemental information that the adaptor needs to keep track of for a
Returns an object storing any additional information required by the adaptor.
!! `getTiddlerRevision(title)`
Gets the revision ID associated with the specified tiddler title.
|!Parameter |!Description |
|title |Tiddler title |
Returns a revision ID.
!! `getStatus(callback)`
Retrieves status information from the server. This method is optional.
|!Parameter |!Description |
|callback |Callback function invoked with parameters `err,isLoggedIn,username` |
|callback |Callback function invoked with parameters `err,isLoggedIn,username,isReadOnly` |
!! `login(username,password,callback)`
@ -70,16 +81,39 @@ Attempts to logout of the server. This method is optional.
|!Parameter |!Description |
|callback |Callback function invoked with parameter `err` |
!! `getUpdatedTiddlers(syncer,callback)`
Retrieves the titles of tiddlers that need to be updated from the server.
This method is optional. If an adaptor doesn't implement it then synchronisation will be unidirectional from the TiddlyWiki store to the adaptor, but not the other way.
The syncer will use the `getUpdatedTiddlers()` method in preference to the `getSkinnyTiddlers()` method.
|!Parameter |!Description |
|syncer |Reference to the syncer object making the call |
|callback |Callback function invoked with parameter `err,data` -- see below |
The data provided by the callback is as follows:
```
{
modifications: [<array of title>],
deletions: [<array of title>],
}
```
!! `getSkinnyTiddlers(callback)`
Retrieves a list of skinny tiddlers from the server.
This method is optional. If an adaptor doesn't implement it then synchronisation will be unidirectional from the TiddlyWiki store to the adaptor, but not the other way.
The syncer will use the `getUpdatedTiddlers()` method in preference to the `getSkinnyTiddlers()` method.
|!Parameter |!Description |
|callback |Callback function invoked with parameter `err,tiddlers`, where `tiddlers` is an array of tiddler field objects |
!! `saveTiddler(tiddler,callback,tiddlerInfo)`
!! `saveTiddler(tiddler,callback)`
Saves a tiddler to the server.
@ -96,11 +130,16 @@ Loads a tiddler from the server.
|title |Title of tiddler to be retrieved |
|callback |Callback function invoked with parameter `err,tiddlerFields` |
!! `deleteTiddler(title,callback,tiddlerInfo)`
!! `deleteTiddler(title,callback,options)`
Delete a tiddler from the server.
|!Parameter |!Description |
|title |Title of tiddler to be deleted |
|callback |Callback function invoked with parameter `err` |
|options |See below |
The options parameter contains the following properties:
|!Property |!Description |
|tiddlerInfo |The tiddlerInfo maintained by the syncer for this tiddler |

View File

@ -1,5 +1,5 @@
created: 20130825213300000
modified: 20191013093910961
modified: 20191206152031468
tags: Concepts
title: TiddlerFields
type: text/vnd.tiddlywiki
@ -42,11 +42,13 @@ Other fields used by the core are:
|`subtitle` |<<lingo subtitle>> |
|`throttle.refresh` |<<lingo throttle.refresh>> |
|`toc-link`|<<lingo toc-link>>|
|`_canonical_uri`|<<lingo _canonical_uri>>|
The TiddlyWebAdaptor uses a few more fields:
|!Field Name |!Description |
|`bag` |<<lingo bag>> |
|`revision` |<<lingo revision>> |
|`_is_skinny` |<<lingo _is_skinny>> |
Details of the fields used in this ~TiddlyWiki are shown in the [[control panel|$:/ControlPanel]] {{$:/core/ui/Buttons/control-panel}} under the <<.controlpanel-tab Info>> tab >> <<.info-tab Advanced>> sub-tab >> Tiddler Fields

View File

@ -1,5 +1,5 @@
created: 20181002131215403
modified: 20190903094711346
modified: 2020031109590546
tags: [[WebServer API]]
title: WebServer API: Get All Tiddlers
type: text/vnd.tiddlywiki
@ -12,11 +12,23 @@ GET /recipes/default/tiddlers.json
Parameters:
* none
* ''filter'' - filter identifying tiddlers to be returned (optional, defaults to "[all[tiddlers]!is[system]sort[title]]")
* ''exclude'' - comma delimited list of fields to excluded from the returned tiddlers (optional, defaults to "text")
In order to avoid denial of service attacks with malformed filters in the default configuration the only filter that is accepted is the default filter "[all[tiddlers]!is[system]sort[title]]"; attempts to use any other filter will result in an HTTP 403 error.
To enable a particular filter, create a tiddler with the title "$:/config/Server/ExternalFilters/" concatenated with the filter text, and the text field set to "yes". For example, the TiddlyWeb plugin includes the following shadow tiddler to enable the filter that it requires:
```
title: $:/config/Server/ExternalFilters/[all[tiddlers]] -[[$:/isEncrypted]] -[prefix[$:/temp/]] -[prefix[$:/status/]]
text: yes
```
It is also possible to configure the server to accept any filter by creating a tiddler titled $:/config/Server/AllowAllExternalFilters with the text "yes". This should not be done for public facing servers.
Response:
* 200 OK
*> `Content-Type: application/json`
*> Body: array of all non-system tiddlers in [[TiddlyWeb JSON tiddler format]]
* 403 Forbidden

View File

@ -1,6 +1,6 @@
caption: port
created: 20180630180552254
modified: 20180702155017130
modified: 20191219123751824
tags: [[WebServer Parameters]]
title: WebServer Parameter: port
type: text/vnd.tiddlywiki
@ -10,6 +10,7 @@ The [[web server configuration parameter|WebServer Parameters]] ''port'' specifi
The ''port'' parameter accepts two types of value:
* Numerical values are interpreted as a decimal port number
** The special value 0 (zero) causes the operating system to assign an available port
* Non-numeric values are interpreted as an environment variable from which the port should be read
This example configures the server to listen on port 8090:

View File

@ -26,6 +26,8 @@ function FileSystemAdaptor(options) {
FileSystemAdaptor.prototype.name = "filesystem";
FileSystemAdaptor.prototype.supportsLazyLoading = false;
FileSystemAdaptor.prototype.isReady = function() {
// The file system adaptor is always ready
return true;

View File

@ -1,20 +0,0 @@
title: $:/plugins/tiddlywiki/tiddlyweb/ServerControlPanel
caption: Server
tags: $:/tags/ControlPanel
<$reveal state="$:/status/IsLoggedIn" type="nomatch" text="yes">
Log in to ~TiddlyWeb: <$button message="tm-login">Login</$button>
</$reveal>
<$reveal state="$:/status/IsLoggedIn" type="match" text="yes">
Logged in as {{$:/status/UserName}} <$button message="tm-logout">Logout</$button>
</$reveal>
----
Host configuration: <$edit-text tiddler="$:/config/tiddlyweb/host" tag="input" default=""/>
<blockquote>//for example, `$protocol$//$host$/folder`, where `$protocol$` is replaced by the protocol (typically `http` or `https`), and `$host$` by the host name//</blockquote>
----
<$button message="tm-server-refresh">Refresh</$button> to fetch changes from the server immediately

View File

@ -0,0 +1,2 @@
title: $:/config/Server/ExternalFilters/[all[tiddlers]] -[[$:/isEncrypted]] -[prefix[$:/temp/]] -[prefix[$:/status/]]
text: yes

View File

@ -0,0 +1,3 @@
title: $:/config/OfficialPluginLibrary
(This core tiddler is overridden by the tiddlyweb plugin to prevent users from installing official plugins via control panel. Instead they should be installed by editing tiddlywiki.info in the root of the wiki folder)

View File

@ -0,0 +1,7 @@
title: $:/core/templates/css-tiddler
<!--
This template is used for saving CSS tiddlers as a style tag with data attributes representing the tiddler fields. This version includes the tiddler changecount as the field `revision`.
-->`<style`<$fields template=' data-tiddler-$name$="$encoded_value$"'></$fields>` data-tiddler-revision="`<<changecount>>`" data-tiddler-bag="default" type="text/css">`<$view field="text" format="text" />`</style>`

View File

@ -0,0 +1,9 @@
title: $:/core/templates/html-div-skinny-tiddler
<!--
This template is a variant of the tiddlyweb plugin's overridden version of $:/core/templates/html-div-tiddler used for saving skinny tiddlers (with no text field)
-->`<div`<$fields exclude='text revision bag' template=' $name$="$encoded_value$"'></$fields>` revision="`<<changecount>>`" bag="default" _is_skinny="">
<pre></pre>
</div>`

View File

@ -0,0 +1,4 @@
title: $:/plugins/tiddlywiki/tiddlyweb/icon/cloud
tags: $:/tags/Image
<svg class="tc-image-cloud tc-image-button" width="22pt" height="22pt" viewBox="0 0 128 128"><g><path d="M24 103C10.745 103 0 92.255 0 79c0-9.697 5.75-18.05 14.027-21.836A24.787 24.787 0 0114 56c0-13.255 10.745-24 24-24 1.373 0 2.718.115 4.028.337C48.628 24.2 58.707 19 70 19c19.882 0 36 16.118 36 36v.082c12.319 1.016 22 11.336 22 23.918 0 12.239-9.16 22.337-20.999 23.814L107 103H24z"/><path class="tc-image-cloud-idle" d="M57.929 84.698a6 6 0 01-8.485 0L35.302 70.556a6 6 0 118.485-8.485l9.9 9.9L81.97 43.686a6 6 0 018.485 8.486L57.929 84.698z"/><path class="tc-image-cloud-progress tc-animate-rotate-slow" d="M44.8 40a3.6 3.6 0 100 7.2h2.06A23.922 23.922 0 0040 64c0 13.122 10.531 23.785 23.603 23.997L64 88l.001-7.2c-9.171 0-16.626-7.348-16.798-16.477L47.2 64c0-5.165 2.331-9.786 5.999-12.868L53.2 55.6a3.6 3.6 0 107.2 0v-12a3.6 3.6 0 00-3.6-3.6h-12zM64 40v7.2c9.278 0 16.8 7.522 16.8 16.8 0 5.166-2.332 9.787-6 12.869V72.4a3.6 3.6 0 10-7.2 0v12a3.6 3.6 0 003.6 3.6h12a3.6 3.6 0 100-7.2l-2.062.001A23.922 23.922 0 0088 64c0-13.255-10.745-24-24-24z"/></g></svg>

View File

@ -0,0 +1,7 @@
title: $:/core/templates/javascript-tiddler
<!--
This template is used for saving JavaScript tiddlers as a script tag with data attributes representing the tiddler fields. This version includes the tiddler changecount as the field `revision`.
-->`<script`<$fields template=' data-tiddler-$name$="$encoded_value$"'></$fields>` data-tiddler-revision="`<<changecount>>`" data-tiddler-bag="default" type="text/javascript">`<$view field="text" format="text" />`</script>`

View File

@ -2,6 +2,6 @@ title: $:/plugins/tiddlywiki/tiddlyweb/save/offline
\import [[$:/core/ui/PageMacros]] [all[shadows+tiddlers]tag[$:/tags/Macro]!has[draft.of]]
\define saveTiddlerFilter()
[is[tiddler]] -[[$:/boot/boot.css]] -[[$:/HistoryList]] -[type[application/javascript]library[yes]] -[[$:/boot/boot.js]] -[[$:/boot/bootprefix.js]] -[[$:/plugins/tiddlywiki/filesystem]] -[[$:/plugins/tiddlywiki/tiddlyweb]] +[sort[title]] $(publishFilter)$
[is[tiddler]] -[[$:/boot/boot.css]] -[[$:/HistoryList]] -[type[application/javascript]library[yes]] -[[$:/boot/boot.js]] -[[$:/boot/bootprefix.js]] -[[$:/plugins/tiddlywiki/filesystem]] -[[$:/plugins/tiddlywiki/tiddlyweb]] -[prefix[$:/temp/]] +[sort[title]] $(publishFilter)$
\end
{{$:/core/templates/tiddlywiki5.html}}

View File

@ -0,0 +1,25 @@
title: $:/core/ui/Buttons/save-wiki
tags: $:/tags/PageControls
caption: {{$:/plugins/tiddlywiki/tiddlyweb/icon/cloud}} Server status
description: Status of synchronisation with server
\define config-title()
$:/config/PageControlButtons/Visibility/$(listItem)$
\end
<$button popup=<<qualify "$:/state/popup/save-wiki">> tooltip="Status of synchronisation with server" aria-label="Server status" class=<<tv-config-toolbar-class>> selectedClass="tc-selected">
<span class="tc-dirty-indicator">
<$list filter="[<tv-config-toolbar-icons>match[yes]]">
{{$:/plugins/tiddlywiki/tiddlyweb/icon/cloud}}
</$list>
<$list filter="[<tv-config-toolbar-text>match[yes]]">
<span class="tc-btn-text"><$text text="Server status"/></span>
</$list>
</span>
</$button>
<$reveal state=<<qualify "$:/state/popup/save-wiki">> type="popup" position="below" animate="yes">
<div class="tc-drop-down">
<$list filter="[all[shadows+tiddlers]tag[$:/tags/SyncerDropdown]!has[draft.of]]" variable="listItem">
<$transclude tiddler=<<listItem>>/>
</$list>
</div>
</$reveal>

View File

@ -0,0 +1,40 @@
title: $:/plugins/tiddlywiki/tiddlyweb/styles
tags: [[$:/tags/Stylesheet]]
\rules only filteredtranscludeinline transcludeinline macrodef macrocallinline macrocallblock
body.tc-dirty span.tc-dirty-indicator svg {
transition: fill 250ms ease-in-out;
}
body .tc-image-cloud-idle {
fill: <<colour background>>;
transition: opacity 250ms ease-in-out;
opacity: 1;
}
body.tc-dirty .tc-image-cloud-idle {
opacity: 0;
}
body .tc-image-cloud-progress {
transition: opacity 250ms ease-in-out;
transform-origin: 50% 50%;
transform: rotate(359deg);
animation: animation-rotate-slow 2s infinite linear;
fill: <<colour background>>;
opacity: 0;
}
body.tc-dirty .tc-image-cloud-progress {
opacity: 1;
}
@keyframes animation-rotate-slow {
from {
transform: rotate(0deg);
}
to {
transform: scale(359deg);
}
}

View File

@ -0,0 +1,6 @@
title: $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/copy-logs
tags: $:/tags/SyncerDropdown
<$button message="tm-copy-syncer-logs-to-clipboard" class="tc-btn-invisible">
{{$:/core/images/copy-clipboard}} Copy syncer logs to clipboard
</$button>

View File

@ -0,0 +1,9 @@
title: $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/login-status
tags: $:/tags/SyncerDropdown
<$reveal state="$:/status/IsLoggedIn" type="match" text="yes">
<div class="tc-drop-down-info">
You are logged in<$reveal state="$:/status/UserName" type="nomatch" text="" default=""> as <strong><$text text={{$:/status/UserName}}/></strong></$reveal><$reveal state="$:/status/IsReadOnly" type="match" text="yes" default="no"> (read-only)</$reveal>
</div>
<hr/>
</$reveal>

View File

@ -0,0 +1,8 @@
title: $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/login
tags: $:/tags/SyncerDropdown
<$reveal state="$:/status/IsLoggedIn" type="nomatch" text="yes">
<$button message="tm-login" class="tc-btn-invisible">
{{$:/core/images/unlocked-padlock}} Login
</$button>
</$reveal>

View File

@ -0,0 +1,8 @@
title: $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/logout
tags: $:/tags/SyncerDropdown
<$reveal state="$:/status/IsLoggedIn" type="match" text="yes">
<$button message="tm-logout" class="tc-btn-invisible">
{{$:/core/images/cancel-button}} Logout
</$button>
</$reveal>

View File

@ -0,0 +1,9 @@
title: $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/refresh
tags: $:/tags/SyncerDropdown
<$reveal state="$:/status/IsLoggedIn" type="match" text="yes">
<$button tooltip="Get latest changes from the server" aria-label="Refresh from server" class="tc-btn-invisible">
<$action-sendmessage $message="tm-server-refresh"/>
{{$:/core/images/refresh-button}} <span clas]s="tc-btn-text"><$text text="Get latest changes from the server"/></span>
</$button>
</$reveal>

View File

@ -0,0 +1,9 @@
title: $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/save-snapshot
tags: $:/tags/SyncerDropdown
<$button class="tc-btn-invisible">
<$wikify name="site-title" text={{$:/config/SaveWikiButton/Filename}}>
<$action-sendmessage $message="tm-download-file" $param={{$:/config/SaveWikiButton/Template}} filename=<<site-title>>/>
</$wikify>
{{$:/core/images/download-button}} Save snapshot for offline use
</$button>

View File

@ -0,0 +1,2 @@
title: $:/tags/SyncerDropdown
list: $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/login-status $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/login $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/refresh $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/logout $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/save-snapshot $:/plugins/tiddlywiki/tiddlyweb/syncer-actions/copy-logs

View File

@ -27,6 +27,12 @@ function TiddlyWebAdaptor(options) {
TiddlyWebAdaptor.prototype.name = "tiddlyweb";
TiddlyWebAdaptor.prototype.supportsLazyLoading = true;
TiddlyWebAdaptor.prototype.setLoggerSaveBuffer = function(loggerForSaving) {
this.logger.setSaveBuffer(loggerForSaving);
};
TiddlyWebAdaptor.prototype.isReady = function() {
return this.hasStatus;
};
@ -50,6 +56,11 @@ TiddlyWebAdaptor.prototype.getTiddlerInfo = function(tiddler) {
};
};
TiddlyWebAdaptor.prototype.getTiddlerRevision = function(title) {
var tiddler = this.wiki.getTiddler(title);
return tiddler.fields.revision;
};
/*
Get the current status of the TiddlyWeb connection
*/
@ -147,6 +158,9 @@ TiddlyWebAdaptor.prototype.getSkinnyTiddlers = function(callback) {
var self = this;
$tw.utils.httpRequest({
url: this.host + "recipes/" + this.recipe + "/tiddlers.json",
data: {
filter: "[all[tiddlers]] -[[$:/isEncrypted]] -[prefix[$:/temp/]] -[prefix[$:/status/]]"
},
callback: function(err,data) {
// Check for errors
if(err) {
@ -220,7 +234,7 @@ TiddlyWebAdaptor.prototype.deleteTiddler = function(title,callback,options) {
return callback(null);
}
// If we don't have a bag it means that the tiddler hasn't been seen by the server, so we don't need to delete it
var bag = options.tiddlerInfo.adaptorInfo.bag;
var bag = options.tiddlerInfo.adaptorInfo && options.tiddlerInfo.adaptorInfo.bag;
if(!bag) {
return callback(null);
}

View File

@ -1450,6 +1450,10 @@ html body.tc-body.tc-single-tiddler-window {
fill: <<colour foreground>>;
}
.tc-drop-down .tc-drop-down-info {
padding-left: 14px;
}
.tc-drop-down p {
padding: 0 14px 0 14px;
}
@ -1978,24 +1982,26 @@ html body.tc-body.tc-single-tiddler-window {
.tc-alerts {
position: fixed;
top: 0;
top: 28px;
left: 0;
max-width: 500px;
right: 0;
max-width: 50%;
z-index: 20000;
}
.tc-alert {
position: relative;
margin: 28px;
padding: 14px 14px 14px 14px;
border: 2px solid <<colour alert-border>>;
margin: 14px;
padding: 7px;
border: 1px solid <<colour alert-border>>;
background-color: <<colour alert-background>>;
}
.tc-alert-toolbar {
position: absolute;
top: 14px;
right: 14px;
top: 7px;
right: 7px;
line-height: 0;
}
.tc-alert-toolbar svg {
@ -2005,6 +2011,12 @@ html body.tc-body.tc-single-tiddler-window {
.tc-alert-subtitle {
color: <<colour alert-muted-foreground>>;
font-weight: bold;
font-size: 0.8em;
margin-bottom: 0.5em;
}
.tc-alert-body > p {
margin: 0;
}
.tc-alert-highlight {