Commit Graph

76 Commits

Author SHA1 Message Date
Jeremy Ruston 9d94459c5d Syncer: fix object reference
We should be passing the syncer object, not the task object
2023-12-12 15:48:09 +00:00
Jeremy Ruston 622512c380 Further reduce syncer logging
The rationale is that the deeper logs are only useful for debugging the syncer logic, and are overwhelming for most users
2023-11-29 11:24:54 +00:00
Jeremy Ruston fe17f16675 Fix syncer not exiting when used on CLI
Fixes #7867
2023-11-29 09:31:19 +00:00
Jeremy Ruston 3b84088b27 Syncer: Reduce logging intensity 2023-11-28 11:44:21 +00:00
Jeremy Ruston 1cb607249e
Fix syncer race condition (#7843)
* Initial commit

* Log task choosing

* A tiny bit more logging

* Typo

* Restructure syncer to use a single state machine
2023-11-24 13:02:09 +00:00
jeremy@jermolene.com 8ca0bf10e4 Lazy loading a tiddler should trigger a sync
Fixes #7138
2023-01-03 14:28:48 +00:00
Joshua Fontany 99249a3160
Fix syncer logout alert (#5936)
* call self.displayError

* Revert "call self.displayError"

This reverts commit 5d599aa979.

* fix Syncer logout alert
2021-08-05 14:50:22 +01:00
Joshua Fontany 0413c3a38e
Fix syncer error handling for getStatus (#5914) 2021-07-30 16:21:02 +01:00
Arlen22 be026aa308
Revert "Add server sent events (#5279)" (#5880)
This reverts commit 17b4f53ba2 according to Github Desktop.

git checkout that commit
revert commit in GitHub Desktop
git switch -c revert-sse
uncommit in Github Desktop
switch to master, bringing changes
resolve deletions with command line
2021-07-14 17:16:57 +01:00
jeremy@jermolene.com 0b71f25f74 Revert "Update sync methods (#5467)"
This reverts commit 8d7930f660.

See the discussion at https://github.com/Jermolene/TiddlyWiki5/pull/5467#issuecomment-873590578https://github.com/Jermolene/TiddlyWiki5/pull/5467#issuecomment-873590578
2021-07-05 19:26:20 +01:00
Joshua Fontany 8d7930f660
Update sync methods (#5467) 2021-05-24 21:16:23 +01:00
Joshua Fontany bfa062f23d
Fix filesystem (#5465) 2021-02-04 16:11:07 +00:00
Nicolas Petton 17b4f53ba2
Add server sent events (#5279)
* Create server-sent-events.js

* Create sse-change-listener.js

* Implement server sent events

* Convert to ES5 and wrap in function

* Use the host string from tiddlyweb

* Improve comments in sse-server.js

* Can't use object reference as key

* Add retry timeout

* Fix a bug

* bug fix

* Fix formatting

* Fix ES5 compat

* capitalize comments

* more fixes

* Refactor tiddlywek/sse-server.js

* Extract helper functions for handling wikis and connections.
* Replace JSDoc comments.
* Fix formatting according to TW core.
* Simplify the logic for adding and removing connections.

* Fix formatting of tiddlyweb/sse-client.js

Fix formatting according to TW core.

* Fix formatting of server-sent-events.js

Fix formatting and comments following TW core guidelines.

* Extract a debounce function in sse-client.js

* Avoid using startsWith in server-sent-events.js

startsWith is part of ES2015, while TiddlyWiki uses the 5.1 dialect.

* New sse-enabled WebServer parameter

* If not set to "yes", disabled SSE request handling.
* Add documentation for the parameter in core/language/en-GB/Help/listen.tid
* Add new tiddler editions/tw5.com/tiddlers/webserver/WebServer Parameter_ sse-enabled.tid

* Disable polling for changes if SSE is enabled

* Add sse_enabled to /status JSON response
* Store syncer polling status in $:/config/SyncDisablePolling
* Handled disabling polling in core/modules/syncer.js

* Simply boolean logic in syncer.js

* Delete trailing whitespaces in syncer.js

Co-authored-by: Arlen22 <arlenbee@gmail.com>
2021-01-15 10:37:55 +00:00
Joshua Fontany dde4182830
Fix filesystem adaptor (#5113)
* ignore .env

testing new implementation

almost there

closer

bug, desyncing

fixed

final testing

final testing

cleanup

cleanup

* isEditableFile flow fixed

* removed `basepath` logic

* callback to delete title from $tw.boot.files

* comment fix

* have syncer delete from boot.files

* syntax

* bugfix: error on missing directory

* bugifx

* remove !draft check

* fix relative filepaths

* cleanup

* cleanup !draft

* catch undefined filepaths in deleteTiddlerFile()

* typo

* whitelist wiki dir, encodeURIComponent otherwise

* test for wikiPath, not wikiPath/tiddlers

* don't need to .normailze()

* whitelist wiki directory, move cleanup to util

* use cleanup util & fail EPERM & EACCESS gracefully

* comments

* final bugs fixed

* improved sync error
2020-11-30 22:31:48 +00:00
saqimtiaz e2bea854b6
System tiddler syncing (#4987)
* First pass at a fix for system tiddler sync issues

* Add new filter syncFromServer

* Undo previous attempt at a fix

* Added a flag to control sync of system tiddlers from server
2020-11-19 15:59:02 +00:00
jeremy@jermolene.com f7f55e8eff Add support for username/password parameters for tm-login message 2020-10-25 16:33:44 +00:00
jeremy@jermolene.com 69c12618d9 Syncer: Allow syncadaptor to customise the login prompt 2020-10-14 12:41:51 +01:00
Arlen22 81f07cdf85
Use this.wiki where applicable (#4601)
* Use state.wiki

* use local wiki where available

* fix a couple this vs self cases
2020-05-06 11:27:50 +01:00
jeremy@jermolene.com 2649d2f7dc Syncer: Fix typos
Fixes a bug introduced in #4584
2020-04-20 13:19:27 +01:00
jeremy@jermolene.com 15d7255728 Syncer: Should not call syncadaptor for missing tiddlers
Fixes #4580
2020-04-20 11:35:11 +01:00
Daniel Rodríguez Rivero 4ea79cb0c3
fix(syncer): restore tiddlerInfo param to saveTiddler (#4584)
This was added on a159b5baf3 and lost in #4373.
Will be a good idea to introduce tests against this kind of changes
2020-04-20 11:17:27 +01:00
Jeremy Ruston 1154372a7b Update syncer to distinguish connection errors from other errors
We can automatically remove connection errors when things resume working
2020-03-31 10:47:17 +01:00
Jeremy Ruston b95723a022
Fix syncer to handler errors properly (#4373)
* First commit

* Add throttling of saves

Now we refuse to save a tiddler more often than once per second.

* Wait for a timeout before trying again after an error

* Modest optimisations of isDirty() method

* Synchronise system tiddlers and deletions from the server

Fixes two long-standing issues:

* Changes to system tiddlers are not synchronised from the server to the browser
* Deletions of tiddlers on the server are not propagated to browser clients

* Make sure we update the dirty status even if there isn't a task to perform

* Replace save-wiki button with popup sync menu

* Remove the "Server" control panel tab

We don't need it with the enhanced sync dropdown

* Add indentation to the save-wiki button

* Fix spacing in dropdown menu items

* Switch between cloud icons according to dirty status

* Add a menu item to copy syncer logs to the clipboard

* Improve animated icon

* Remove indentation from save-wiki button

@pmario the annoying thing is that using `\trim whitespace` trims significant whitespace too, so it means we have to use <$text text=" "/> when we need a space that won't be trimmed. For the moment, I've removed the indentation but will keep thinking about it.

* Further icon, UI and copy text tweaks

Move the icons and styles from the core into the TiddlyWeb plugin

* Clean up PR diff

* Tweak animation durations

* Break the actions from the syncer dropdown into separate tiddlers

@pmario I think this makes things a bit easier to follow

* Refactor syncadaptor creation and logging

The goal is for the syncadaptor to be able to log to the same logger as the syncer, so that the "copy syncer logs to clipboard" data is more useful.

* Don't transition the dirty indicator container colour, just the SVG's colour

* Only trigger a sync for changes to tiddlers we're interested in

Otherwise it is triggered by the creation of the alert tiddlers used to display errors.

* Restore deleting local tiddlers removed from the server

(I had commented it out for some testing and accidentally commited it).

* Guard against missing adaptor info

* We still need to trigger a timeout when there was no task to process

* Avoid repeatedly polling for changes

Instead we only trigger a timeout call at if there is a pending task (ie a tiddler that has changed but isn't yet old enough to save).

* Lazy loading: include skinny versions of lazily loaded tiddlers in the index.html

* Introduce _is_skinny field for indicating that a tiddler is subject to lazy loading

* Remove savetrail plugin from prerelease

It doesn't yet work with the new syncer

* Make the savetrail plugin work again

* Clear outstanding alerts when synchronisation is restored

* Logger: only remove alerts from the same component

Missed off 9f5c0de07

* Make the saving throttle interval configurable (#4385)

After switching Bob to use the core syncer the throttle interval makes saving feel very sluggish compared to the message queue setup that I had before.
The editing lock that I use to prevent conflicts with multiple users doesn't go away until the save is completed, and with the 1 second delay it means that if you edit a tiddler and save it than you have to wait one second before you can edit it again.

* Tweaks to appearance of alerts

* Exclude temp tiddlers from offline snapshots

Otherwise alerts will persist

* Tweak appearance of status line in dropdown

* Update release note

* Web server: Don't include full path in error messages

Fixes #3724

* In change event handler check for deletions

* Disable the official plugin library when the tiddlyweb plugin is loaded

* Hide error details from browser for /files/ route

See https://github.com/Jermolene/TiddlyWiki5/issues/3724#issuecomment-565702492 -- thanks @pmario

* Revert all the changes to the relationship between the syncer and the syncadaptor

Previously we had some major rearrangements to make it possible for the syncadaptor to route it's logging to the logger used by the syncer. The motivation is so that the "copy logs to clipboard" button is more useful.

On reflection, changing the interface this drastically is undesirable from a backwards compatibility perspective, so I'm going to investigate other ways to achieve the logger sharing

* Make the tiddlyweb adaptor use the syncer's logger

So that both are availavble when copying the syncer logs to the clipboard

* Update release note

* Support setting port=0 to get an OS assigned port

Quite useful

* Update code comment

* UI: Use "Get latest changes from server" instead of "Refresh"

* Add getUpdatedTiddlers() method to syncadaptor API

See https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-573579495

* Refactor revision handling within the syncer

Thanks @pmario

* Fix typo in tiddlywebadaptor

* Improve presentation of errors

See https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-573695267

* Add docs for getTiddlerRevision()

* Remove unused error animation

* Update comment for GET /recipes/default/tiddlers/tiddlers.json

* Optimise SVG cloud image

* Add optional list of allowed filters for get all tiddlers route

An attempt to address @Arlen22's concern here:

https://github.com/Jermolene/TiddlyWiki5/pull/4373#pullrequestreview-342146190

* Fix network error alert text translatability

* Fix error code and logging for GET /recipes/default/tiddlers/tiddlers.json

Thanks @Arlen22

* Flip GET /recipes/default/tiddlers/tiddlers.json allowed filter handling to be secure by default

* Validate updates received from getUpdatedTiddlers()

* Add syncer method to force loading of a tiddler from the server

* Remove the release note update to remove the merge conflict

* Fix crash when there's no config section in the tiddlywiki.info file

* Use config tiddler title to check filter query (merge into fix-syncer) (#4478)

* Use config tiddler title to check filter query

* Create config-tiddlers-filter.tid

* Add config switch to enable all filters on GET /recipes/default/tiddlers/tiddlers.json

And update docs

* Fix bug when deleting a tiddler with a shadow

Reported by @kookma at https://github.com/Jermolene/TiddlyWiki5/pull/4373#issuecomment-604027528

Co-authored-by: jed <inmysocks@fastmail.com>
Co-authored-by: Arlen22 <arlenbee@gmail.com>
2020-03-30 15:24:05 +01:00
Jeremy Ruston 43d18e74d5 Syncer: add hidden setting for disabling lazy loading 2019-10-25 10:02:57 +01:00
Jermolene 6b03ba9876 Syncer: support configurable polling interval 2018-10-19 16:32:23 +01:00
Jermolene d50e2df57b Syncer: fix problem with incoming tiddlers
The bug here is that incoming tiddlers (ie tiddlers that were updated on the server and synced back to the browser) will retain any fields that are currently present but deleted in the incoming tiddler
2018-10-14 15:35:26 +01:00
snlhnk c05c0d3df6 Module-ize server routes, add static file support and other enhancements(#2679)
* Module-ize server routes and add static file support (#2510)

* Refactor server routes to modules

New module type: serverroute

Caveats: Loading order is not deterministic but this would only matter
if two route modules attempted to use the same path regexp (that would
be silly).

* Add static assets plugin

This plugin allows the node server to fetch static assets in the /assets
directory. I felt that this was a feature that goes above the core
functionality. That is why I added it as a plugin. with the modular
route extensions this was a breeze.

* Add serverroute description to ModuleTypes

* Coding standards tweaks

* Fix filename typo

* Move support for attachments from a plugin into the core

* Missing "else"

* Refactor server handling

* Introduce a new named parameter scheme for commands
* Move the SimpleServer class into it's own module
* Deprecate the --server command because of the unwieldy syntax
* Add a new --listen command using the new syntax

For example:

tiddlywiki mywiki --listen host:0.0.0.0 port:8090

* Add check for unknown parameters

* Add support for multiple basic authentication credentials in a CSV file

Beware: Passwords are stored in plain text. If that's a problem, use an authenticating proxy and the trusted header authentication approach.

* Refactor module locations

* Rename "serverroute" module type to "route"

* Remove support for verifying optional named command parameters

The idea was to be able to flag unknown parameter names, but requiring a command to pre-specify all the parameter names makes it harder for (say) the listen command to be extensible so that plugins can add new optional parameters that they handle. (This is particularly in the context of work in progress to encapsulate authenticators into their own modules).

* Refactor the two authenticators into separate modules and add support for authorization

* Correct mistaken path.join vs. path.resolve

See https://stackoverflow.com/a/39836259

* Docs for the named command parameters

I'd be grateful if anyone with sufficient Windows experience could confirm that the note about double quotes in "NamedCommandParameters" is correct.

* Be consistent about lower case parameter names

* Do the right thing when we have a username but no password

With a username parameter but no password parameter we'll attribute edits to that username, but not require authentication.

* Remove obsolete code

* Add support for requiring authentication without restricting the username

* Refactor authorization checks

* Return read_only status in /status response

* Fix two code typos

* Add basic support for detecting readonly status and avoiding write errors

We now have syncadaptors returning  readonly status and avoid attempting to write to the server if it's going to fail

* Add readonly-styles

We hide editing-related buttons in read only mode

I've made this part of the tiddlyweb plugin but I think a case could be made for putting it into the core.

* Add custom request header as CSRF mitigation

By default we require the header X-Requested-With to be set to TiddlyWiki. Can be overriden by setting csrfdisable to "yes"

See https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)_Prevention_Cheat_Sheet#Protecting_REST_Services:_Use_of_Custom_Request_Headers

* Add support for HTTPS

* First pass at a route for serving rendered tiddlers

cc @Drakor

* Tweaks to the single tiddler static view

Adding a simple sidebar

* Switch to "dash" separated parameter names

* Typo

* Docs: Update ServerCommand and ListenCommand

* First pass at docs for the new web server stuff

Writing the docs is turning out to be quite an undertaking, much harder than writing the code!

* Get rid of extraneous paragraphs in static renderings

* Rejig anonymous user handling

Now we can support wikis that are read-only for anonymous access, but allow a user to login for read/write access.

* More docs

Slowly getting there...

* Static tiddler rendering: Fix HTML content in page title

* Docs updates

* Fix server command parameter names

Missed off 30ce7ea

* Docs: Missing quotes

* Avoid inadvertent dependency on Node.js > v9.6.0

The listenOptions parameter of the plain HTTP version of CreateServer was only introduced in v9.6.0

cc @Drakor @pmario

* Typo
2018-07-18 16:54:43 +01:00
Daniel Rodríguez Rivero 4c89bbabbe Retain $:/status/UserName when logged out (#2893)
The user name tiddler should not be removed just because we are not logged in
2018-05-09 14:22:55 +01:00
Jermolene ab5e5795e8 Fix issue with lazy loading temporary tiddlers
Fixes #3235
2018-05-03 18:27:17 +01:00
Jermolene 05a3d9a302 Syncer: allow logging to be disabled
We don't want the syncer used by the savetrail plugin to be logging
2017-09-26 17:10:57 +01:00
Jermolene 73953080e3 SaveTrailPlugin: Fix syncer UI interactions
The syncer used by the savetrail plugin shouldn't perform any UI interactions
2017-09-20 16:28:11 +01:00
Jermolene 51b1ead5c9 Add more colour to command line output 2017-09-04 14:55:12 +01:00
Jermolene db056a84a5 Remove obsolete code 2017-02-04 18:14:20 +00:00
Jermolene 6c65aa2a6d Make the syncer more configurable, including names for sync adaptors
@danielo515 you may want to add a name to your sync adaptor 😄
2017-02-04 17:25:30 +00:00
Jermolene 18dd8d4433 Fix problem with lazy loading under Node.js
Fixes #2514
2016-08-16 15:48:14 +01:00
Jermolene 7326a3a4cc Only call isReady() for sync adaptors that support it
Fixes #2522
2016-08-06 14:45:33 +01:00
Jermolene c3d18364c1 Don't use syncadaptors until they are ready
Fixes #2453
2016-07-05 11:29:59 +01:00
Myeongjin 6388f859d1 Add localisable strings for error (#2447)
* Add localisable strings for error

* Update tiddler Transclusion Basic Usage in fr-FR
2016-05-17 21:58:47 +01:00
Myeongjin 8afc1e9354 Add localisable strings (#2438)
* Add localisable string for recursive transclusion error

* Add localisable string for count

* Add localisable strings for syncer
2016-05-12 08:44:28 -06:00
Jermolene 1e0d019610 Don't repeatedly lazy load the same tiddler
If we don’t get a “text” field back the first time we shouldn’t keep
asking for it.
2016-04-18 14:50:13 +01:00
Jermolene 48dcf959ff Refactor browser unload task handling
Make it possible to register multiple task functions that will be
called when the window is unloaded
2015-05-03 16:23:35 +01:00
Jermolene 3657bbe920 Switch from addEventListener to onbeforeunload
This makes it possible for TiddlyDesktop to reuse these functions to
handle detecting unsaved changes
2015-01-10 18:08:12 +00:00
Jermolene eacb9e53eb Delete syncer tiddlerInfo when tiddlers are deleted 2014-09-24 16:21:59 +02:00
Jermolene a159b5baf3 Add tiddlerInfo to the syncadaptor saveTiddler method 2014-09-24 13:15:16 +02:00
Jermolene 2f6d16b0fa Improve comments and coding style 2014-09-01 20:39:08 +01:00
Jermolene 3a67fdb768 Obeisance to JSHint for core modules
There are still some warnings about making functions in a loop, but
I’ll fix those as a separate pull request because the fixes are more
than typographic errors.
2014-08-30 20:44:26 +01:00
Jermolene 2f69ea362c Rename "tw-*" messages to "tm-*" 2014-08-28 21:43:44 +01:00
Jermolene 787481a804 Change more "tw-*" classes to "tc-*"
Part of #764
2014-08-28 18:21:08 +01:00
Jermolene f75af2c983 Separate the saver handling out of the syncer 2014-08-14 11:43:07 +01:00
Jermolene 27f1f82a70 Rejigging syncer structuring
The goal is to separate out the saver handling from the syncadaptor
handling; it will take a few steps to get there
2014-08-14 11:12:25 +01:00