Tools for working with Amazon Web Services:
* Templates for saving a TiddlyWiki as a single JavaScript file in a
ZIP file that can be executed as an AWS Lambda function. In this form,
TiddlyWiki is a self contained single file containing both code and
data, just like the standalone HTML file configuration
* Commands that can be used to interact with AWS services, under both
the Node.js and Lambda configurations of TiddlyWiki
We now use highlight.js in raw HTML mode on the server, rather than
trying to use it with the fakedom. This causes problems with fakedoms
inability to get textContent for a node that has been created by
assigning innerHTML. So we extend the fakedom to allow the original
text content to be saved.
See #2778 for discussion.
The code here had got a bit broken by some PRs that I should have
checked more carefully. I’ve done a major refactoring which will
hopefully make it easier to understand, and fixes a number of problems:
* Problem with eg .md tiddlers not being deleted correctly
* Problem with Windows path separators not being usable within
$:/config/FileSystemPaths on Windows
* Problem with filename clashes not being detected correctly when
saving to a different directory via $:/config/FileSystemPaths
* Enables slashes within tiddler titles to be mapped into folders
* Enables plain text files like .md and .css to be saved with .meta
files instead of as .tid files (see #2558)
* No longer replaces spaces with underscores
As this is such a major update, I’d be grateful if Node.js users could
give it a careful run through — in particular, you’ll need to try
creating new tiddlers of various types and ensure that the expected
files are created.
See the readme:
“This plugin causes TiddlyWiki to continuously save the contents of
each tiddler that is changed as a JSON file. Configured correctly, the
browser will download the files silently in the background, and they
can be used as a backup in case of accidental data loss.”
Inspired by @telmiger’s comment (3) here:
https://github.com/Jermolene/TiddlyWiki5/issues/2741#issuecomment-276128
871
* trying to implement new googleanalytics tracker
* trying to put new google tracker. Not working...
* more dev options for testing
still don"t understand wants goes wrong
* New version. Seems to work
* achieved update for new tests
brought back tiddlywiki.com settings value
created a settings tab to make it easier
* adding settings to plugin
add a settings tab to plugin to make it easier to use and see which GA account is in use
* fixes bug with GA_ACCOUNT and GA_DOMAIN tiddlers containing newlines at their end, preventing plugins to work
* soft rebase on jermolene's master
* revert to oldest version of GA_account and GA_domain tiddlers - had been overwritten by ones with a new line at the end
* Integrates some @tobibeer comments
* googleanalytics.js : removed "rebranding", var declaration and console log. Did not manage to get a non-minified version of Google script. But as far as I can see, jermolene's original plugin did same way
* plugin.info : placed "readme" first
* readme : back to previous "legacy" version. No more mention to temporary fork. Added mention and link to google official code.
* signed CLA
Add support for skipping an entire tiddler if a particular column is
blank
Add support for reading a row by column, making each of the columns
into a fieldname.
Also significantly refactored the code to break up the main, monolithic
function.
Hi @welford I wondered if you could kindly review this commit, since
you authored the original code? Before this commit, I was running into
a crash when running `prerelease-bld.sh` from
`build.jermolene.github.io`, caused by using raw HTML for the
highlighted block. Switching to the fake dom seems to fix things, but
I’d like a second pair of eyes.
TiddlyWiki passes the MIME type of the tiddler to highlight.js as the
"language brush", but it turns out that highlight.js doesn't actually
understand MIME types. This commit introduces a configuration mapping
between common MIME types and highlight.js language brushes
Fixes#2535
We were using `String.prototype.replace()` without addressing the
wrinkle that dollar signs in the replacement string have special
handling. This caused problems in situations where the replacement
string is derived from user input and contains dollar signs.
Fixes#2517
Fixes#2503
This bug was introduced in commit c4c7b18 where it would append
additional .tid extensions to a file every time the node server was
restarted.
Here we check the filepath does not have the extension already before
appending it.
The plan had been to switch template depending on the content type, but
we’d only implemented support for saving wikitext tiddlers. That meant
that creating a tiddler with any non-wikitext content type failed to
write the file correctly under Node.js.
Now we just always save in .tid file format.
* Save binary tiddlers with meta file
The filesystemadaptor plugin was a little simplistic in its
understanding of a binary file. It was using the typeInfo dictionary to
choose what tiddler types were binary (and hence needed a meta file when
saving).
I looked as if it was trying to be smart by looking for the hasMetaFile
*OR* had the encoding of base64. Unfortunately the typeInfo only defined
image/jpeg and so any other base64 encoded tiddler was assumed to be of
type text/vnd.tiddlywiki.
The net effect was only JPG images got a meta file and everything else
were saved as .tid files with base64 encoding. It all still worked but
made working with binary data in a Git repo a bit daunting.
There is enough information in the $tw.config.contentTypeInfo to
determine if a tiddler type is encoded with base64 or not. A better list
is available from boot/boot.js who registers all the types thorough the
registerFileType and marks then with base64 were appropriate.
This commit uses the typeInfo dictionary first for any filesystem
specific overrides, then the contentTypeInfo, and finally defaults to
the typeInfo["text/vnd.tiddlywiki"]. It also eliminates the now
unnecessary override for image/jpeg.
I think this might have been the original intent from commit 10b192e7.
From my limited testing all files described in boot/boot.js (lines
1832-1856) with an encoding of base64 now save as the original binary
and a meta file. Meaning that when you start the node server and then
drag-n-drop a binary file (i.e. image/png) it will PUT to the server
and then save it on the filesystem as-is allowing the file to be managed
as a binary file and not a text file. (Binary diffs are better and
GitHub supports them as well).
* Prevent duplicate file extensions
A side effects of using the $tw.config.contentFileInfo in the previous
commit is that it will always append a file extension to the tiddler
title when saving. In most cases this is the correct course of action.
However, sometimes that title is already a proper filename with an
extension (for example importing 'foobar.png' would save a file named
'foobar.png.png') which seemed silly.
This commit simply checks to make sure the title does not already end
with the file extension before appending it to the filename. A little
convenience really.
Since IE apparently doesn't have the String endsWith method I took the
liberty to add a helper method to $tw.utils trying to follow the other
polyfill patterns. I figured this was more generic and readable then
attempting to use a one-off solution inline. I got the polyfill code
from MDN.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/endsWith#Polyfill
Is strEndsWith the best method name?
When saving new tiddlers on node.js, allow the user to override the path of the
generated .tid file. This is done by creating a tiddler
$:/config/FileSystemPaths which contains one or more filter expressions, one
per line. These filters are applied in turn to the tiddler to be saved, and
the first output produced is taken as a logical path relative to the wiki's
tiddlers directory. Any occurences of "/" in the logical path are replaced with
the platform's path separator, the extension ".tid" is appended, illegal
characters are replaced by "_" and the path is disambiguated (if necessary) in
order to arrive at the final tiddler file path. If none of the filters matches,
or the configuration tiddler does not exist, fall back to the previous file
naming scheme (i.e. replacing "/" by "_").
This implies we will now, for tiddlers matching the user-specified filters,
create directory trees below the tiddlers directory. In order to avoid
cluttering it with empty directory trees when renaming or removing tiddlers, any
directories that become empty by deleting a tiddler file are removed
(recursively).
Benefits of this configuration option include the ability to organize git
repositories of TiddlyWikis running on node.js, ability to replace characters
that cause trouble with particular operating systems or workflows (e.g. '$' on
unix) and the ability to replicate tiddler "paths" in the filesystem (by
including a filter like "[!has[draft.of]]") without forcing such a (potentially
problematic) change on all users.
* Add icon for view template segment
* Muted background colour for view template segment
* Include links to parent tiddlers in the view template segment
* Fix logic for hiding/showing documents/headings (“open”/“close” were
transposed)
Without modified dates, new imports do not appear in recent file list. The equivalent of modified dates in ENEX is the 'updated' tag, with a date format similar to that of TW with the exception that the date format consists of a date stamp, a letter T, and a time stamp followed by Z for UTC presumably. There is no millisecond indicator. Not all ENEX notes have an update tag. The solution here is to apply creation date by default and then apply the update date if it is available. Dates are converted by stripping out Z and T and appending '000'.
@sukima the main issue with the previous code was that it incorrectly
used comma to delimit tags. We actually use spaces, and double square
brackets to delimit tags containing spaces. Better is to leave the tags
field as an array; the core will serialise it correctly as required.
I also made some minor consistency tweaks.
Relates to Issue #2268
I tried to map over the list of tags but NodeLists are not arrays and so
need to be converted. This looks ugly and probably should be abstracted
to a function. Come to think of it should we have a `$tw.utils.map()`
function?
Relates to Issue #2268
Based in the [example XML][1] attachments are listed in the <resources>
node. Since in TiddlyWiki these would be media tiddlers I add then one
by one as separate tiddlers.
There are some things that still need to happen. There should be a mime
type check so we don't attempt to import media tha TiddlyWiki doesn't
support. Also the example suggests the data is base64 encoded so I
blindly use that for the text attribute. Should there be a
`data:mediatyp;base64,…` prefix?
[1]: https://gist.github.com/evernotegists/6116886
Integrating this module allows us to do HTML/XML parsing under Node.js
(there is no built-in support for Node.js; we can already do HTML/XML
parsing in the browser). The implementation chosen is pure JavaScript,
and will work in all configurations of TiddlyWiki.
The immediate motivation is the work I’m doing to integrate with Amazon
Web Services.
We should be able to use JSZip to export multiple tiddlers in a single
operation from the browser.
Instead of showing a sequence of all documents in the preview column,
we just show a single document, with a dropdown to choose which one is
shown.
The change makes it easier to deal with large numbers of documents.
The metadata for each field is in a tiddler tagged
`$:/tags/DocumentMetaData` with the field `caption` being the displayed
caption for the field and `field` being the name of the field.
`field-type` is the type of the field, and can be “string” or “list”.
Previously, newly created image files would end up being saved as a
base64-encoded .tid file. Now they are saved as an ordinary binary file
with an accompanying .meta file for the metadata.
By making the docs available as a plugin we make it easier for people
to fork their own copy of the docs without losing the ability to get
updates in the future (which can be done just be updating the docs
plugin)
Many reasons:
* to allow subtrees to be grafted more easily
* to keep the tags for an entry clean by removing structural tags and
leaving the semantic tags
* to avoid the duplication of expressing the same relationship through
both the tags and list fields
1. Switch from using the text field of lists for storing the associated
filter to using the field `toc-list-filter` (to make it harder to
accidentally parse the text of an ordinary tiddler as a filter)
2. Fix several bugs
The exclude filter `+[tag[intro]]` will produce a document that only
includes the paragraphs with the tag “intro”. These are derived from
the paragraphs in the original document with the CSS class “intro”.
* Add warning in document tiddler toolbar if tiddler already exists
* Live preview document in new window
* Fix slicer.js bug that was preventing the list field of headings from
being filled in correctly
* Rationalise some class names
Now includes a special document view column on the left. Headings can
be expanded/collapsed, and tiddler titles can be inspected and renamed
via the toolbar. Clicking on an entry opens the associated tiddler. The
default tiddler view template includes a special section for tiddlers
that are part of a document
Now we process the rendered HTML of tiddlers, which allows us to
process HTML generated by MS Word. In fact, the HTML that MS Word
generates is so awful, I’ve instead been using Mammoth to do the
conversion: https://github.com/mwilliamson/mammoth.js
Also some necessary improvements to the fake dom implementation.
1. Introduce template tiddlers for the document, each tiddler, and the
tiddler toolbar
2. Move the text slicer toolbar button to the left of the edit button
3. Add a selectable toolbar, currently just containing the tiddler title
We’d rather present this information as a panel within the tiddler
editor so that we can (for example), allow clicks on a snippet to
insert it automatically into the editor at the current cursor position.
For the moment, we’ll just remove the cheatsheet. Post-5.1.8 we’ll add
it back to the editor
* Moved “add new plugin” into a modal wizard
* Adopt big friendly buttons
* Add plugin icons and readmes to “add new plugin” modal
* Use tabs for splitting plugins/themes/languages
* Consistent styling between the “add new plugin” modal and the
“installed plugins” control panel tab
* Behind the scenes, moved from addressing the library as
`recipes/defaults/tiddlers/<etc>` to `recipes/library/tiddlers<etc>`
There was a rendering mess in the CodeMirror readme file. This was cause
by attempting to bold the `$:/tags/Stylesheet` text. Also the context of
the sentence implies this should be backticked not bold (it refers to a
tag not a tiddler).
This is just an idea really.
The help plugin is only included in the tw5.com wiki; the idea is to
also include it in the empty.html wiki, either as a plugin or as part
of the core.
If it stays as a plugin, the idea would be to have different help
plugins for different languages.
Fixes#1450
Provides support for an integrated plugin library that can be used to
install plugins from tiddlywiki.com directly to wikis hosted online or
offline. See the Plugins tab of Control Panel.
Todo:
* Error checking(eg libraryserver.js HTTP GET)
* Translatability
* Documentation
** $:/tags/ServerConnection
** savelibrarytiddlers command
split the latter half of highlight.pack.js into a separate file which
registers all the languages and exports hljs. this is for nodejs
compatibility and mimics the way the highlightjs module works in nodejs.
Without sorting the list, we had ordinary tiddlers listed before shadow
tiddlers. That meant that as soon as you started to type in a shadow
tiddler it moved up in the list.
* Added support for translating the core readme
* Fixed problem with `Misc.multids` rendering shadow tiddlers
* Fixed problem with single tiddler files rendering shadow tiddlers
The user interface needs some refinements but it shows the basic
principles.
The idea is that the translator would email their modified file, which
would then be merged into the core repo (some additional tools will be
helpful for this).
Fixes#984
This code was contributed by @natecain and added in #176. It was almost
immediately disabled because of problems in the field.
I’m removing the code now to simplify the adaptor in advance of some
planned refactoring.
Fixes#855 - albeit it’s a bit of a kluge. The internal format returned
by `markdown.toHTMLTree()` isn’t sufficiently well documented for me to
be confident that the new code is resilient to all the things that
might be returned.
The control panel isn’t the right place for tools; it’s a place for
settings and internal configuration.
Once again apologies to the translators for wiping out your hard work!
Fixing problems caused by c4b76ceb0b:
* We still need to initialise the saver-handler even when syncing to a
server, otherwise offline snapshots can’t be saved
* We need to override the default save template a bit further up the
stack, to avoid the server side serving the offline version of the wiki
at `/`
Now the usual “save changes” button in the sidebar will save an offline
copy of the wiki that excludes the TiddlyWeb plugin. Previously, this
functionality was only available in the control panel, leading to
several problems such as that discussed here:
https://groups.google.com/d/topic/tiddlywikidev/U61pO-TR854/discussion
Importing an encrypted wiki ordinarily doesn’t place the password in
the password vault on the basis that one ought to be able to import
from a file without automatically inheriting its password.
Now there’s a configuration option that can be used by the upgrade
plugin to cause the password vault to be updated with any password
entered by the user. The end result is that the user only needs to
enter their password once.
This addon is already included into the plugin, but is not used, neither mentioned how to use it. It is a very useful but lightweight addon. I consider it is better to include the full functionality of the current configuration in the config example.
By rearranging the `[all[]]` operator we are able to ensure that shadow
tiddlers get processed before ordinary tiddlers. This makes it easier
to create custom stylesheets that override the core.
Now we get decent visual indication of sync errors, for instance. Still
work to do to coalesce alerts so that the screen doesn’t fill up with
them after an interval. And probably we should add a button to clear
all alerts.
We re-use some of the existing syncer mechanism. It was already keeping
track of changes to tiddlers in the store when working with a tiddler
syncadaptor. Now it also tracks changes when there is no syncadaptor,
allowing us to provide a warning if there are unsaved changes.
The current implementation of the highlight plugin only works properly
in the browser, and doesn’t work under Node.js. It also doesn’t work
when rendering to the fakedom in the browser (as happens when rendering
stylesheets, for example).
These tiddlers were being handled as `<script>` tags to make it easier
to debug them. But in fact modern dev tools are quite happy to debug
code that has been eval’d, and this arrangement was causing problems
for importing.
Fixes#335
