mirrors/TiddlyWiki5
1
0
Fork 0
mirror of https://github.com/Jermolene/TiddlyWiki5 synced 2024-11-23 10:07:19 +00:00
Code Issues Releases Wiki Activity

Replace Markdown parsing library with Remarkable (#3876)

Browse Source 
* Replace Markdown parsing library with Remarkable

* Fix handling of block-level elements

* Update documentation

* Add config options for Parser actions

* Add Config options for Remarkable library

* Match code style

* Update documentation

* Handle ordered lists and horizontal rules

* Update to v2.0.0 of Remarkable library
This commit is contained in:
Brooks Boyd 2020-01-20 07:13:36 -06:00 committed by Jeremy Ruston
parent 7795ed9230
commit b258afea0d
16 changed files with 261 additions and 1856 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
editions/markdowndemo/tiddlers/HelloThere.tid
View File

@ -1,6 +1,6 @@
title: HelloThere
This is a demo of TiddlyWiki5 incorporating a plugin for the [[markdown-js|https://github.com/evilstreak/markdown-js]] Markdown parser from Dominic Baggott. The MarkdownExample tiddler below is written in Markdown.
This is a demo of TiddlyWiki5 incorporating a plugin for parsing tiddlers written in the Markdown language. The plugin uses the [[Remarkable|https://github.com/jonschlinkert/remarkable]] Markdown parser internally. The MarkdownExample tiddler below is written in Markdown.
! Installation
@ -8,31 +8,4 @@ To add the plugin to your own TiddlyWiki5, just drag this link to the browser wi
[[$:/plugins/tiddlywiki/markdown]]
! Markdown Dialects
By default the markdown parser recognises the original dialect of Markdown [[as described by John Gruber|http://daringfireball.net/projects/markdown/]]. An extended dialect called "Maruku" is also included that provides table support and other advanced features. The syntax extensions are modelled on those of [[PHP Markdown Extra|https://michelf.ca/projects/php-markdown/extra/]].
The configuration tiddler [[$:/config/markdown/dialect]] determines which dialect is used:
|!Dialect |!Description |
|Gruber |Standard Markdown |
|Maruku |Extended Maruku Markdown |
! Creating ~WikiLinks
Create wiki links with the usual Markdown link syntax targeting `#` and the target tiddler title:
```
[link text](#TiddlerTitle)
```
! Images
Markdown image syntax can be used to reference images by tiddler title or an external URI. For example:
```
![alt text](/path/to/img.jpg "Title")
![alt text](Motovun Jack.jpg "Title")
```
{{$:/plugins/tiddlywiki/markdown/usage}}

41
editions/markdowndemo/tiddlers/MarkdownExample.tid
View File

@ -19,11 +19,10 @@ web application that allows you type your own Markdown-formatted text
and translate it to XHTML.
**Note:** This document is itself written using Markdown; you
can [see the source for it by adding '.text' to the URL] [src].
can see the source for it by editing this tiddler.
[s]: /projects/markdown/syntax "Markdown Syntax"
[d]: /projects/markdown/dingus "Markdown Dingus"
[src]: /projects/markdown/basics.text
[s]: https://daringfireball.net/projects/markdown/syntax "Markdown Syntax"
[d]: https://daringfireball.net/projects/markdown/dingus "Markdown Dingus"
## Paragraphs, Headers, Blockquotes ##
@ -47,7 +46,7 @@ Markdown:
A First Level Header
====================
A Second Level Header
---------------------
@ -57,11 +56,11 @@ Markdown:
The quick brown fox jumped over the lazy
dog's back.
### Header 3
> This is a blockquote.
>
>
> This is the second paragraph in the blockquote.
>
> ## This is an H2 in a blockquote
@ -70,23 +69,23 @@ Markdown:
Output:
<h1>A First Level Header</h1>
<h2>A Second Level Header</h2>
<p>Now is the time for all good men to come to
the aid of their country. This is just a
regular paragraph.</p>
<p>The quick brown fox jumped over the lazy
dog's back.</p>
<h3>Header 3</h3>
<blockquote>
<p>This is a blockquote.</p>
<p>This is the second paragraph in the blockquote.</p>
<h2>This is an H2 in a blockquote</h2>
</blockquote>
@ -100,7 +99,7 @@ Markdown:
Some of these words *are emphasized*.
Some of these words _are emphasized also_.
Use two asterisks for **strong emphasis**.
Or, if you prefer, __use two underscores instead__.
@ -108,10 +107,10 @@ Output:
<p>Some of these words <em>are emphasized</em>.
Some of these words <em>are emphasized also</em>.</p>
<p>Use two asterisks for <strong>strong emphasis</strong>.
Or, if you prefer, <strong>use two underscores instead</strong>.</p>
## Lists ##
@ -164,7 +163,7 @@ list item text. You can create multi-paragraph list items by indenting
the paragraphs by 4 spaces or 1 tab:
* A list item.
With multiple paragraphs.
* Another item in the list.
@ -176,7 +175,7 @@ Output:
<p>With multiple paragraphs.</p></li>
<li><p>Another item in the list.</p></li>
</ul>
### Links ###
@ -271,7 +270,7 @@ Output:
<p>I strongly recommend against using any
<code>&lt;blink&gt;</code> tags.</p>
<p>I wish SmartyPants used named entities like
<code>&amp;mdash;</code> instead of decimal-encoded
entites like <code>&amp;#8212;</code>.</p>
@ -294,7 +293,7 @@ Output:
<p>If you want your page to validate under XHTML 1.0 Strict,
you've got to put paragraph tags in your blockquotes:</p>
<pre><code>&lt;blockquote&gt;
&lt;p&gt;For example.&lt;/p&gt;
&lt;/blockquote&gt;

3
plugins/tiddlywiki/markdown/config_breaks.tid Normal file
View File

@ -0,0 +1,3 @@
title: $:/config/markdown/breaks
false

3
plugins/tiddlywiki/markdown/config_linkNewWindow.tid Normal file
View File

@ -0,0 +1,3 @@
title: $:/config/markdown/linkNewWindow
true

3
plugins/tiddlywiki/markdown/config_linkify.tid Normal file
View File

@ -0,0 +1,3 @@
title: $:/config/markdown/linkify
false

3
plugins/tiddlywiki/markdown/config_quotes.tid Normal file
View File

@ -0,0 +1,3 @@
title: $:/config/markdown/quotes
“”‘’

3
plugins/tiddlywiki/markdown/config_renderWikiText.tid Normal file
View File

@ -0,0 +1,3 @@
title: $:/config/markdown/renderWikiText
true

3
plugins/tiddlywiki/markdown/config_renderWikiTextPragma.tid Normal file
View File

@ -0,0 +1,3 @@
title: $:/config/markdown/renderWikiTextPragma
\rules only html image macrocallinline syslink transcludeinline wikilink filteredtranscludeblock macrocallblock transcludeblock

3
plugins/tiddlywiki/markdown/config_typographer.tid Normal file
View File

@ -0,0 +1,3 @@
title: $:/config/markdown/typographer
false

3
plugins/tiddlywiki/markdown/dialect.tid
View File

@ -1,3 +0,0 @@
title: $:/config/markdown/dialect
Gruber

1725
plugins/tiddlywiki/markdown/files/markdown.js
View File

File diff suppressed because it is too large Load Diff

1
plugins/tiddlywiki/markdown/files/remarkable.js Normal file
View File

File diff suppressed because one or more lines are too long

4
plugins/tiddlywiki/markdown/files/tiddlywiki.files
View File

@ -1,10 +1,10 @@
{
"tiddlers": [
{
"file": "markdown.js",
"file": "remarkable.js",
"fields": {
"type": "application/javascript",
"title": "$:/plugins/tiddlywiki/markdown/markdown.js",
"title": "$:/plugins/tiddlywiki/markdown/remarkable.js",
"module-type": "library"
}
}

2
plugins/tiddlywiki/markdown/readme.tid
View File

@ -1,6 +1,6 @@
title: $:/plugins/tiddlywiki/markdown/readme
This is a TiddlyWiki plugin for parsing Markdown text, based on the [[markdown-js|https://github.com/evilstreak/markdown-js]] project from Dominic Baggott.
This is a TiddlyWiki plugin for parsing Markdown text, using the [[Remarkable|https://github.com/jonschlinkert/remarkable]] library.
It is completely self-contained, and doesn't need an Internet connection in order to work. It works both in the browser and under Node.js.

24
plugins/tiddlywiki/markdown/usage.tid
View File

@ -1,15 +1,15 @@
title: $:/plugins/tiddlywiki/markdown/usage
! Markdown Dialects
By default the markdown parser recognises the original dialect of Markdown [[as described by John Gruber|http://daringfireball.net/projects/markdown/]]. An extended dialect called "Maruku" is also included that provides table support and other advanced features. The syntax extensions are modelled on those of [[PHP Markdown Extra|https://michelf.ca/projects/php-markdown/extra/]].
The configuration tiddler [[$:/config/markdown/dialect]] determines which dialect is used:
|!Dialect |!Description |
|Gruber |Standard Markdown |
|Maruku |Extended Maruku Markdown |
! Plugin Configuration
|!Config |!Default |!Description |
| <code>[[breaks|$:/config/markdown/breaks]]</code>| ``false``|Remarkable library config: Convert '\n' in paragraphs into ``<br>`` |
| <code>[[linkify|$:/config/markdown/linkify]]</code>| ``false``|Remarkable library config: Autoconvert URL-like text to links |
| <code>[[linkNewWindow|$:/config/markdown/linkNewWindow]]</code>| ``true``|For external links, should clicking on them open a new window/tab automatically? |
| <code>[[quotes|$:/config/markdown/quotes]]</code>| ``“”‘’``|Remarkable library config: Double + single quotes replacement pairs, when ``typographer`` enabled |
| <code>[[renderWikiText|$:/config/markdown/renderWikiText]]</code>| ``true``|After Markdown is parsed, should any text elements be handed off to the ~WikiText parser for further processing? |
| <code>[[renderWikiTextPragma|$:/config/markdown/renderWikiTextPragma]]</code>| ``\rules only html image macrocallinline syslink transcludeinline wikilink filteredtranscludeblock macrocallblock transcludeblock``|When handing off to the ~WikiText parser, what pragma rules should it follow? |
| <code>[[typographer|$:/config/markdown/typographer]]</code>| ``false``|Remarkable library config: Enable some language-neutral replacement + quotes beautification |
! Creating ~WikiLinks
@ -19,6 +19,12 @@ Create wiki links with the usual Markdown link syntax targeting `#` and the targ
[link text](#TiddlerTitle)
```
If the target tiddler has a space in its name, that name must be URL-escaped to be detected as a URL:
```
[link text](#Test%20Tiddler)
```
! Images
Markdown image syntax can be used to reference images by tiddler title or an external URI. For example:

265
plugins/tiddlywiki/markdown/wrapper.js
View File

@ -3,7 +3,7 @@ title: $:/plugins/tiddlywiki/markdown/wrapper.js
type: application/javascript
module-type: parser
Wraps up the markdown-js parser for use in TiddlyWiki5
Wraps up the remarkable parser for use as a Parser in TiddlyWiki
\*/
(function(){
@ -12,78 +12,211 @@ Wraps up the markdown-js parser for use in TiddlyWiki5
/*global $tw: false */
"use strict";
var markdown = require("$:/plugins/tiddlywiki/markdown/markdown.js");
const { Remarkable, linkify, utils } = require("$:/plugins/tiddlywiki/markdown/remarkable.js");
var CONFIG_DIALECT_TIDDLER = "$:/config/markdown/dialect",
DEFAULT_DIALECT = "Gruber";
function transformNodes(nodes) {
var results = [];
for(var index=0; index<nodes.length; index++) {
results.push(transformNode(nodes[index]));
}
return results;
///// Set up configuration options /////
function parseAsBoolean(tiddlerName) {
return $tw.wiki.getTiddlerText(tiddlerName).toLowerCase() === "true";
}
function transformNode(node) {
if($tw.utils.isArray(node)) {
var p = 0,
widget = {type: "element", tag: node[p++]};
if(!$tw.utils.isArray(node[p]) && typeof(node[p]) === "object") {
widget.attributes = {};
$tw.utils.each(node[p++],function(value,name) {
widget.attributes[name] = {type: "string", value: value};
});
}
widget.children = transformNodes(node.slice(p++));
// Massage images into the image widget
if(widget.tag === "img") {
widget.type = "image";
if(widget.attributes.alt) {
widget.attributes.tooltip = widget.attributes.alt;
delete widget.attributes.alt;
}
if(widget.attributes.src) {
widget.attributes.source = widget.attributes.src;
delete widget.attributes.src;
}
}
// Convert internal links to proper wikilinks
if (widget.tag === "a" && widget.attributes.href.value[0] === "#") {
widget.type = "link";
widget.attributes.to = widget.attributes.href;
if (widget.attributes.to.type === "string") {
//Remove '#' before conversion to wikilink
widget.attributes.to.value = widget.attributes.to.value.substr(1);
}
//Children is fine
delete widget.tag;
delete widget.attributes.href;
}
return widget;
} else {
return {type: "text", text: node};
}
}
var MarkdownParser = function(type,text,options) {
var dialect = options.wiki.getTiddlerText(CONFIG_DIALECT_TIDDLER,DEFAULT_DIALECT),
markdownTree = markdown.toHTMLTree(text,dialect),
node = $tw.utils.isArray(markdownTree[1]) ? markdownTree.slice(1) : markdownTree.slice(2);
this.tree = transformNodes(node);
var pluginOpts = {
linkNewWindow: parseAsBoolean("$:/config/markdown/linkNewWindow"),
renderWikiText: parseAsBoolean("$:/config/markdown/renderWikiText"),
renderWikiTextPragma: $tw.wiki.getTiddlerText("$:/config/markdown/renderWikiTextPragma").trim()
};
var remarkableOpts = {
breaks: parseAsBoolean("$:/config/markdown/breaks"),
quotes: $tw.wiki.getTiddlerText("$:/config/markdown/quotes"),
typographer: parseAsBoolean("$:/config/markdown/typographer")
};
/*
var md = new Remarkable(remarkableOpts);
[ 'html',
[ 'p', 'something' ],
[ 'h1',
'heading and ',
[ 'strong', 'other' ] ] ]
if (parseAsBoolean("$:/config/markdown/linkify")) {
md = md.use(linkify);
}
*/
function findTagWithType(nodes, startPoint, type, level) {
for (var i = startPoint; i < nodes.length; i++) {
if (nodes[i].type === type && nodes[i].level === level) {
return i;
}
}
return false;
}
/**
* Remarkable creates nodes that look like:
* [
* { type: 'paragraph_open'},
* { type: 'inline', content: 'Hello World', children:[{type: 'text', content: 'Hello World'}]},
* { type: 'paragraph_close'}
* ]
*
* But TiddlyWiki wants the Parser (https://tiddlywiki.com/dev/static/Parser.html) to emit nodes like:
*
* [
* { type: 'element', tag: 'p', children: [{type: 'text', text: 'Hello World'}]}
* ]
*/
function convertNodes(remarkableTree, isStartOfInline) {
let out = [];
function wrappedElement(elementTag, currentIndex, currentLevel, closingType, nodes) {
var j = findTagWithType(nodes, currentIndex + 1, closingType, currentLevel);
if (j === false) {
console.error("Failed to find a " + closingType + " node after position " + currentIndex);
console.log(nodes);
return currentIndex + 1;
}
let children = convertNodes(nodes.slice(currentIndex + 1, j));
out.push({
type: "element",
tag: elementTag,
children: children
});
return j;
}
for (var i = 0; i < remarkableTree.length; i++) {
var currentNode = remarkableTree[i];
if (currentNode.type === "paragraph_open") {
i = wrappedElement("p", i, currentNode.level, "paragraph_close", remarkableTree);
} else if (currentNode.type === "heading_open") {
i = wrappedElement("h" + currentNode.hLevel, i, currentNode.level, "heading_close", remarkableTree);
} else if (currentNode.type === "bullet_list_open") {
i = wrappedElement("ul", i, currentNode.level, "bullet_list_close", remarkableTree);
} else if (currentNode.type == 'ordered_list_open') {
i = wrappedElement('ol', i, currentNode.level,'ordered_list_close', remarkableTree);
} else if (currentNode.type === "list_item_open") {
i = wrappedElement("li", i, currentNode.level, "list_item_close", remarkableTree);
} else if (currentNode.type === "link_open") {
var j = findTagWithType(remarkableTree, i + 1, "link_close", currentNode.level);
if (currentNode.href[0] !== "#") {
// External link
var attributes = {
href: { type: "string", value: currentNode.href }
};
if (pluginOpts.linkNewWindow) {
attributes.target = { type: "string", value: "_blank" };
}
out.push({
type: "element",
tag: "a",
attributes: attributes,
children: convertNodes(remarkableTree.slice(i + 1, j))
});
} else {
// Internal link
out.push({
type: "link",
attributes: {
to: { type: "string", value: decodeURI(currentNode.href.substr(1)) }
},
children: convertNodes(remarkableTree.slice(i + 1, j))
});
}
i = j;
} else if (currentNode.type.substr(currentNode.type.length - 5) === "_open") {
var tagName = currentNode.type.substr(0, currentNode.type.length - 5);
i = wrappedElement(tagName, i, currentNode.level, tagName + "_close", remarkableTree);
} else if (currentNode.type === "code") {
out.push({
type: "element",
tag: currentNode.block ? "pre" : "code",
children: [{ type: "text", text: currentNode.content }]
});
} else if (currentNode.type === "fence") {
out.push({
type: "codeblock",
attributes: {
language: { type: "string", value: currentNode.params },
code: { type: "string", value: currentNode.content }
}
});
} else if (currentNode.type === "image") {
out.push({
type: "image",
attributes: {
tooltip: { type: "string", value: currentNode.alt },
source: { type: "string", value: currentNode.src }
}
});
} else if (currentNode.type === "softbreak") {
out.push({
type: "element",
tag: "br",
});
} else if (currentNode.type == 'hr') {
out.push({
type: 'element',
tag: 'hr',
});
} else if (currentNode.type === "inline") {
out = out.concat(convertNodes(currentNode.children, true));
} else if (currentNode.type === "text") {
if (!pluginOpts.renderWikiText) {
out.push({
type: "text",
text: currentNode.content
});
} else {
// The Markdown compiler thinks this is just text.
// Hand off to the WikiText parser to see if there's more to render
// If we're inside a block element (div, p, td, h1), and this is the first child in the tree,
// handle as a block-level parse. Otherwise not.
var parseAsInline = !(isStartOfInline && i === 0);
var textToParse = currentNode.content;
if (pluginOpts.renderWikiTextPragma !== "") {
textToParse = pluginOpts.renderWikiTextPragma + "\n" + textToParse;
}
var wikiParser = $tw.wiki.parseText("text/vnd.tiddlywiki", textToParse, {
parseAsInline: parseAsInline
});
var rs = wikiParser.tree;
// If we parsed as a block, but the root element the WikiText parser gave is a paragraph,
// we should discard the paragraph, since the way Remarkable nests its nodes, this "inline"
// node is always inside something else that's a block-level element
if (!parseAsInline
&& rs.length === 1
&& rs[0].type === "element"
&& rs[0].tag === "p"
) {
rs = rs[0].children;
}
// If the original text element started with a space, add it back in
if (rs.length > 0
&& rs[0].type === "text"
&& currentNode.content[0] === " "
) {
rs[0].text = " " + rs[0].text;
}
out = out.concat(rs);
}
} else {
console.error("Unknown node type: " + currentNode.type, currentNode);
out.push({
type: "text",
text: currentNode.content
});
}
}
return out;
}
var MarkdownParser = function(type, text, options) {
var tree = md.parse(text, {});
//console.debug(tree);
tree = convertNodes(tree);
//console.debug(tree);
this.tree = tree;
};
exports["text/x-markdown"] = MarkdownParser;
})();