Creating easier to use unified wiki markup for release 2.0
Contents
Problems with current markup
refactor, collect more links to related bugs, feature requests and user observations. This part should be short and clear: what are the REAL problem that we need to solve?
There are also comments or complaints about Moin's WikiMarkup. These can be defined into some categories:
- features which do exist, but in their current markup cause confusion
- finnish abbreviations use colon
- include versus link versus an inclusion which also offers a link
- exactly how should we invoke Parser, Processor or Macro anyway? which one's which? should a user have to care?
Markup should be selected because we have decided it is reasonable, by our own reasons, not just because someone else thought so. In short if we cannot express the reasoning here I would not see the point of making a change.
There is another page, MarkupProposals, which is heavily edited (hooray for the wiki way) but suffers the problem of trying to be all three of those, with a bit of online help page thrown in. It also starts off abrasively, with professorial diagrams and a decidedly non-neutral point of view.
There are also people reporting feeling lost in the wealth of our HelpContents pages.
So, it's clear the refactoring under and over the hood as it were, need to happen at the same time. We need to unify the markup... so the new, unified markup becomes part of the design too.
Related work
There are currently efforts to unify a number of things:
UnifyPagesAndAttachments - unifying their storage mechanism and simplifying their use
UnifyParsersAndMacros - simplifying extensions using unified markup and code
Parsers, Processors and Macros are called sometimes PPM, or simply Extensions. See also: FeatureRequests/PPMCalledFromPragma.
Goals for new markup
replace most of the text with common goals from the proposal page. It must be short and clear: without clear goals we can't evaluate any proposal.
Unifying the markup needs the following traits:
- It needs to make sense. Not the "I think everyone in the world can possibly guess this" kind of sense, that may not be possible. But at least we should decide what our reasons are, and that some things that are very similar from the user's point of view, should also be using similar markup.
- Coders should discuss here how a particular markup redesign will affect the underlying code-design.
I'd like this page to describe specific suggested markup, in a style rather similar to that found in the various HelpOnEditing pages, with discussion following, in the style of the discussion about any possible new MoinDev/MoinMoinLogo.
Initial examples are from the IRC log that spurred me to write this up, more is up to you.
How to tell if it makes sense?
When we've decided for ourselves why it does make sense, people can read these defining thoughts at the top of the related HelpContents page and not feel nearly so lost as they do today.
How can we tell if it can make sense so easily? Try to describe it in plain language to people whose idea of "computer savvy" is that they can find their web browser icon without being told what Earth look like. For a good test I recommend trying to say your description out loud, and suppose your audience is such an ordinary person.
In this regard all wiki are imperfect. It was also well noted on the IRC channel that redefining the markup can actually effect how the code under the hood looks - in order to handle it. Also, if it can't be coded,efficiently, it's possible that describing it won't be so effective either.
Link vs. Include
We were talking about merging the SectionParser, and it wandered in the direction of what code makes sense or not. Yes, this is very long. I have ellided intervening threads about different topics and people popping in and out with the assorted hello and bye noises.
moin_markup_ideas_irc_29apr05.txt
Hmm. This means we have three things to consider:
- plain links
- inclusions purely for display
inclusions which also display a link
- What is this?
The topic turned (twisted? ) a few other directions, then back again. Eventually I dug out a browser and started this page. Hope it helps
Conclusions I think I saw:
- leaning toward links, inclusions, macros (or PPM or whatever they are called this week) each getting a different paired symbol.
- a preference for a paired symbol, the same symbol being easier to type twice even if it is shifted, than two different characters, even if a little more readable.
- combining along the lines of code behavior where possible.
- typing less characters for support rather than more, so, maybe three characters instead of two, to replace long words like attachment: and not favoring wikipedia-style Image: for instance.
Markup Specification
Markup proposals
Easy markup
PositionalMarkup is not a good name for the proposal, because it was never intended to allow only positional arguments in extensions.
- But Easy Markup is also a bad name, you can call any proposal "Easy".
- If you can think of a good name, just change it.
Goals
- Easy to learn
- only a few simple familiar concepts
- as consistent as possible
- Easy to use
- Easy to type
- compact syntax (less then html)
- Easy to read
- clean syntax
- uses symbols in a familiar way
- no unnecessary redundancy
Small MoinMoin-Core, use of plugins where it makes sense
Similar to MediaWiki when possible, as some new MoinMoin users already know MediaWiki
The other stated goals are more important than similarity to MediaWiki
Links
Rules:
Anything inside [[]] is a link
- A Link will never include (unlike current markup)
- The second optional argument can be used as the caption for the link.
- The second optional argument can also be an image (attachment or page, ends with .gif, .png or .jpg). This image will then be turned into a link.
- The arguments are separeted by a commma.
Parser should discard spaces after the comma; Both [[Page Name, Title]] and [[Page Name,Title]] should both be displayed as Title
Markup |
Display |
Comments |
[[PageName]] |
|
|
[[Page Name]] |
|
|
[[PageName, Title]] |
|
|
[[Page Name, Title]] |
|
|
[[Page Name, moinmoin.png]] |
(links to "Page Name") |
|
http://example.com |
same for all other schemas |
|
[[http://example.com]] |
same for all other schemas |
|
[[http://example.com, Example]] |
same for all other schemas |
|
[[moinmoin.png]] |
link to an attachment, will open the image in a window. Assume that attachments are pages. |
|
[[moinmoin.png, 1000 words]] |
same with custom title |
|
[[http://example.com, moinmoin.png]] |
(links to http://example.com) |
|
[[http://example.com/moinmoin.png]] |
link to external image - does not include the image! |
|
[[http://example.com/moinmoin.png, Funny Man]] |
link to external image with title |
|
[[wiki:MoinMaster/Remote Page Name]] |
interwiki link to: MoinMaster/Remote Page Name |
interwiki link |
[[wiki:MoinMaster/Remote Page Name, page from Master]] |
page from Master (links to MoinMaster/Remote Page Name) |
interwiki link |
info@somedomain.org |
|
Extensions (macros, parsers)
Rules:
Anything inside {{ }} is an extension ( / plugin / PPM )
- Extensions support positional arguments, keyword arguments and keywords as arguments.
- Arguments are in parentheses
- The arguments are separeted by a commma
Parser should discard spaces after the comma. Both {{Image(sunset.jpg, left)}} and {{Image(sunset.jpg,left)}} should get the same result.
description: markup example
linebreak macro: {{BR}} WordCount macro: {{ WordCount(FrontPage, 1000) }} indent table parser: {{IndentTable 1 2 3 4 }} Python code: {{python import sys, os print 'hello!' }} SectionParser, uses keyword arguments: {{section(class=cssclass, format=rst) RST markup here... }} Image macro, uses keywords as arguments, for example the keyword "right" means to add an image on the right side of the page: {{ Image(sunset.jpg, thumb, right) }} {{ Image(sunset.jpg, left) }} {{ Image(sunset.jpg, left, thumb) }} Empty extension default to plain plugin: {{ plain text as is }}
Inclusion
A stated goal is to reduce the number of concepts for the user and the moin developer. IMHO the moin core should be as small as possible. As a consequence, inclusion should be implemented as macros.
markup example |
description: includes ... |
|
page "Featured Article" |
|
"moinmoin.png" |
|
"moinmoin.png" |
|
"sunset.jpg" as thumbnail |
|
"sunset.jpg" as thumbnail, image caption "a sunset" |
"sunset.jpg", the image is sent over to the right |
|
"sunset.jpg", forced to the left |
|
"sunset.jpg", scales image to maximum size 100 |
|
"sunset.jpg", scales image to maximum size 200, image caption "a sunset" |
If images are pages or sub pages, then we don't need Image, only Include:
- include the page content
- include an image saved as "Some Image"
But then size and alignment don't not make sense for Include.
Alternative Extension Syntax
Use a colon instead of brackets to separate the macro name from the macro arguments.
{{ Image: sunset.jpg, right }} {{ Image: sunset.jpg }} {{ section: class=cssclass, format=rst RST markup here... }} {{BR}} {{IndentTable 1 2 3 4 }}
Backward compability
- a conversion script is needed, which converts the old markup into the new markup
- as positional arguments, keyword arguments and keywords as arguments are supported for PPMs, it is possible to convert the old syntax for parsers and macros to the new syntax for PPMs
- tbc
Discussion
This proposal is more complex than KeywordMarkup, as you have both positional markup and keywords, its more like a programing language. For example, the function calling convetion: name(value, value), compared with simpler html: <tag key="value" key="value">. This also add the complexity of dealing with keyword and positional arguments in same call, just like in Python. For example, is this call correct? name(key=value, value)?
This will be harder to use for the new user, easier for a programmer. -- NirSoffer 2005-07-11 07:30:30
Yes, the proposal is more complex than KeywordMarkup, because it offers someone who creates a macro more choice. If we take away that choice, we loose backward compability with the current MoinMoin markup. Keyword arguments make sense if you have a macro with many arguments and you can't use "keywords as arguments". When you have to use keyword arguments where they are not nesessary (for example with macros with only one or two arguments), this constraint adds unnesessary complexibility from the perspective of the user of your extension. New users normally don't use macros or parsers with 5, 10 or 15 arguments. To offer them a simple syntax for the extensions they normally use by allowing positional arguments and keywords as arguments, makes it IMHO easier for them.
Keyword markup
This proposal try to:
- Be clear, don't use funny characters if there is a readable way
- Be consistent as possible, so user learn only few things
- Use current syntax or syntax we use in other parts of the system
Be like WikiPedia markup if it make sense
Be like html - well known, simple, easy to merge in stuff like class:value in a natural way
The basic idea is [[]] make link, {{}} include extensions. Everything but links is implemented with extensions and can be replaced with farm or wiki plugins.
Technical note: http, page, image and like will be plugins. To have external links that use "rel=nofollow", simply install a 4 line nofollow http plugin, that extend the builtin http handler by sending the attribute to the formatter. To have fancy include with 17 arguments, install a custom page plugin.
This proposal try to use key:value pairs when possible. We already use this for ACL:
#acl user:read,write All:
and search:
[[FullSearch(linkto:PageName title:Help)]]
And we many want to use this for meta data:
Author: Name Status: Draft ...
Links
Rules:
Anything inside [[]] is a link
- Link will never include (unlike current markup)
title:, image: keywords used just like in our search for special cases.
No keyword in the first item means page: - [[PageName]] == [[page:PageName]]. Because the most important markup in a wiki is a page link.
http://, ftp:// are (funny) keywords too
- Support old link types with no change, until at least until next major release
Markup |
Display |
Comments |
[[PageName]] |
|
|
[[Page Name]] |
|
|
[[PageName title:Title]] |
|
|
[[Page Name title:Title]] |
|
|
[[Page Name image:moinmoin.png]] |
(links to "Page Name") |
Its not hard to parse but smell |
[[Page Name title:Title image:moinmoin.png]] |
|
|
[[http://example.com]] |
same for all other schemas |
|
[[http://example.com title:Example]] |
same for all other schemas |
|
[[moinmoin.png]] |
link to an attachment, will open the image in a window. Assume that attachments are pages. |
|
[[moinmoin.png title:1000 words]] |
same with custom title |
|
[[http://example.com/moinmoin.png]] |
link to external image - does not include the image! |
|
[[http://example.com/moinmoin.png title:Funny Man]] |
link to external image with title |
|
[[wiki:MoinMaster/RemotePageName]] |
interwiki link |
|
[[wiki:MoinMaster/RemotePageName title:page from Master]] |
interwiki link |
Backward compatibility:
Markup |
Display |
Comments |
works unless turned off in config |
||
same for all other schemas |
Additions:
Markup |
Display |
Comments |
[[Page Name class:value]] |
Page Name (use css class) |
add the css class of the link <a class="original-class value"> |
[[Page Name accesskey:value]] |
Page Name (use accesskey) |
Set accesskey for the link <a accesskey="value"> |
Problems:
Can't have page named "Page Name title:Something"
Unless we allow: [[page:"Page Name" title:Title]]
Include
Rules:
Anything inside {{}} will include the content in the page
- The content can be some extenstion (called now macro/parser)
- Include of pages and attachments is done with extenstions
Will not support old call syntax because links took the [[]]`
page:, image:, media: are plugins
- The first item is the plugin, the next are the arguments
This is little wired, but thats how http://domain title:text works in the links, if you see http: as a plugin.
- Order of arguments does not matter
Markup |
Display |
Comments |
text {{BR}} text |
text |
include <br> |
{{PageCount}} |
16001 |
simple macro |
{{page:Page Name from:"^= heading 1 =$" to:"^= heading 2 =$"}} |
(include parts from "Page Name") |
macro with arguments |
{{page:Page Name}} |
(include the contents of "Page Name") |
include a page |
{{image:Image Name}} |
(include "Image Name") |
or an image |
{{media:Media.mpeg}} |
(include "Media.mpeg") |
include time based media |
{{http://example.com/moinmoin.png}} |
|
or an external image |
{{http://example.com/}} |
(include the html from example.com/) |
unsafe, unless we sanitize the html |
{{image:Image Name class:cssclass title:Simple Figure}} |
(Figure section) |
a simpler way to do figures {{{#!figure\n#class cssclass\n}}} |
Parsers extenstions:
Empty extension default to plain plugin: {{ plain text as is }} Python code: {{python import sys, os print 'hello!' }} New SectionParser: {{section:cssclass format:rst RST markup here... }} or: {{section class:cssclass format:rst RST markup here... }} ? {{figure some markup <!> we must have nesting if we want to include here! }}
Backward compatibility:
Markup |
Display |
Comments |
attachment:file.txt |
|
|
inline:file.txt |
1 I'm a file.
|
|
drawing:drawing.png |
drawing.png |
|
http://moinmoin.wikiwikiweb.de/wiki/classic/img/moinmoin.png |
|
|
Maybe we should use the same words: inline, attachment and drawing instead of page and image?
Discussion
IMHO key:value pairs should only be used when nessesary. One advantage of a wiki markup language is that the user has to type less when he uses it instead of html. With a wiki markup similar to html you loose that advantage.
I like
{{Image(sunrise.jpg,right,nofloat,thumb,"A sunrise")}}
more than
{{Image:sunrise.jpg alignment:right float:nofloat size:thumb caption:"a sunrise"}}.
I also think that a syntax without key:value pairs is easier to learn for a new user.
WikkaWiki uses a syntax similar to html ({{image class="center" alt="DVD logo" title="An Image Link" url="images/dvdvideo.gif" link="RecentChanges"}}) and one of the reasons I chose MoinMoin was because I liked the MoinMoin markup more than the WikkaWiki markup.
Because of these reasons, I prefer the proposed PositionalMarkup to the proposed KeywordMarkup.
- Try to explain a new user how to call this:
{{Image(sunrise.jpg, right, nofloat, thumb, "A sunrise")}} The order of the arguments is important? How do you leave the default value? How do you find the function signature? You must add keywords for this type of call.
The order of the arguments is not improtent. {{Image(sunrise.jpg, nofloat, thumb, "A sunrise", right )}} also works.
As a consequence the plugin is easy to use, even for new users. A new user would probably use not all optional arguments, she would probably use {{Image(sunrise.jpg)}} or {{Image(sunrise.jpg, "A sunrise" )}}.
The syntax {{Image:sunrise.jpg alignment:right float:nofloat size:thumb caption:"a sunrise"}} is IMHO harder to learn, because a user has to know the keys and the values which are allowed for each key. The argument "right" to send an image over to the right is easy to remember. If a user also needs to remember the keyword "alignment", it gets harder.
In KeywordMarkup there is also no clear distinction between the name of the macro and the arguments of the macro, which might be confusing for some users.
I don't think it will work - arguments without any specific order that are magically bound to the correct variable?
For the Image macro and similar macros it works, because the arguments are or contain keywords. Have a look at http://www.druidwiki.org/HelpOnThumbnail and http://en.wikipedia.org/wiki/Wikipedia:Picture_tutorial.
What if I have some arbitary strings like:
{{section, cssclass, cssid, title}} {{Action, edit, Edit, BacktoPage}} {{Include, from here, up to there}}
Either order matter, or you need keys. keys are easier to read.
- Agreed, in these cases you have to use keyword arguments or positional arguments.
Discussion
Links contain two types of information: the address (URL or WikiName) and how it should be displayed (text or image). The two are sometimes separated by a space and at other times separated by a vertical bar in the suggested markup. The markup would be simplified if I could always separate the two by a vertical bar.
- I changed it.
- Better yet, if I could always separate the two by a comma, the link syntax would be similar to the macro syntax. A link would appear to be a special case macro.
I prefer the usage of ["Moin Moin Etymology"] displaying as Moin Moin Etymology, ["Moin Moin Etymology" Etymology] displaying as Etymology, and urls being the way they are now. This is how we have implemented it at DavisWiki.org. The only possible confusion is that sometimes people put quotes around external links, such as ["http://daviswiki.org" page], which would be solved (apparently) in this proposed markup, but there's no reason the same couldn't be done with our quoted markup -- just automatically route "http://*" to be external. We have chosen to notify the user when they make this mistake, which teaches them how to make internal links. Maybe I'll change this. I do like the idea of unified [[Image]] macro that can toggle thumb/no thumb. Right now we have our [[Thumbnail]] macro in addition to the attachment: syntax, which is ugly, but we will unify soon.
I would also propose that if you want to simplify markup you turn off WikiNames by default. Giving the user two radically different linking mechanisms is very counterintuitive. Also, I noticed that in the above examples commas were used within the macros. As was noted, this is inconsistant -- but does it matter? I think if you are going to use "|" as the delimator of links that it ought to delimate everything, just as wikipedia does it. I like the use of " quotes because they are used in natural english to quote things, and commas are used to seperate ideas, not | bars. --PhilipNeustrom
I agree both on WikiNames and delimiter. We should have one default way to do things, not 2. -- NirSoffer 2005-07-06 21:39:20
Refactoring proposals
The goals stated in both proposal are duplicated, we should factor them out to the main page, leaving only the specific goals of each proposal.
It will be interesting to try a version which is even more like WikiPedia, not because its better, but because its poplar, and the best interface is the familiar interface. Here is a MediaWiki syntax reference.
A html like (WikaWiki) syntax also can be nice and very easy to learn. And there are more proposals on the wiki, maybe its good to collect them and compare here.
Python like calls
Since some users prefer positional arguments, and some keywords, and calls with one or two arguments seems bloated with keywords, and calls with more arguments are impossible without keywords, we need both
Maybe use the python tokenizer that Oliver wrote, so both [["Page Name", image="moinmoin.png"]] and [["Page Name", "moinmoin.png"]] will work. This will create a more complex system, but can satisfy all users.
The problems with Python like calls is its too complex. In Python you have both strings and bound names. In a wiki page, everything is a string, so we don't need any quoting:
.
{{plugin arg1, kw=value}}
.
call = "plugin" argskw = ["arg1", "kw=value"]
.
args" = ["value",] kargs = {"kw": "value"}