Contents

  1. Proposal
  2. Discussion

/!\ A more current discussion is there: http://etherpad.osuosl.org/moin2-PluginSystem

Proposal

The current (i.e. "traditional") plugin concept in MoinMoin implies early distinction of plugin types. That is, plugins are organized by type (or call it category) first, and by application focus second. This is reflected in the plugin directory structure: 

wiki
 data
  plugin
   action
    ActionA
    ActionB
   parser
    ParserA
    ParserB
   ...

If every plugin would be naturally restricted to one and only one of the predefined types, the given organization would not impose a real problem.

However, I often find myself in a situation where an extension I write for MoinMoin is a mini-application, consisting of two or more technical plugin types. From the viewpoint of a plugin author, having to bounce up and down the above deeply nested directory structure is a maintenance nightmare.

This made me think about the plugin concept in general.

I think there is a misconception in that concept. It enforces the view that a plugin is a unit of a certain type. A much more appropriate view would be that a plugin is a unit of functionality, and parsers, actions and other "types" are the mechanisms through which the plugin "hooks into" moin. This is true at the level of deployment as well as the level of implementation (thereby virtually removing the difference in packaging between the two).

In fact, most other established plugin-enabled applications (like WordPress or Drupal) follow this pattern, rather than imposing a "type-centered" approach.

I propose to gradually shift the plugin organization to one that distinguishes plugin packages first, and by type second: 

wiki
 data
  plugin
   A              (<= all stuff of plugin "A" goes here)
    action         (<= it's action(s))
     ActionA
     AnotherActionOfA
    parser         (<= it's parser(s))
     ParserA
    ...
   B              (<= all stuff of plugin "B" goes here)
    action         (<= it's action(s))
     ActionB
    parser         (<= it's parser(s))
     ParserB
     AnotherParserOfB
    ...

Contrast to the current organization, the proposal is in alignment with a natural deployment and development schemes, supporting local focus for both.

Discussion

Personally I would prefer to put everything from a related set of plugins into one file (module) or directory (package) and just have some way of telling Moin which classes re actions, macros, parsers, etc. This could be done either with new-style clasess and their __subclasses__ attribute, or more explicitly with some dict or lists. -- RadomirDopieralski

I think about to move the instance plugins in different dirs instead of adding plugins several times to wiki instances. Since 1.8 we can have plugins in different dirs. e.g. plugin_dirs = ['/some/where/extensions/data/plugin']. That can also be plugin_dirs = ['/srv/share/moin/extensions/arnica', '/srv/share/moin/extensions/mathtran', '/srv/share/moin/extensions/slideshow']. That means you have only to configure the plugin_dirs var to get acces to the plugins you want. The plugins are separated from each other which makes it much easier to handle the extension repo plugins and may be other 3rd party plugins. The arnica css can be added for example to /srv/share/moin/extensions/arnica/htdocs. We have to add an arnica Alias to the server configurations pointing to that dir. I think that makes it much easier for me to make a release of a plugin of that repo. And it looks to me that this becomes easy to describe. For arnica it is something similiar to: unpack arnica on /srv/share/moin/extensions/ set an Alias add to wikiconfig.py the plugin_dirs var and the usual html_head line. ReimarBauer/Photo/img.png -- ReimarBauer 2009-03-18 22:52:15

Of course, having the data files and the code of plugins both reside together and be automatically detected is the ideal way in the perfect world. Unfortunately, that's not possible with the current state of affairs and the way web servers work. The plugin code has to be accessible for the moinmoin python process running on the server, and it is indeed relatively easy to write code that would scan the directories provided and import apropriate code as needed. The data files, on the other hand, are needed by the html and javascript running in the user's browser. To be available to the browser, they need to be served by the web server running MoinMoin (or any other, in fact). We can scan the plugin directories and serve the needed data files when MoinMoin is being run standalone, with its own built-in web server, but that's really only suitable for personal wikis. Public wikis will usually need some setup to tell the web server where the static data files are and how to serve them. Obviously, we can't configure the web server for the user. It seems wise to keep the required setup of web server at minimum, though, and this is achieved by only having one htdocs directory with all the static files, so that the user doesn't have to configure his web server for each single plugin he adds separately. We can't keep the plugins code in that directory, because they may contain passwords and other sensitive data, and really shouldn't be accessible to the world. -- RadomirDopieralski

There is no question that we want to make it easy, otherwise we wouldn't even bother thinking about it. There are generally two ways in which PHP apps do it:

  1. Serve the static files by itself, by reading them from disk and streaming to the web server. This is slow and wastes CPU, but gives you full control and flexibility. Apache and some other web servers even have mechanisms to speed it up, by just telling the web server which file to use, and leave the reading and serving up to it. We are planning to do this in MoinMoin as a fallback, for when the URL prefix is not configured, so that you can get //something// working fast, and maybe improve it later if it's necessary.

  2. Mix the data files with the code, by putting both together in the same directory subtree. This may work because .php files are by default executable by the web server, so even if you attempt to access them directly, you get the execution results, and not the actual content with passwords and private user data. It still poses a security threat in situations when PHP execution may get temporarily disabled on your web server. It also complicates the setups that require URL rewriting or don't execute PHP files directly.

All in all, 1. is already being worked on, but can't replace serving static files, 2. is out of question for Python. We've been looking around for other options (symlinks, redirects) but none are flexible enough and cross-platform at the same time to implement them and not break existing MoinMoin instances. We would very much like to hear any (even crazy) new ideas.

By the way, MoinMoin can be installed without root permissions pretty easily as cgi, or even as a wsgi app, if the host has it enabled. All you have to do is copying the htdocs directory somewhere, and setting the URL prefix to that location.

Maybe it's good to list the possible methods of making plugins be able to add their own static files:

  1. Serve the static files by MoinMoin. Advantage: you can do anything with them, put them anywhere you want, even in a database backend. Disadvantage: slow, high server load, non-portable speedup hacks.

  2. Put the files together with their plugins in subdirectories of one big directory that is accessible through http. Advantages: fast access, easy install. Disadvantages: the directory must be set up to be world-accessible, security concerns -- especially with auth plugins.

  3. Let MoinMoin make symlinks to from the files in plugin directories to the world-accessible directory. Advantages: fast, no security risk. Disadvantages: web server must follow symlinks, directory needs to be set up to be world-accessible, Microsoft Windows doesn't support symlinks, htdocs must be writable by moin.

  4. Let MoinMoin copy the files to the htdocs directory. Advantages: works on windows. Disadvantages: duplicated size, changes don't appear immediately when upgrading.

-- RadomirDopieralski

So is there any light at the end of the tunnel? Right now, I don't see any decision, just opinions. Perhaps it helps if I summarized them:

Here is a question that comes to my mind:

Is it possible to have both mechanisms in place, and serve most static files from "htdocs", and some from the plugin folder (the latter being served by moin)? Is there a way to easily switch between the two? That way, a plugin can be easily installed into one folder, and the wikiadmin can decide to move some of the files to "htdocs" if he detects performance issues.

-- ZoranIsailovski 2009-04-17 14:04:18

in many case you use the "main" apache process(es) to serve the static files (or even running this on a separated system, for kind of "poor man's load balancing"). and some wsgi processes/threads are executing the moinmoin code with different user rights (for security reason). With your static files together with the program code you have in my eyes drawbacks like:

-- MarcelHäfner 2009-04-17 20:07:22

I would also like to see a new, generalized plugin system, with just one MoinMoin.plugins package (instead of actions, macros, parsers, ...) and additionally, per-wiki configurable plugin directories. Moin could load all plugin modules on startup, and each plugin module registers itself with all handlers and ressources it offers (including static ressources). For the static stuff, it would be great to handle them by moin by default, but make handling by external webserver easily possible, too. So, who writes the code for that? :) moin 2.0 would be a good time to implement that. -- ThomasWaldmann 2009-05-10 17:40:20

I see Marcel's points, but not necessarily their justification:

In my view, farms and scaling are additional arguments PRO overhauling the plugin system.

I suggest we start gathering the NewPluginSystemRequirements (some of them are here already, just scattered throughout the discussiuon). -- ZoranIsailovski 2009-05-20 19:24:28

May be we could also find a way to overwrite default values defined for the argumentparser by a wikiconfig setting.

Currently, MoinMoin already can serve the static files through it, and will do that if you don't setup the Alias directive. I wonder if we could do something similar for plugin's static files. Several variations are possible:

-- RadomirDopieralski 2010-02-14 22:16:53

About overhaul of the whole plugin system, I had recently an opportunity to work with Trac, and I think its plugin system is something we could learn a lot from. They use the hook mechanisms provided by setuptools. The plugins are standard python packages, in form of eggs -- the archive may actually also contain data files. You can install the plugins easily with easy_install, and just enable in the configuration file. -- RadomirDopieralski 2010-02-14 22:26:19

See also FeatureRequests/EggFilePluginSystem

CategoryFeatureRequest CategoryUsabilityObservation

MoinMoin: PluginConceptOverhaul (last edited 2016-12-30 23:15:35 by MikeCrute)