Summercode Finland 2007, man/info editor with MoinMoin

This page is about Ville-Pekka Vainio's project during the Summercode Finland 2007 event.
The project is mentored by Karsten Wade.

• I'll be working for Fedora, the goal is to implement a man/info page editor based on MoinMoin.

• This will give Linux users a good way of searching and reading man/info pages online.
• It will also provide convenient URLs to be used as references in other documentation.
• The system should have support for man/info pages from multiple distributions and versions, eg. Fedora Core 6, Fedora 7, Fedora Development (Rawhide), etc.
• The system should also support package updates and handle those situation correctly
• Everything should be designed so that the system can be extended to cover other that Red Hat based distros, too.
• The user should be able to compare man/info pages in one release to the same pages in another release, like between FC6 and F7. The regular Moin diff tool can be used here, but some modifications may be needed.
• The system should give regular users the possibility of editing man/info pages
• Admins and other users could then check and possibly revert/fix those changes, as normally in a wiki
• Then admins could take a specific set of changes and export those in some form to the upstream projects, by simple Bugzilla reports or patches.

Please see my project page at Fedora Wiki, it includes a schedule and more details about this project that are not MoinMoin specific.

Links

• COSS Main Page

• Summercode Finland 2007

• Summercode Finland FAQ

• My project page at Fedora Wiki, includes a schedule

• My project blog

Open questions and talking points

These are the same points than in my Fedora Wiki project page, but as there are some Moin specific things here, posting them on this wiki too is important, IMO.

Internal storage format of man/info pages?

• Basic txt format and then generate textual diffs for Bugzilla reports
  • What is a "basic txt format"?
    • What I mean here is that we would have the pages stored just like they are displayed if you do eg. "man man", formatted with spaces, newlines and such, not in *roff or texinfo.
      • The question is whether that would be useful to the people creating and maintaining man pages or if it would just double the work.
    • The man and info sources seem to be quite simple, though, so maybe this idea could be discarded and I should look for better solutions, see below.
  • Probably the easiest to implement but requires manual work from upstream people

• *roff, DocBook, TeX, etc.

  • There is some unmerged docbook work left from last year. Maybe have a look at it.
  • I think it would be possible to use nroff and texinfo formats internally and then eventually make the system produce diffs against those formats

  • This depends a bit from what the Fedora project wants, I know DocBook is like a de facto standard there

  • These are more difficult to implement but it could be maybe possible to then generate diffs that upstream could merge more easily

  • This is somewhat of a problem now. If we take only man pages into consideration here to begin with, the publication of those is pretty easy. I could write a Moin parser for man source. The editing is lot more difficult:

  • If the man pages are stored into Moin "as-is", then generating patches would be easy, but editing the pages would require *roff knowledge, not easy for the regular users
  • The pages could be imported and stored as wiki markup, which would be probably the easiest format for regular users to edit. But producing meaningful, clean patches against upstream man source would be quite difficult.

  • Even if I made a wiki markup -> man source exporter, the diffs might not be usable, since the computer generated man source could be too different from the original upstream source.

  • The main point here is this: What do we want from this system? Do we want it to be a source of information and ideas for the man page maintainers (maybe by sending notification emails from wiki edits) or a tool that produces patches which can be applied with minimal human interaction?

• After some discussion on #fedora-docs, the idea is to store the man/info pages as wiki markup and then convert those to DocBook XML when exporting. That XML can then be transformed into whichever different format is needed (see eg. http://docbook2x.sourceforge.net/)

• During the first phase (publication) doclifter will be used to store the man/info pages as DocBook XML. Moin can parse that XML so that it's viewable as "normal" wiki pages.

Where to look for these man/info files?

• Repositories, CVS, etc.
• The system should handle a lot of different sources and be extendable for different version management systems and package managers

• Remember, this is something that we would like other distros to use eventually, too

Diff tool enhancements

• Moin's diff tool probably has to be changed somehow to support diffs between different pages, not just different revisions of the same page.

  • I have no real interest in your project - not a unix user - but by pure coincidence I implemented a DiffPage action today - quick and dirty, took less than an hour but maybe you can use it (-- RobertSeeger 2007-06-06 21:46:57):

    • DiffPage.py

• Or do we need / should we have a better versioning scheme in general, where Moin could have multiple "main" versions of a page?
  • How exactly would that work?

  • I guess this would require major changes when implemented in the backend and would conflict with HeinrichWendels current work.

• Probably more points to come during the summer

Decisions

• Which MoinMoin version to base the work on?

  • 1.7 was chosen, see the corresponding fedora-infrastructure-list thread. This was also discussed at #moin-dev.

TODO

Phase 1

1. Test doclifter

2. Make an importer for Moin that converts man source into DocBook XML through doclifter and saves the results in a clean namespace hierarchy

3. Modify Moin's diff functionality or make an action that takes the diff of two different pages
4. Extend the importer to handle info pages too. The conversion can be made through GNU makeinfo, but it needs Texinfo sources, it doesn't work on Info sources.

Source code

The code is in it's own 1.7 branch in Moin's Mercurial, http://hg.moinmo.in/moin/1.7-maninfo-vpv/. Also see MoinDev/MercurialGuide on how to use Mercurial.

About my work

Usually I'll be working about 40 hours a week, during the weekdays. I'm in the UTC+3 time zone, so the best chance of contacting me on IRC is probably between 05:00 UTC — 13:00 UTC, but I may be online during the evenings also, especially if there are meetings or something with other people from different time zones.

Comments

I suggest that discussion about MoinMoin specific things is kept here and other topics related to my project in the Fedora Wiki page. Feel free to give your comments and suggestions MoinMoin's codebase is pretty new to me, I'm learning as I go on with the project.