Global event system using IRC protocol

Contents

  1. Publisher
    1. Getting old messages
    2. Data provided
  2. IRC server
    1. Private IRC server
    2. Using mode +R freenode channels
    3. Multiple publishers
  3. Subscriber
  4. Usage examples
    1. Antispam Updates
    2. iRC - IRC Recent Changes
  5. Experiments

NetworkOfMoinMoinWikis needs a way to communicate with each other. Users want to be updated on real time when a change happens. See also related AdoptedPages server.

Example test cases:

Here is an early draft for a global event system using IRC protocol:

Publisher

Any wiki that want to publish events. The wiki pack the event in a way that can be transfered using IRC and then unpacked on the other side. For example by: 

from MoinMoin.util import web
event = {'wiki': name, 'page': u'פתיחה', 'action': 'SAVE', 'user': u'פלוני אלמוני'}
text = web.makeQueryString(event)

The wiki send the event to the wiki channel on the IRC server, using custom irc client running on the same machine the wiki run. The client can be easily written using Twisted.

Getting old messages

If listeners get disconnected they might miss event. This is not accepable for RecentChanges entries. Therefore the Publisher should offer a method to query past entries.

One implementation would be to have some commands that work via private messages.

As response the Publisher sends back the entries. To avoid flood kicks the data could/should be sent back via DCC connection.

Another option is to use HTTP for this kind of queries. Add a action=changes&from=time&to=time action, and return xml with the data.

A typical client will do this:

  1. Connect to the chanel
  2. Get old entries not in local cache using http
  3. Respond to new events on the channel

This is also possible, but we should define a way to get the URL and the data format of the file sent via http then -- FlorianFesti 2005-05-25 13:29:16

Data provided

The implementation of the AdoptedPages server has shown that for good RecentChanges integration a lot of additional data is needed (like diff, history, ... links). These data can be easily calculated if the foreign wiki engine is known. The question is if the client has to do these calculation or if we simply ship as much information as possible (and increase the traffic by factor 5-10). If we don't we should offer the clients the calculation rules for these links. See WikiDescriptionBlock and the links listed there.

The topic of the channel can provide a link to a description block accessible through http. A client can update itself with this data each time it joins the chanel.

IRC server

For each publisher, one or more channels should be registered, where only the publisher can talk and other might only listen. There should be a safe way to authenticate the publisher, and prevent other user from taking the publisher nick.

Authentication is a non trivial problem. If another user steal the publisher nick, the data in the channel can not be trusted. In any situation, only the publisher can talk on the channel, and if it is disconnected, it should reconnect again.

Private IRC server

The simplest solution to bypass the auth problems is to have a custom irc server (maybe implemented in Twisted) and run it on the same machine of the publisher wiki. The same process can have both publisher client and server.

Problems:

Using mode +R freenode channels

Another option is using freenode with registered nicks and channels that need registration to get a voice (mode +R)?. If there is real traffic some day we can setup an own server/network with the same components/ nickserv bot. That way we could leave the auth stuff out of our implementation and the standard we want to develop.

Multiple publishers

One channel can be shared by several wikis to publish events. I can make sense if by registering to one channel you get interesting events about some topic.

IMHO it is the responsibility of the clients to create trust. Simply only process the messages of publishers you know and trust - ignore everything else. If the channels are +R only registerd user can post messages and therefore the nicks are save. The clients shoudl check the mode of the channel ofcourse...

Subscriber

Each subscriber run an IRC client and join the interesting channels. Users might subscribe to nice plain text events channels, and use their client filtering to highlight interesting events. Wiki would subscribe to package event channel, then unpack each event to decide if its interesting.

The wiki client can do several things on events:

The client can be easily written using Twisted or irclib.py.

Usage examples

Antispam Updates

iRC - IRC Recent Changes

(name invented by starshine)

Experiments

I started to play with the idea. Here is a patch that add an IRC client to the twisted server, and publish SAVE events.

ges.patch

To experiment, join #MoinMoinEvents on irc.freenode.net, and edit pages in http://nirs.dyndns.org:8080/ .

MoinMoin: IrcEventSystem (last edited 2007-10-29 19:18:06 by localhost)