We need an event system for moin. Whenever something of note happened, a message would be distributed throughout MoinMoin, Plugins (MoinMoin/event/ + data/plugin/event) would receive events as well.

So, for example, when a page is opened for edit, an event is announced. When a page is saved, an event is announced. If something is deleted, an event is announced, ...

Contents

    1. Event Broker
    2. Event Objects
    3. Event Handler Objects
    4. Triggers
    5. some ideas for event plugins / handlers
      1. page read,write,...
      2. caching stuff
      3. timer handlers
      4. related objects
    6. Discussion
  2. Alternative Implementation
    1. Use cases
  3. Alternative 2 - Notifier
    1. Example - renaming a page
    2. Object identification
    3. Remote notification
  4. Implementation with an search index
  5. Discussion 2

LionKimbro announced he will implement this himself. He is motivated because he wants it to be easy to connect to his subscribable event system DingDing.

OliverGraf noted, that we should do it with transactions.

Event Broker

Event Objects

Event Handler Objects

Triggers

Triggers have a unique triggername (simple string?). Triggering some trigger leads to generation and processing of one or more event objects.

RequestTrigger (RT) means, that a user (or other external entity) has triggered something, that should be processed, if possible and if allowed. ProcessedTrigger (PT) means, that moin has really done and completed something, that should trigger other stuff.

some ideas for event plugins / handlers

page read,write,...

caching stuff

timer handlers

  1. Page
  2. Page parent page if any
  3. Category page
  4. Wiki - we don't have this object today, but that could be the object that know stuff about all the wiki
  5. Site or Farm - another new object that can do thing globaly

Discussion

The start, commit/abort calls are a bit tricky... start has to check everything that is needed to do a commit. If there is something not possible, it raises the abort exception, ok. If no object has objections against the transaction, the commit methods are called to do the real thing. If now something bad happens (system may have changed in the meantime...), how is the rollback of the already commited objects done? -- OliverGraf 2004-07-17 17:55:49

IMHO we should not use the EventSystem to implement the normal MoinMoin processing logic. Actions and normal code is sufficient for this. Everything else will make things only much more complicated. -- FlorianFesti 2004-12-21 18:34:05

We need global events and events per page. Parts of MoinMoin must easily be able to subscribe to this events. This subscription should be data driven. -- FlorianFesti 2004-12-21 18:34:05

It looks too complex and use too many special objects. Why not let any object to post or register events?

Also, there is too much connections between the objects. The broker should not care what the subscribers are doing with the events or what type of object they are.

If we need specific order, the system can register the standard objects for standard events first, then let any object register events. First registers will be posted firsts. We don't have to build everything on the events system, it's just a way to make the system easy to extend and customize.

All we need to do is decide on the signature of the event callback, like somename(event), where event is a subclass of Event. Any object that like to get an event, will register with the event broker with the object method that handle the event.

Here is a sketchy code that works like that: 

   1 #
   2 # In MoinMoin/events.py
   3 #
   4 
   5 class Event:
   6     def __init__(self, sender, info=None):
   7         self.sender = sender
   8         self.info = {}
   9         if info: self.info.update(info)
  10         # Check that the event has the needed info:
  11         for key in self.neededInfo:
  12             if not self.info.has_key(key):
  13                 raise EventError('%s info should have %s' % (self.__class__, key))
  14 
  15 # Page events
  16 class PageDidReadEvent(Event):
  17     self.neededInfo = ['pagename', 'user']
  18 class PageDidEditEvent(Event):
  19     ...
  20 class PageDidTimeoutEvent(Event):
  21     ...
  22 class PageDidSaveEvent(Event):
  23     ...
  24 class PageDidRenameEvent(Event):
  25     ....
  26 class PageDidDeleteEvent(Event):
  27     ...
  28 
  29 # User events
  30 ...
  31 
  32 # Wiki events
  33 ...
  34 
  35 # Farm events
  36 ...
  37 
  38 class EventBroker:
  39     def registerEvent(self, event, meth, sender=None, info=None)
  40         # First registers will be posted first
  41         self.events[event.__class__].append((meth, sender, info))
  42 
  43     def postEvent(self, event):
  44         """ The broker filter the events according to the subscribers terms, so the 
  45         subscribers code could be simplified. They will get just this kind of event, from 
  46         this object with this info. Post events in order of registration.
  47         """
  48         for meth, sender, info in self.events[event.__class__]:
  49             if sender and sender != event.sender:
  50                 continue
  51             if info:
  52                 # Check that info keys and values are in event.info
  53                 ...
  54             meth(event)
  55             
  56          
  57 #
  58 # In some class code:
  59 #
  60 
  61 from MoinMoin.events import *
  62 
  63 
  64 class someClassThatWouldLikeToGetEvents(AnyClass):
  65 
  66     def registerEvents(self):
  67         
  68         # Register page save event, sent by any object
  69         self.request.broker.registerEvent(PageDidSaveEvent, pageDidSave)
  70     
  71         # Register any event that come from specific object
  72         self.request.broker.registerEvent(Event, pageEvent, sender=self.request.page)
  73 
  74         # Register another object for any event for page with specific name
  75         self.request.broker.registerEvent(Event, anotherObject.frontPageEvent,
  76                                          info={pagename: 'FrontPage'})
  77  
  78     # Events this object handles:
  79     
  80     def pageDidSave(self, event):
  81         # handle the event here...
  82 
  83     def pageEvent(self, event):
  84         # handle the event here...
  85 
  86     # don't handle frontPageEvent, anotherObject will handle that

-- NirSoffer 2004-07-17 21:15:10

Using a class is more elegant on the one hand, but that from events import * isn't that nice, as every event has to be defined there and every module handling events has to import it.

Or maybe this will be better: 

   1 # In MoinMoin/events.py
   2 
   3 events = dict([(name, obj) for name, obj in globals().items() 
   4                    if name.endswith('Event') and type(obj) == type(Event)])
   5 
   6 # Then in client code:
   7 
   8 from MoinMoin.events import events
   9 
  10 # register event
  11 broker.registerEvent(events['PageDidSaveEvent'], ...)
  12 
  13 #or post an event:
  14 event = events['PageDidSave'](sender=self, 
  15                               info={'pagename': 'PageName', 'user': self.user})
  16 broker.postEvent(event)

But I'm not sure, events['PageDidSaveEvent'] is also not nice.

-- NirSoffer 2004-07-17 23:27:04

How about events.Page_Save.subscribe(self.handle_pagesave)? -- AlexanderSchremmer 2004-12-21 19:53:14

Alternative Implementation

As we have a root page global subscriptions can be attached to it.

Events:

Subscribers: 

   1 # interface
   2 
   3 notify(request, eventname, page_from, **kwargs)
   4 
   5 Event.subscribe(event, subscriber, kwargs)
   6 Event.unsubscribe(event, subscriber, kwargs)
   7 
   8 # Example
   9 # Pages could subscribe themselfs to all Pages they link to (including non existent ones).
  10 # Then they can create a cache of the rendered content that treats all page links as static
  11 # If one of the pages it links to get created/deleted it can update the cache.
  12 
  13 ## page_refresh_cache.py
  14 notify(request, eventname, page, pagename, **kwargs):
  15         page = Page(pagename)
  16         page.clearcache(remotepage)
  17 
  18 ### in Page.py
  19 for link in links:
  20     page = Page(request, link)
  21     page.event_delete.subscribe("page_refresh_cache", {pagename=self.pagename})
  22 
  23 
  24 # Implementation
  25 class Event:
  26    
  27     def __init__(self, request, page, name):
  28         self.request = request
  29         self.page = page
  30         self.name = name
  31 
  32     def subscribe(self, receiver, kwargs={}):
  33         found = True
  34         line = repr((receiver, kwargs))
  35         page.aquire_lock()
  36         subscriptions = page.get_data("Subscription", self.name).splitlines()
  37         index = bisect.bisect_left(subscriptions, line)
  38         if subscriptions[index]!=line:
  39             subscriptions.insert(index, line)
  40             page.set_data("Subscription", self.name, "\n".join(subscriptions))
  41             found = False
  42         page.release_lock()
  43         return found
  44 
  45     def unsubscribe(self, receiverclass, kwargs={}):
  46         # return if found
  47 
  48     def notify(self):
  49         subscriptions = self.page.get_data("Subscription", self.name).splitlines()
  50         for subscription in subscriptions:
  51             receiver, kwargs = eval(subscription)
  52             
  53             receiver_function  = importPlugin(self.request, "receiver", receiver)
  54 
  55             kwargs["pagename"] = self.page.pagename
  56             kwargs["eventname"] = self.name
  57             receiver_function(**kwargs)

Use cases

There are different kind of use cases which take different amount of advantage of such a system:

  1. Single objects (Pages, User UI, language, ...) subscribing to single Pages.
    • Page links - everywhere we have dynamic links that are created with Page.link_to(): page content, UI - when a page renamed, change all links to it on all pages automatically.

    • IncludeMacro - subscribe to included page, invalidate cache when they change.

  2. Single objects subscribe to global events
    • usefull for getting and filtering creation of new pages
    • pagelist macro
  3. Global objects subscribe to single pages
    • Groups
    • Dicts
    • Interwikimaps in pages
  4. Global objects subscribe to global events (we could use normal code insted)
    • Category list
    • Template list

The categorization to global / single object is not needed. All objects are wiki objects, pages, categories, groups, users. Any object can subscribe to any other object.

Alternative 2 - Notifier

Here is a another alternative, which use a 3 parts system: 

Subscriber <->  Notifer <-> Target

The Subscriber subscribe events with the Notifier, which is always available as part of the wiki object (see WikiClass). The Notifier save the subscription data structure. The Target does not know which other objects subscribed to its events, and does not care. The Target notify the Notifier on all events, and the Notifier check which objects want to be notified and call each object callback.

With this system any object can subscribe to list of events on any object, even objects that do not exists. For example, an admin can subscribe to UserDidCreate event, using regular expression for the user name. User can subscribe to any change made by other user on any page, etc.

With this system, all the relevant code in the Notifier, and we keep the model object simple.

Example - renaming a page

PageA subscribe to events of PageB: 

wiki.notifier.registerEvents(events=[PageDidRename, PageDidDelete],
                             sender='PageB',
                             name=self.name,
                             callback='linkedPageDidChange')

UserC subsribe to all events of PageB: 

wiki.notifier.registerEvents(events=[], # any event
                             sender='PageB',
                             name=self.name,
                             callback='mailEvent')

ACL system subscribe to all events for any page group page. When ever a group page is created, deleted or changed, acl system will update its groups from that page. This will replace scandicts. 

wiki.notifier.registerEvents(events=[], # any event
                             sender=wiki.cfg.page_group_regex,
                             name=self.name,
                             callback='updateGroups')

2 week later PageB renamed by user UserC: 

event = PageDidRename(user='UserC', newname='PageBee', comment='I like CamelCase')
wiki.notifier.notifyEvent(event=PageDidRename, sender=self.Name)

Notifier notify all subscribers: 

for subscriber in getSubscribersForEvent(PageDidRename):
    subscriber = wiki.pages[name]
    call = getattr(subscriber, 'linkedPageDidChange')
    call(PageDidRename, sender)

PageA rename all links to PageB to PageBee: 

body = self.getRevision(current)
# should probably use re, simplified for the example
body = body.replace('PageB', 'PageBee')
self.save()

UserC send mail to the real user: 

# code from util mail here

Object identification

There are some problems to solve here, like how do you save the subscriber in notifier data. Maybe use the path to the object, like (type, name), for example, ('pages', 'FrontPage') or ('config', 'SecurityPolicy').

Then we can get the object at runtime by code like this: 

obj = parent # notifier is a child of the wiki
for item in subscriber:
    obj = getattr(obj, item)

Same method could be used to save the sender of the event, since the subscriber need to specify the correct object - its can be a user named Foo or a page named Foo.

What about using seperate Classes to be notified. These would have a fixed interface and would be created when needed. Each subscriber could implement a class on its own. 

class UpdatePageCache(EventReceiver):

    def __init__(self):
        self.pagename = pagename

    def notify(self, request, eventtype, name, message):
        Page(request, message).clearcache(name)
        

notifier.subscribe(events=["PageCreate", "PageDelete"], name="EditorGroup", message=self.pagename)

Remote notification

We can add subscribers from other wikis, and use wikirpc calls as callbacks. For example, ('UntitledWiki', 'pages', 'BadContent') can subscribe to ('MoinMaster', 'pages', 'BadContent') for events PageDidChange, then BadContent will be updated automatically on all wikis - without the need for checking the page timestamp and revision on each save on every wiki.

To make this work, we need a secure method to call other wikis - when a wiki get a callback, it should contain a signature of the sender, so spammer can't DOS wikis with false wikirpc calls.

Implementation with an search index

We could use a search index to (in addition to indexing page contents) manage the subscriptions. Each entity (page) that wants to be notified create an entry in the index with all pages/other entities it want to subscribe to. If one page changes it does a search for all pages/entities that subscribed to it which should be resonably fast. And we don't have to think about proper data structures and their implementation.

Discussion 2

How do we ensure persistency? What happens if someone subscribes an event two times? Who should check if a call to this system is needed, i.e. if the event etc. was subscribed already? (This question is posed on every run of the portion of code that is interested in events, often for every run). Should the notification layer ignore duplicates? How to check for duplicates? How to delete a subscription? Which key to supply when you want to delete a subscription?

How should RPC work for this? Other wikis will like to subscribe to remote events as well.

MoinMoin: EventSystem (last edited 2009-03-27 18:54:43 by ReimarBauer)