Any MoinMoin installation which is not based on plain CGI uses threads. We have several shared data structures that should be protected.

Use this decorator:

   1 def sync(f):
   2     import threading
   3     l = threading.Lock()
   4     def _(*args, **kwargs):
   5        try:
   6            l.acquire()
   7            return f(*args, **kwargs)
   8        finally:
   9            l.release()
  10     return _
  11  
  12 def mylockedfunction(x):
  13     #thread-senitive stuff
  14     pass
  15  
  16 mylockedfunction = sync(mylockedfunction)

Example of the problem code:

   1 import threading
   2 _plugins_lock = threading.Lock()
   3 
   4 #[...]
   5     global _plugins
   6     global _plugins_lock
   7 
   8     # Try to import from the wiki. Wiki plugins are located under
   9     # 'wikiconfigname.plugin' module. Try cache first (fast!) or
  10     # import from disk (slow)
  11     modulename = '%s.plugin.%s.%s' % (request.cfg.siteid, kind, name)
  12     key = modulename + '/' + function
  13 
  14     _plugins_lock.acquire()
  15     try:
  16         try:
  17             plugin = _plugins[key]
  18         except KeyError:
  19             plugin = pysupport.importName(modulename, function)
  20             _plugins[key] = plugin
  21     finally:
  22         _plugins_lock.release()

Here is a draft for these - a dict like object that use locking for setitem and delitem methods.

   1 import threading
   2 
   3 class ThreadSafeMixin:
   4     """ Thread safe mixin, can be used by multiple threads 
   5     
   6     The mixin provide thread safe __getitem__, __delitem__ and 
   7     __setitem__ methods. The real work should be done by the class
   8     getItem, delItem and setItem methods, wich will operate on the
   9     instance data.
  10     """
  11     
  12     def __init__(self):
  13         self._lock = threading.Lock()
  14         
  15     def __setitem__(self, key, value):
  16         self._lock.acquire()
  17         try:
  18             self.setItem(key, value)
  19         finally:
  20             self._lock.release()
  21             
  22     def __delitem__(self, key):
  23         self._lock.acquire()
  24         try:
  25             self.delItem(key)
  26         finally:
  27             self._lock.release()
  28 
  29     def __getitem__(self, key):
  30         self._lock.acquire()
  31         try:
  32             self.getItem(key)
  33         finally:
  34             self._lock.release()

This kind of thread savety doesn't help anything. You get lost updates if you get data and write it back:

   1 # on module load
   2 statistics = ThreadSaveDict()
   3 ...
   4 
   5 # in request code:
   6 oldvalue = statistics.get("hitcounts", 0)
   7 # here another thread can change statistics internal state 
   8 statistics["hitcounts"] = oldvalue + 1
   9 # XXX not thread save!!!!!

On the other hand: Dicts are already thread save! Because of the Global interpreter Lock (GIL) all dict operations are atomic.

To make a really save dict you need to aquire the lock of the dict before you get the value you want to modify and release it after you wrote the data back:

   1 statistics.aquire() # blocks until other thread frees the lock
   2 oldvalue = statistics.get("hitcounts", 0)
   3 statistics["hitcounts"] = oldvalue + 1
   4 statistics.release()

If your code generate invalid internal state you need an even more aggressive locking. You need protect reading by locks, too.

MoinMoin: MultiThreadingSafety (last edited 2007-10-29 19:06:59 by localhost)