Change config on the fly, without stooping a long running process moin.

In the current system, we have to stop a long running server each time we want to change a configuration option or add a wiki instance. This is no a big problem, just little annoying when you try to configure a wiki, and change a settting, restart the server, change more, restart the server... and so on. We should look into this later if we think its important to have this.

First try

should be updated with the module-less code

In my fix branch, while refactoring multiconfig, I came up with this simple solution:

   1 _timestamp = {}
   2 def importConfig(name):
   3     """ Import a config file, reloading when file changed
   4 
   5     Each time a config file is imported, a timestamp is saved. If the
   6     file modification time changed, it is reloaded.
   7 
   8     This will make it possible to changed a config file without stooping
   9     a long running process. The next request will reload the config and
  10     use the new settings.
  11 
  12     return a tuple (module, changed). The interface is different from the
  13     built in import, but this makes the calling code much simpler. 
  14     """
  15     # Get timestamp or raise error if there is no such file
  16     try:
  17         timestamp = os.path.getmtime(os.path.join(os.getcwd(), name + '.py'))
  18     except OSError, why:
  19         raise ImportError(why)
  20 
  21     # If the file did not change, return module from sys.modules
  22     module = sys.modules.get(name)
  23     if module and _timestamp.get(name) == timestamp:
  24         return module, 0
  25         
  26     # Reload module and save timestamp
  27     # Using locks make it thread safe (require Python 2.3)
  28     try:
  29         if hasattr(imp, 'acquire_lock'):
  30             imp.acquire_lock()
  31         fp, pathname, description = imp.find_module(name)   
  32         module = imp.load_module(name, fp, pathname, description)
  33         _timestamp[name] = timestamp
  34     finally:
  35         if fp:
  36             fp.close()
  37         if hasattr(imp, 'release_lock'):
  38             imp.release_lock()
  39     return module, 1

This code works nicely in my branch, I can play with the config while Twisted is serving pages, and the next requests just catch the new settings. But this solution have few problems:

• Reload is risky, see next section
• If the module import another module, like moin_config imorting farmconfig, then the other module is not updated. We need more code to recorsive reloading.
• What will happen on python 2.2 where there are no locks when too requests will try to reload the same module?

Reload problem

There is one big problem with this solution: when you reload a module, old names in the module dict are not remved. This can cause hard to find bugs. See the python docs here: http://docs.python.org/lib/built-in-funcs.html

• "When a module is reloaded, its dictionary (containing the module's global variables) is retained. Redefinitions of names will override the old definitions, so this is generally not a problem. If the new version of a module does not define a name that was defined by the old version, the old definition remains."

This small test demonstrate this problem: oldnames.py

module-less solution

We can use a system similar to Twisted tac configuration files. A tac file is a fragment of python code. It is executed using exec, with a dict, so names from the code get into the dict. Then you get the names you want from the dict.

Our config file can be the same file we use today, probably with another extension, to make it mode clear that its not a module, like moin.pcf for "python config file".

   1 def readNameFromFile(path, Name):
   2     """ Read name from file without importing a module
   3 
   4     Using this method, we can change the file at path in runtime, and read
   5     the file again and again, without the "reload issues" of modules.
   6     
   7     Idea stolen from twisted.persisted.sob, maintained by Moshe Zadka.   
   8     
   9     @param path: python source containing name.
  10     @param name: name to get from file
  11     @rtype: object
  12     @return name from file, or None
  13     """
  14     f = file(path)
  15     globals = {}
  16     locals = {}
  17     exec f in globals, locals
  18     f.close()
  19     return locals.get(name, None)

If we use similar system, we can execute our config code each time a config file changes.