Requirements

The new storage code must support or be extendable to support the following requirements:

• Support multiple Backends, e.g. File, SQL or SVN
• Compatible with current file backend implementation
• ACL Checking
• Locking

• MimeTypes (handle Attachments / Pages / Users)

• High performance via lazy loading, only saving the changes, caching and optimizing the file backend
• Handle Data and Metadata
• Implement the local and global edit-log as metadata / cache
• Some kind of query interface

Proposed Storage Design

Internal Interface

http://hg.moinmo.in/moin/1.7-storage-hwendel/file/244ab8bab48b/MoinMoin/storage/interfaces.py

Implementation for 1.6 compatibility

http://hg.moinmo.in/moin/1.7-storage-hwendel/file/244ab8bab48b/MoinMoin/storage/backends/moin16.py

External Interface

http://hg.moinmo.in/moin/1.7-storage-hwendel/file/244ab8bab48b/MoinMoin/storage/external.py

Using the External Interface

The entry point to the external interface is the ItemCollection which takes two arguments: the backend to operate on and a request object which can be None. To initialize the ItemCollection use the following code:

   1 item_collection = ItemCollection(request.cfg.data_backend, request)

The ItemCollection behaves like a dictionary and has an additional operation to create an item:

   1 # get an item
   2 item_test = item_collection["test"]
   3 
   4 # delete an item
   5 del item["test2"]
   6 
   7 # create a new item
   8 item_test = item_collection.new_item("test3")

Now you have an Item object which contains all revisions accessable like a dictionary again, for creating revisions there is a new method. Before you do any write operation on an item you have to lock it to avoid concurrent modifications. When you forget to lock the item you will get a LockingError when trying to modify the item. Don't forget to unlock the item at the end.

   1 # get revision one
   2 revision_one = item_test[1]
   3 
   4 # lock item
   5 item_test.lock = True
   6 
   7 # create new revision
   8 revision_two = item_test.new_revision()

The revision has two important properties, metadata and data. The metadata is a dictionary again, the data a file like object. After changing the data and metadata you have to call save() on the revision.

   1 # write metadata
   2 revision_one.metadata["key"] = "value"
   3 
   4 # write data
   5 revision_one.data.write("test...")
   6 
   7 # save
   8 revision_one.save()
   9 
  10 # unlock the item
  11 item_test.lock = False

Error Hierarchy

http://hg.moinmo.in/moin/1.7-storage-hwendel/file/244ab8bab48b/MoinMoin/storage/error.py

Metadata Keys

There current set of metadata keys is described in the following. They are accessible as properties of a Revision.

Per Item Metadata:

• lock (whether the page is edit-locked or not)
• edit-lock

Per Revision Metadata:

• mtime (Timestamp the change, seconds since 1.1.1970)
• userid (The user id of the editor)
• addr (The ip address of the editor)
• hostname (The hostname of the editor)
• size (Size of the page)
• comment
• acl (a list of ACLs)
• deleted (whether the page is deleted)
• action (currently mostly SAVE, or SAVE/REVERT or ...)
• extra (Some extra stuff, e.g. when renaming a page)

Configuration

See HelpOnConfiguration/Storage.

ACLs

ACL checking is done on every access to the Metadata, Data, Revision and Item methods. When the user, which can be taken from the request which is passed to the ItemCollection, does not have the correct access rights an ACLError will be raised. Currently you can already get the ACLs by the corresponding property on the Item. The checking and exception raising is not yet implemented.

Locking

Locking is done on two levels. On the Item level and on the Backend level

Only one thread can modify an Item simultaneously, which is indicated by the lock property of an Item. This avoids transactional corruptions. Other threads will have to wait for the lock getting released or an LockingError will be raised. Reading the item data is still possible.

The Backend implementation itself has to make sure that the operations are atomic and consistent. For unix this is true in the fs_moin16.py code, for windows not. See further comments there.

Status

The current status is described here:

http://hg.moinmo.in/moin/1.7-storage-hwendel/file/tip/docs/CHANGES.storage