Description
For a page with a number of includes, which may contain other includes; and included pages having pictures, total ~40 printed pages and ~20 pictures:
- some pictures fail to load the first time, but load OK after righ-click, Show Picture
the moin.log shows locking and caching errors
- See Also
Configuration
- Python Version
- 2.4.2 (#67, Oct 30 2005, 16:11:18) [MSC v.1310 32 bit (Intel)]
- MoinMoin Version
- Release 1.6.0 [Revision release]
- Web Server
- Windows XP, IIS, CGI
Steps to reproduce
download pages.zip
- place into moin instance
- navigate "Fusion (Full)"
- observe pictures and logs
Example
Here are some log entries
Error in caching.py
2008-01-30 18:04:10,173 ERROR UnboundLocalError: local variable 'data' referenced before assignment
Traceback (most recent call last):
File "d:/Moin/Lib/site-packages\MoinMoin\request\request_cgi.py", line 26, in __init__
RequestBase.__init__(self, properties)
File "d:/Moin/Lib/site-packages\MoinMoin\request\__init__.py", line 251, in __init__
self.user = self.get_user_from_form()
File "d:/Moin/Lib/site-packages\MoinMoin\request\__init__.py", line 647, in get_user_from_form
user_obj=None)
File "d:/Moin/Lib/site-packages\MoinMoin\request\__init__.py", line 658, in get_user_default_unknown
user_obj = self.get_user_default_None(**kw)
File "d:/Moin/Lib/site-packages\MoinMoin\request\__init__.py", line 674, in get_user_default_None
cookie=cookie)
File "d:/Moin/Lib/site-packages\MoinMoin\auth\__init__.py", line 491, in moin_session
setSessionCookie(request, u, securitystringcache=ussc, secidx=secidx)
File "d:/Moin/Lib/site-packages\MoinMoin\auth\__init__.py", line 296, in setSessionCookie
securitystringcache.update(secidx)
File "d:/Moin/Lib/site-packages\MoinMoin\auth\__init__.py", line 94, in update
secrets, lru = self._load()
File "d:/Moin/Lib/site-packages\MoinMoin\auth\__init__.py", line 85, in _load
return self.ce.content()
File "d:/Moin/Lib/site-packages\MoinMoin\caching.py", line 178, in content
data = pickle.loads(data)
Component selection
- caching.py
- auth
- lock.py
Details
Workaround
The attached changes seem to fix the errors.
moin-1.6.1-CachingLockFail.zip
![/!\](/moin_static19/modernized/img/alert.png "/!\")
file attachment wasn't uploaded (moin-1.6.0-CachingLockFail.zip -- older patch)
Look for # BUG ... entries.
-- OlegKobchenko 2008-01-30 23:49:04
I did hg diff after applying these sources to list changes.
1 diff -r 5494361ffa11 MoinMoin/auth/__init__.py
2 --- a/MoinMoin/auth/__init__.py Sun Feb 03 14:41:59 2008 +0100
3 +++ b/MoinMoin/auth/__init__.py Sun Feb 03 17:02:41 2008 +0100
4 @@ -93,7 +93,8 @@ class UserSecurityStringCache:
5 """
6 secrets, lru = self._load()
7 # just move this secret to the front of the LRU queue
8 - lru.remove(secidx)
9 + if secidx in lru: # BUG CachingLockFail
10 + lru.remove(secidx)
11 lru.insert(0, secidx)
12 self.ce.update((secrets, lru))
13
14 diff -r 5494361ffa11 MoinMoin/caching.py
15 --- a/MoinMoin/caching.py Sun Feb 03 14:41:59 2008 +0100
16 +++ b/MoinMoin/caching.py Sun Feb 03 17:02:41 2008 +0100
17 @@ -129,7 +129,7 @@ class CacheEntry:
18 content = pickle.dumps(content, PICKLE_PROTOCOL)
19 elif self.use_encode:
20 content = content.encode(config.charset)
21 - if not self.locking or self.locking and self.wlock.acquire(1.0):
22 + if not self.locking or self.locking and self.wlock.acquire(10.0): # BUG CachingLockFail
23 try:
24 # we do not write content to old inode, but to a new file
25 # so we don't need to lock when we just want to read the file
26 @@ -162,8 +162,9 @@ class CacheEntry:
27 self.request.log("Can't acquire write lock in %s" % self.lock_dir)
28
29 def content(self):
30 + data = "" # BUG CachingLockFail
31 try:
32 - if not self.locking or self.locking and self.rlock.acquire(1.0):
33 + if not self.locking or self.locking and self.rlock.acquire(10.0): # BUG CachingLockFail
34 try:
35 f = open(self._filename(), 'rb')
36 data = f.read()
37 diff -r 5494361ffa11 MoinMoin/util/lock.py
38 --- a/MoinMoin/util/lock.py Sun Feb 03 14:41:59 2008 +0100
39 +++ b/MoinMoin/util/lock.py Sun Feb 03 17:02:41 2008 +0100
40 @@ -110,7 +110,7 @@ class ExclusiveLock:
41 # log('acquired exclusive lock: %s\n' % (self.lockDir, ))
42 return True
43 except OSError, err:
44 - if err.errno != errno.EEXIST:
45 + if err.errno not in [errno.EEXIST,errno.EACCES]: # BUG CachingLockFail
46 raise
47 if self.expire():
48 continue # Try immediately to acquire
49 @@ -170,12 +170,20 @@ class ExclusiveLock:
50
51 def _removeLockDir(self):
52 """ Remove lockDir ignoring 'No such file or directory' errors """
53 - try:
54 - os.rmdir(self.lockDir)
55 - except OSError, err:
56 - if err.errno != errno.ENOENT:
57 + timer = self.timerClass(1) # BUG CachingLockFail
58 + timer.start()
59 + removed = False # else undefined error
60 + while timer.haveTime():
61 + try:
62 + os.rmdir(self.lockDir)
63 + removed = True
64 + break
65 + except OSError, err:
66 + # log( "_removeLockDir delayed\n" )
67 + timer.sleep()
68 + if not removed:
69 + if err.errno != errno.EEXIST:
70 raise
71 -
72
73 class WriteLock(ExclusiveLock):
74 """ Exclusive Read/Write Lock
Discussion
I have not tested this code yet, just did the diff. There is a tab used in def _removeLockDir(self): tabs are interepreted as 8 spaces (two indenting levels). Tabs should be replaced by spaces, see PRP8 for detailed instructions;
- the data = "" change in content() is covering a problem: if it can't access the cache file, it will just return this empty string as cache content (without the upper levels of code knowing that something wrent wrong). This means it will work with wrong data and destroy the previous cache content if it writes this data back to disk.
-- ReimarBauer 2008-02-03 16:13:06
Update: after updating to Moin 1.6.1 the same errors persist. So I had to re-apply these patches. There are a few small corrections based on feedback from ReimarBauer, which I much appreciate. (Diff needs to be updated.)
the data = "" is replaced with cathing the corresponding exception.
excessive TAB is replaced with spaces
-- OlegKobchenko 2008-03-03 21:08:34
I think the condition when it keeps looping should be more precise (not just any OSError, but THE OSError that goes away some milliseconds later). Also, if this is a generic problem on win32, the code should be moved to MoinMoin.util.filesys and specialcased for win32 (see other compat stuff in that module). -- ThomasWaldmann 2008-03-06 18:17:41
If someone wants to have this fixed, please provide more information and a tested, well-formed patch (and see the comments in the paragraph above).
I'm pretty sure this is because of disk indexing or virus scanning software. The other item is marked do-not-fix, but I think saying "Windows SHOULD work, so we aren't going to fix it" is [CENSORED]. Hint: It seems to fail more when Google Desktop Search Indexing is turned on... On the other hand, I understand not wanting to investigate other locking paradigms when it is only Windows+BadService that fails... --TomOehser
Plan
- Priority:
- Assigned to:
Status: workaround by http://hg.moinmo.in/moin/1.8/rev/d85005213fd9 (I tested it with the "Fusion (Full)" page from the pages.zip under Windows 7RC and didn't have failures with 1.8.5).