Description
[on Debian's moinmoin 1.9, with werkzeug 0.6... needs to be confirmed on plain moinmoin]
Xapian content indexing raise some errors, while indexing some macros like: <<PageList()>> and <<FullSearchCached()>> (in pages HelpIndex and CategoryCategory).
Steps to reproduce
- create a wiki
Add those instruction in the config file:
xapian_search = True xapian_stemming = TrueExecute moin index build --wiki-url=http://localhost/wiki --mode=add
Example
short log:
2010-05-05 07:48:09,508 ERROR MoinMoin.macro:132 Macro PageList (page: 'HelpIndex') raised an exception:
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.5/MoinMoin/macro/__init__.py", line 122, in execute
return execute(self, args)
File "/usr/lib/pymodules/python2.5/MoinMoin/macro/PageList.py", line 32, in execute
sort='page_name')
File "/usr/lib/pymodules/python2.5/MoinMoin/search/__init__.py", line 35, in searchPages
return _get_searcher(request, query, sort, mtime, historysearch, **kw).run()
File "/usr/lib/pymodules/python2.5/MoinMoin/search/builtin.py", line 262, in run
hits, estimated_hits = self._search()
File "/usr/lib/pymodules/python2.5/MoinMoin/search/Xapian/search.py", line 53, in _search
search_results = index.search(self.query, sort=self.sort, historysearch=self.historysearch)
File "/usr/lib/pymodules/python2.5/MoinMoin/search/builtin.py", line 135, in search
return self._search(query, **kw)
File "/usr/lib/pymodules/python2.5/MoinMoin/search/Xapian/indexing.py", line 183, in _search
hits = searcher.search(query, 0, document_count, **kw)
File "/usr/lib/python2.5/site-packages/xappy/searchconnection.py", line 1635, in search
raise _errors.SearchError("Field %r was not indexed for sorting" % sortby)
SearchError: Field 'pagename' was not indexed for sorting(click on comment for more)
Component selection
- Xapian
Details
MoinMoin Version |
1.9 from Debian + some HG patches |
OS and Version |
|
Python Version |
2.5 |
Server Setup |
WSGI |
Server Details |
Xapian |
Language you are using the wiki in (set in the browser/UserPreferences) |
|
Workaround
Discussion
Hmm, looks a bit like you try to update an invalid index.
Could you try to (quoting from docs/CHANGES):
moin index build --mode=buildnewindex <stop wiki> moin index build --mode=usenewindex <start wiki>
AFTER this, try to reproduce the problem.
- I thought --mode=add can be used for a newly created wiki? Should one use the safe 2-stage approach in that case, too?
I see the same errors on "plain" MoinMoin 1.9.2 with Xapian 1.2.2, Python 2.6.5 .
- I tried to find the conditions to reproduce it. At first it looked like it always happens on my farm with new wikis. But after some experimenting the error was completely gone. I installed a fresh underlay and now I can reproduce the problem:
- new wiki
- --mode=add
- fresh underlay
- one of the pages with the mentioned macros contains no "text_html" file in the cache dir (i.e. underlay/pages/HelpIndex/cache/)
- if the page was visited once (e.g. shared underlay) the file is created and the error is not raised
- cache/pagelinks is generated when indexing
- As the indexing is succesful anyway and the error is gone with another run with --mode=add or the 2-stage approach I can not say if this causes any real problems.
The error is gone because now you have a cache file of the pagecontent. If you clean the page cache it will reappear (moin ... maint cleancache) see HelpOnMoinCommand
- Nope. As mentioned above "pagecontent" cache is not created when indexing (only pagelinks), it is created when visiting the wiki page. So I would say the error is gone in this case, as an index now exists.
What does it do when it executes the macro?
- You mean when indexing (with or w/o error)? I have no idea if there is a difference in the indexes build and I did not examine that.
- To get the error again, I have to do exactly what I mentioned above. A new wiki without pre-existing index and an underlay without "pagecontent" file. Both conditions must be met and the "add" mode must be used to get the error.
- So using "maint cleancache" will lead to the error only together with a new wiki or one without index.
Did you tried? I remember excactly the same behaviour before I refactored all old help pages for some macro calls. At that time I could reproduce it after calling maint cleancache
- Hmm, I always manually removed pagecontent and/or the xapian index. The problem is that in my farm setup I have a shared underlay and I have no idea what happens in that case. To answer the question I never used "maint cleancache" while reproducing this error, but I thought it only cleans the cache of pages and maybe some other places (i18n).
The macros can have a dependency for caching too
- Nope. As mentioned above "pagecontent" cache is not created when indexing (only pagelinks), it is created when visiting the wiki page. So I would say the error is gone in this case, as an index now exists.
- I tried to find the conditions to reproduce it. At first it looked like it always happens on my farm with new wikis. But after some experimenting the error was completely gone. I installed a fresh underlay and now I can reproduce the problem:
Can you please look at the raw text of the pages you got this error. Make sure it is the uptodate underlay page of that content. (compare it with http://master19.moinmo.in/HelpIndex?action=raw)
Plan
- Priority:
- Assigned to:
- Status:
2010-05-05 07:47:26,860 WARNING MoinMoin.log:139 using logging configuration read from built-in fallback in MoinMoin.log module! 2010-05-05 07:47:26,951 WARNING MoinMoin.log:111 /usr/lib/pymodules/python2.5/pygments/plugin.py:39: UserWarning: Module pygments was already imported from /usr/lib/pymodules/python2.5/pygments/__init__.py, but /usr/lib/pymodules/python2.5 is being added to sys.path 2010-05-05 07:47:27,088 INFO MoinMoin.config.multiconfig:93 using farm config: /etc/moin/farmconfig.py 2010-05-05 07:47:27,112 INFO MoinMoin.config.multiconfig:127 using wiki config: /etc/moin/mywiki.py 2010-05-05 07:47:29,503 INFO MoinMoin.search.Xapian.indexing:551 indexing 243 pages... 2010-05-05 07:47:30,020 ERROR MoinMoin.macro:132 Macro FullSearchCached (page: 'ModèleDeCatégorie') raised an exception: Traceback (most recent call last): File "/usr/lib/pymodules/python2.5/MoinMoin/macro/__init__.py", line 122, in execute return execute(self, args) File "/usr/lib/pymodules/python2.5/MoinMoin/macro/FullSearch.py", line 106, in execute results = search.searchPages(request, needle, sort='page_name') File "/usr/lib/pymodules/python2.5/MoinMoin/search/__init__.py", line 35, in searchPages return _get_searcher(request, query, sort, mtime, historysearch, **kw).run() File "/usr/lib/pymodules/python2.5/MoinMoin/search/builtin.py", line 262, in run hits, estimated_hits = self._search() File "/usr/lib/pymodules/python2.5/MoinMoin/search/Xapian/search.py", line 53, in _search search_results = index.search(self.query, sort=self.sort, historysearch=self.historysearch) File "/usr/lib/pymodules/python2.5/MoinMoin/search/builtin.py", line 135, in search return self._search(query, **kw) File "/usr/lib/pymodules/python2.5/MoinMoin/search/Xapian/indexing.py", line 183, in _search hits = searcher.search(query, 0, document_count, **kw) File "/usr/lib/python2.5/site-packages/xappy/searchconnection.py", line 1635, in search raise _errors.SearchError("Field %r was not indexed for sorting" % sortby) SearchError: Field 'pagename' was not indexed for sorting 2010-05-05 07:47:58,045 ERROR MoinMoin.macro:132 Macro PageList (page: 'IndexDeL'Aide') raised an exception: Traceback (most recent call last): File "/usr/lib/pymodules/python2.5/MoinMoin/macro/__init__.py", line 122, in execute return execute(self, args) File "/usr/lib/pymodules/python2.5/MoinMoin/macro/PageList.py", line 32, in execute sort='page_name') File "/usr/lib/pymodules/python2.5/MoinMoin/search/__init__.py", line 35, in searchPages return _get_searcher(request, query, sort, mtime, historysearch, **kw).run() File "/usr/lib/pymodules/python2.5/MoinMoin/search/builtin.py", line 262, in run hits, estimated_hits = self._search() File "/usr/lib/pymodules/python2.5/MoinMoin/search/Xapian/search.py", line 53, in _search search_results = index.search(self.query, sort=self.sort, historysearch=self.historysearch) File "/usr/lib/pymodules/python2.5/MoinMoin/search/builtin.py", line 135, in search return self._search(query, **kw) File "/usr/lib/pymodules/python2.5/MoinMoin/search/Xapian/indexing.py", line 183, in _search hits = searcher.search(query, 0, document_count, **kw) File "/usr/lib/python2.5/site-packages/xappy/searchconnection.py", line 1635, in search raise _errors.SearchError("Field %r was not indexed for sorting" % sortby) SearchError: Field 'pagename' was not indexed for sorting 2010-05-05 07:48:01,144 ERROR MoinMoin.macro:132 Macro FullSearchCached (page: 'CatégorieCatégorie') raised an exception: Traceback (most recent call last): File "/usr/lib/pymodules/python2.5/MoinMoin/macro/__init__.py", line 122, in execute return execute(self, args) File "/usr/lib/pymodules/python2.5/MoinMoin/macro/FullSearch.py", line 106, in execute results = search.searchPages(request, needle, sort='page_name') File "/usr/lib/pymodules/python2.5/MoinMoin/search/__init__.py", line 35, in searchPages return _get_searcher(request, query, sort, mtime, historysearch, **kw).run() File "/usr/lib/pymodules/python2.5/MoinMoin/search/builtin.py", line 262, in run hits, estimated_hits = self._search() File "/usr/lib/pymodules/python2.5/MoinMoin/search/Xapian/search.py", line 53, in _search search_results = index.search(self.query, sort=self.sort, historysearch=self.historysearch) File "/usr/lib/pymodules/python2.5/MoinMoin/search/builtin.py", line 135, in search return self._search(query, **kw) File "/usr/lib/pymodules/python2.5/MoinMoin/search/Xapian/indexing.py", line 183, in _search hits = searcher.search(query, 0, document_count, **kw) File "/usr/lib/python2.5/site-packages/xappy/searchconnection.py", line 1635, in search raise _errors.SearchError("Field %r was not indexed for sorting" % sortby) SearchError: Field 'pagename' was not indexed for sorting 2010-05-05 07:48:02,416 ERROR MoinMoin.macro:132 Macro FullSearchCached (page: 'CategoryHomepage') raised an exception: Traceback (most recent call last): File "/usr/lib/pymodules/python2.5/MoinMoin/macro/__init__.py", line 122, in execute return execute(self, args) File "/usr/lib/pymodules/python2.5/MoinMoin/macro/FullSearch.py", line 106, in execute results = search.searchPages(request, needle, sort='page_name') File "/usr/lib/pymodules/python2.5/MoinMoin/search/__init__.py", line 35, in searchPages return _get_searcher(request, query, sort, mtime, historysearch, **kw).run() File "/usr/lib/pymodules/python2.5/MoinMoin/search/builtin.py", line 262, in run hits, estimated_hits = self._search() File "/usr/lib/pymodules/python2.5/MoinMoin/search/Xapian/search.py", line 53, in _search search_results = index.search(self.query, sort=self.sort, historysearch=self.historysearch) File "/usr/lib/pymodules/python2.5/MoinMoin/search/builtin.py", line 135, in search return self._search(query, **kw) File "/usr/lib/pymodules/python2.5/MoinMoin/search/Xapian/indexing.py", line 183, in _search hits = searcher.search(query, 0, document_count, **kw) File "/usr/lib/python2.5/site-packages/xappy/searchconnection.py", line 1635, in search raise _errors.SearchError("Field %r was not indexed for sorting" % sortby) SearchError: Field 'pagename' was not indexed for sorting 2010-05-05 07:48:03,374 ERROR MoinMoin.macro:132 Macro FullSearchCached (page: 'CategoryTemplate') raised an exception: Traceback (most recent call last): File "/usr/lib/pymodules/python2.5/MoinMoin/macro/__init__.py", line 122, in execute return execute(self, args) File "/usr/lib/pymodules/python2.5/MoinMoin/macro/FullSearch.py", line 106, in execute results = search.searchPages(request, needle, sort='page_name') File "/usr/lib/pymodules/python2.5/MoinMoin/search/__init__.py", line 35, in searchPages return _get_searcher(request, query, sort, mtime, historysearch, **kw).run() File "/usr/lib/pymodules/python2.5/MoinMoin/search/builtin.py", line 262, in run hits, estimated_hits = self._search() File "/usr/lib/pymodules/python2.5/MoinMoin/search/Xapian/search.py", line 53, in _search search_results = index.search(self.query, sort=self.sort, historysearch=self.historysearch) File "/usr/lib/pymodules/python2.5/MoinMoin/search/builtin.py", line 135, in search return self._search(query, **kw) File "/usr/lib/pymodules/python2.5/MoinMoin/search/Xapian/indexing.py", line 183, in _search hits = searcher.search(query, 0, document_count, **kw) File "/usr/lib/python2.5/site-packages/xappy/searchconnection.py", line 1635, in search raise _errors.SearchError("Field %r was not indexed for sorting" % sortby) SearchError: Field 'pagename' was not indexed for sorting 2010-05-05 07:48:05,243 ERROR MoinMoin.macro:132 Macro FullSearchCached (page: 'CategoryCategory') raised an exception: Traceback (most recent call last): File "/usr/lib/pymodules/python2.5/MoinMoin/macro/__init__.py", line 122, in execute return execute(self, args) File "/usr/lib/pymodules/python2.5/MoinMoin/macro/FullSearch.py", line 106, in execute results = search.searchPages(request, needle, sort='page_name') File "/usr/lib/pymodules/python2.5/MoinMoin/search/__init__.py", line 35, in searchPages return _get_searcher(request, query, sort, mtime, historysearch, **kw).run() File "/usr/lib/pymodules/python2.5/MoinMoin/search/builtin.py", line 262, in run hits, estimated_hits = self._search() File "/usr/lib/pymodules/python2.5/MoinMoin/search/Xapian/search.py", line 53, in _search search_results = index.search(self.query, sort=self.sort, historysearch=self.historysearch) File "/usr/lib/pymodules/python2.5/MoinMoin/search/builtin.py", line 135, in search return self._search(query, **kw) File "/usr/lib/pymodules/python2.5/MoinMoin/search/Xapian/indexing.py", line 183, in _search hits = searcher.search(query, 0, document_count, **kw) File "/usr/lib/python2.5/site-packages/xappy/searchconnection.py", line 1635, in search raise _errors.SearchError("Field %r was not indexed for sorting" % sortby) SearchError: Field 'pagename' was not indexed for sorting 2010-05-05 07:48:09,486 ERROR MoinMoin.macro:132 Macro FullSearch (page: 'CatégoriePagePersonnelle') raised an exception: Traceback (most recent call last): File "/usr/lib/pymodules/python2.5/MoinMoin/macro/__init__.py", line 122, in execute return execute(self, args) File "/usr/lib/pymodules/python2.5/MoinMoin/macro/FullSearch.py", line 106, in execute results = search.searchPages(request, needle, sort='page_name') File "/usr/lib/pymodules/python2.5/MoinMoin/search/__init__.py", line 35, in searchPages return _get_searcher(request, query, sort, mtime, historysearch, **kw).run() File "/usr/lib/pymodules/python2.5/MoinMoin/search/builtin.py", line 262, in run hits, estimated_hits = self._search() File "/usr/lib/pymodules/python2.5/MoinMoin/search/Xapian/search.py", line 53, in _search search_results = index.search(self.query, sort=self.sort, historysearch=self.historysearch) File "/usr/lib/pymodules/python2.5/MoinMoin/search/builtin.py", line 135, in search return self._search(query, **kw) File "/usr/lib/pymodules/python2.5/MoinMoin/search/Xapian/indexing.py", line 183, in _search hits = searcher.search(query, 0, document_count, **kw) File "/usr/lib/python2.5/site-packages/xappy/searchconnection.py", line 1635, in search raise _errors.SearchError("Field %r was not indexed for sorting" % sortby) SearchError: Field 'pagename' was not indexed for sorting