Description

With an un-stemmable default language, Xapian index fails to build with PyStemmer enabled.

Steps to reproduce

1. Install Xapian 1.0.6 and PyStemmer 1.0.1.

2. Add some configure options in wikiconfig.py like this:

      1     language_default = 'zh'
      2     xapian_search = True
      3     xapian_index_dir = None
      4     xapian_stemming = True
      5     xapian_index_history = True
   

3. Open a console, switch to the wiki directory, and run

   moin index build --mode=rebuild

4. Processing stops with a traceback.

Component selection

• search engine

Details

• Details of traceback

  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "D:\Python25\lib\site-packages\MoinMoin\script\moin.py", line 15, in run
      MoinScript().run(showtime=0)
    File "D:\Python25\lib\site-packages\MoinMoin\script\__init__.py", line 138, in
   run
      self.mainloop()
    File "D:\Python25\lib\site-packages\MoinMoin\script\__init__.py", line 251, in
   mainloop
      plugin_class(args[2:], self.options).run() # all starts again there
    File "D:\Python25\lib\site-packages\MoinMoin\script\__init__.py", line 138, in
   run
      self.mainloop()
    File "D:\Python25\lib\site-packages\MoinMoin\script\index\build.py", line 56,
  in mainloop
      self.command()
    File "D:\Python25\lib\site-packages\MoinMoin\script\index\build.py", line 63,
  in command
      Index(self.request).indexPages(self.files, self.options.mode)
    File "D:\Python25\Lib\site-packages\MoinMoin\search\builtin.py", line 263, in
  indexPages
      self._index_pages(request, files, mode)
    File "D:\Python25\lib\site-packages\MoinMoin\search\Xapian.py", line 674, in _
  index_pages
      mode)
    File "D:\Python25\lib\site-packages\MoinMoin\search\Xapian.py", line 551, in _
  index_page
      id = writer.index(doc)
    File "D:\Python25\Lib\site-packages\MoinMoin\support\xapwrap\index.py", line 8
  76, in index
      xapDoc = doc.toXapianDocument(self.indexValueMap, self.prefixMap)
    File "D:\Python25\lib\site-packages\MoinMoin\support\xapwrap\document.py", lin
  e 143, in toXapianDocument
      analyzer = self.analyzerFactory()
    File "D:\Python25\lib\site-packages\MoinMoin\search\Xapian.py", line 60, in <l
  ambda>
      return (lambda: WikiAnalyzer(request, language))
    File "D:\Python25\lib\site-packages\MoinMoin\search\Xapian.py", line 99, in __
  init__
      self.stemmer = Stemmer(language)
    File "Stemmer.pyx", line 118, in Stemmer.Stemmer.__init__
  TypeError: exceptions must be strings, classes, or instances, not exceptions.Key
  Error

Workaround

• Patch Lib\site-packages\MoinMoin\search\Xapian.py like this:

     1 class WikiAnalyzer:
     2     #...Skip for brevity...
     3 
     4     def __init__(self, request=None, language=None):
     5         """
     6         @param request: current request
     7         @param language: if given, the language in which to stem words
     8         """
     9         if request and request.cfg.xapian_stemming and language:
    10             #self.stemmer = Stemmer(language)
    11             # Catch the exceptions
    12             try:
    13                 # if there is no exception, lang is stemmable
    14                 self.stemmer = Stemmer(language)
    15             except (KeyError, TypeError):
    16                 # lang is not stemmable or not available, fallback to 'en'
    17                 self.stemmer = Stemmer('en')
    18         else:
    19             self.stemmer = None
    20 
    21     # ...Skip for brevity...
  

• Stop the server, and build the index again. It should be OK now.

Discussion

Related bug: ../1.6devXapianPyStemmerFails

Plan

• Priority:
• Assigned to:

• Status: fixed in 1.6 by http://hg.moinmo.in/moin/1.6/rev/97411d558ec0 and 1.7 by http://hg.moinmo.in/moin/1.7/rev/7d9b8040e3be

CategoryMoinMoinBugFixed