Description

If a plugin does a global import the imp base import mechanism of pysupport.importName fails sporadically (always on the first import on the pages, sometimes for all).

Example

This test.py test parser works fine in lib/python/!MoinMoin/parser, but fails in plugin/parser because of the import os. Try to uncomment import os: the parser will work again. Just make a page with a {{{#!test}}} (you need a new-line after the test word) on it.

Added a sample testpage text: TestTest.txt

Details

importName gets an ImportError: No module named path in os. When I check sys.modules['os.path'] I see that it is: <module 'plugin.parser.posixpath' from '/usr/lib/python2.3/posixpath.pyc'> which looks quite mysterious....

To see this you need to print the traceback in the except ImportError section of importName.

Traceback (most recent call last):
  File "/var/www/virtualdomains/moin-main/lib/python/MoinMoin/util/pysupport.py"
, line 64, in importName
    mod = imp.load_module(complete_modulename, md_file, md_fname, md_fd)
  File "/var/www/virtualdomains/moin-main/wiki/data/plugin/parser/VimColor.py", 
line 13, in ?
    import sys, os.path, os, tempfile, re
  File "/usr/lib/python2.3/os.py", line 131, in ?
    from os.path import curdir, pardir, sep, pathsep, defpath, extsep, altsep
ImportError: No module named path

Workaround

1. Remove global imports.

2. use this patch: alternate-importName.diff (deleted) it removes the imp stuff and just add path to sys.path to do the __import__. The path is removed from sys.path after the import, so no pollution occurs. This is the way we import plugins in our epaper project and we encountered no negative sideeffects in the last year using it.

Discussion

The patch is not that good. Adding and removing stuff to sys.path is error prone in multi-threaded environments. -- OliverGraf 2004-10-18 13:29:28

The bug does not happen with py 2.2.3. os.py of 2.2.3 does not do a from os.path import stuff after setting sys.modules['os.path'], as the os.py of 2.3.X does. So this bug seems to be linked to importing os. -- OliverGraf 2004-10-18 14:04:05

The import process seems not completely correct: if the last load_module is called with modulename instead of complete_modulename, the thing works as it should. But that sounds not right, too. We should try to reimplement importName using Python-2.3.3/Docs/imputil/knee.py as an example. -- OliverGraf 2004-10-18 20:36:21

Ok, I have now a working importName function. It does not cache any imports, as before. Caching (via sys.modules) is possible, but it won't work in conjunction with farmwikis, cause all wikis would share one plugin module which maps to different plugin paths (one for each farm wiki). This could be solved with empty dummy modules created by imp.new_module, perhaps named after the farm wiki. -- OliverGraf 2004-10-18 21:45:35

Before I commit this, please test if it works for you. The test should fail without the patch on python 2.3.X.

• a testcase, goes into MoinMoin/_tests/: test_pysupport.py

• the patch for pysupport, try make test without and with this patch: knee-import.diff

-- OliverGraf 2004-10-19 11:22:26

Test Results:

• python 2.2.3 on linux

• python 2.3.4 on linux

• python 2.3.4 on win32

Commited as moin--main--1.3--patch190. -- OliverGraf 2004-10-19 19:44:49

Plan

• Priority: high

• Assigned to: OliverGraf

• Status: fixed in moin--main--1.3--patch190

CategoryMoinMoinBugFixed