Description

Migrating from 1.5.3 to 1.9.2

Steps to reproduce

1. Run "moin --config-dir=/home/Wiki/1.9.2/QAAutomation --wiki-url=http://elway.itgssi.com:8080/QAAutomation migration data"

2. Removed all the '_' from the 3rd column of the rename1.txt file
3. Save save as rename2.txt

4. Run "moin --config-dir=/home/Wiki/1.9.2/QAAutomation --wiki-url=http://elway.itgssi.com:8080/QAAutomation migration data" again

I get this error message:

ValueError: invalid literal for long() with base 10: 'ic_testcases_list.txt'

Example

Component selection

The bug happens for some of my wikis where I delete all the '_' from the right-most column.

It's the wiki's edit-log that seems to cause the problem.

The file /MoinMoin/logfile/editlog.py seems to be having some problems with parsing some lines. In the editlog.py file, the error occurs in the "class EditLog" section, specifically its "def parser" section.

The line in the edit-log causing the error this time is this: 1269557349377804 99999999 ATTNEW Core(2f)Sanity_Regression_Tests_per_Product 172.17.65.168 goldrush-ssi Logic_testcases_list.txt

editlog.py is treaing 'ic_testcases_list.txt' as the result.ed_time_usecs field. But from the line in the edit-log, that string is found in the middle of a different field: "Logic_testcases_list.txt" Error is caused by "ic_testcases_list.txt"

• general

Details

Traceback (most recent call last):
  File "./moin", line 6, in <module>
    run()
  File "/home/Wiki/1.9.2/lib/python2.5/site-packages/MoinMoin/script/moin.py", line 15, in run
    MoinScript().run(showtime=0)
  File "/home/Wiki/1.9.2/lib/python2.5/site-packages/MoinMoin/script/__init__.py", line 138, in run
    self.mainloop()
  File "/home/Wiki/1.9.2/lib/python2.5/site-packages/MoinMoin/script/__init__.py", line 253, in mainloop
    plugin_class(args[2:], self.options).run() # all starts again there
  File "/home/Wiki/1.9.2/lib/python2.5/site-packages/MoinMoin/script/__init__.py", line 138, in run
    self.mainloop()
  File "/home/Wiki/1.9.2/lib/python2.5/site-packages/MoinMoin/script/migration/data.py", line 48, in mainloop
    curr_rev = execute(self, data_dir, curr_rev)
  File "/home/Wiki/1.9.2/lib/python2.5/site-packages/MoinMoin/script/migration/1089999.py", line 23, in execute
    pagenames = script.request.rootpage.getPageList(user='', include_underlay=False)
  File "/home/Wiki/1.9.2/lib/python2.5/site-packages/MoinMoin/Page.py", line 1756, in getPageList
    if not include_underlay and page.getPageStatus()[0]: # is an underlay page
  File "/home/Wiki/1.9.2/lib/python2.5/site-packages/MoinMoin/Page.py", line 486, in getPageStatus
    underlay, path = self.getPageBasePath(use_underlay)
  File "/home/Wiki/1.9.2/lib/python2.5/site-packages/MoinMoin/Page.py", line 452, in getPageBasePath
    pagefile, rev, exists = self.get_rev(use_underlay=0)
  File "/home/Wiki/1.9.2/lib/python2.5/site-packages/MoinMoin/Page.py", line 387, in get_rev
    cache_data = request.cfg.cache.meta.getItem(request, cache_name, cache_key)
  File "/home/Wiki/1.9.2/lib/python2.5/site-packages/MoinMoin/Page.py", line 88, in getItem
    self.refresh(request)
  File "/home/Wiki/1.9.2/lib/python2.5/site-packages/MoinMoin/Page.py", line 115, in refresh
    new_pos, items = elog.news(old_pos)
  File "/home/Wiki/1.9.2/lib/python2.5/site-packages/MoinMoin/logfile/editlog.py", line 262, in news
    for line in self:
  File "/home/Wiki/1.9.2/lib/python2.5/site-packages/MoinMoin/logfile/__init__.py", line 304, in next
    result = self.__next()
  File "/home/Wiki/1.9.2/lib/python2.5/site-packages/MoinMoin/logfile/__init__.py", line 291, in __next
    result = self.parser(self.__buffer.lines[self.__rel_index])
  File "/home/Wiki/1.9.2/lib/python2.5/site-packages/MoinMoin/logfile/editlog.py", line 237, in parser
    result.ed_time_usecs = long(result.ed_time_usecs or '0') # has to be long for py 2.2.x
  ValueError: invalid literal for long() with base 10: 'ic_testcases_list.txt'

Workaround

Discussion

Looks like your edit-log is corrupted (== the structure of the lines is not as excpected). Some old moins had the problem that the comment fields was not escaped good enough, so special chars (like \n) got through into the edit-log, making the next line invalid.

There's no good way the converted could automatically deal with that, so you have to fix this manually, using some editor that preserves tabs and linefeeds.

my response (Apr 6, 2010)--

Is there a size limit for the length of a line in the edit log?

• No. Maybe just check what I told you, this is usually easily fixed with a text editor (like vim or any other that respects tabs and unix-style linefeeds). Also please be aware that there are 2 kinds of edit-logs: the global one (data/edit-log) and one local per page (data/pages/PageName/edit-log). the traceback above looks like the global edit-log causes your problems, but better also look at the local one (usually they are the same, so if one is damaged, maybe the other one is damaged, too).

--Yes, it would appear to be the global edit-log. I used vim to open it. There are no instances of '\n' characters appearing in the middle of any lines followed by text. Each line that does feature a comment appears to have only 1 '\n' character. I'm looking at both the data/edit-log and the data.pre160/edit-log files. Would you happen to know how this might impact the migrated wiki? If there won't be any problems, then I think we can live with this.

• If you get a traceback while migrating data, the migrated data is likely incomplete (because the crash interrupted it in the middle). Btw, you usually won't see a \n, it'ld just make a new line. So what you have to search is a line that doesn't have the structure of all the others. Especially look out for the place where it should have numbers.

Plan

• Priority:
• Assigned to:
• Status: corrupt input data, not a converter bug

CategoryMoinMoinNoBug