1 2010-04-02T00:07:01  *** pchen has joined #moin-dev
   2 2010-04-02T00:07:14  <pchen> hello
   3 2010-04-02T00:07:25  <pchen> I'm getting an error when running /bin/moin
   4 2010-04-02T00:07:44  <pchen> The error occurs when /MoinMoin/logfile/editlog.py is called by the moin script
   5 2010-04-02T00:45:33  *** pchen has left #moin-dev
   6 2010-04-02T00:50:17  *** valeuf_ has joined #moin-dev
   7 2010-04-02T00:52:49  *** valeuf has quit IRC
   8 2010-04-02T00:52:49  *** valeuf_ is now known as valeuf
   9 2010-04-02T04:42:35  *** AshishG has quit IRC
  10 2010-04-02T06:02:18  *** moin0 has joined #moin-dev
  11 2010-04-02T06:02:26  <moin0> hi
  12 2010-04-02T06:02:43  *** moin0 is now known as yo_colombia
  13 2010-04-02T06:02:57  <yo_colombia> is too hard python?
  14 2010-04-02T06:37:34  *** yo_colombia has quit IRC
  15 2010-04-02T09:06:23  *** valeuf has quit IRC
  16 2010-04-02T09:12:13  *** valeuf has joined #moin-dev
  17 2010-04-02T11:07:01  *** selevt has joined #moin-dev
  18 2010-04-02T15:42:01  *** JosefMeier has joined #moin-dev
  19 2010-04-02T15:42:18  <JosefMeier> Moin
  20 2010-04-02T15:48:54  *** selevt_ has joined #moin-dev
  21 2010-04-02T15:52:23  <JosefMeier> dreimark: Do you have news about the xapian probs in 1.9.2 ?
  22 2010-04-02T15:52:25  *** selevt has quit IRC
  23 2010-04-02T15:53:42  <dreimark> JosefMeier: i have a patch added to the page
  24 2010-04-02T15:54:22  <dreimark> http://moinmo.in/MoinMoinBugs/1.9.2XapianRegexNeedsCase?action=AttachFile&do=view&target=xapian_re_search_opt.patch
  25 2010-04-02T16:12:53  *** JosefMeier has quit IRC
  26 2010-04-02T16:58:36  *** JosefMeier has joined #moin-dev
  27 2010-04-02T16:59:14  <JosefMeier> dreimark: Thanks for the info. I'll try that patch. Is this patch also installed on the official moin homepage?
  28 2010-04-02T17:03:37  <dreimark> JosefMeier: not yet
  29 2010-04-02T17:14:07  <ThomasWaldmann> re
  30 2010-04-02T17:18:25  * ThomasWaldmann had a small chat with lfrodrigues about searching/indexing
  31 2010-04-02T17:25:18  <ThomasWaldmann> btw, that xapian bug is still much too unclear / confused. cause of the problem is unclear and the patch is unclear also.
  32 2010-04-02T17:28:45  <ThomasWaldmann> in general, one should be able to clearly identify the cause of a bug (== point to it in the source) before one starts "fixing" it
  33 2010-04-02T17:35:25  <dreimark> the graphic shows exactly the problem we have nothing to search in
  34 2010-04-02T17:37:02  *** amartani has joined #moin-dev
  35 2010-04-02T17:40:24  <dreimark> have a look at it, we have no content to search by xapian
  36 2010-04-02T17:43:36  <ThomasWaldmann> (!) "case:" triggers post-processing of Xapian search results by MoinSearch (because xapian indexing/search is not case-sensitive). Strange: MoinSearch will get fed with a pagelist pre-selected by xapian search. So if xapian search finds nothing, post-processing should not help either.
  37 2010-04-02T17:43:42  <ThomasWaldmann> still unanswered
  38 2010-04-02T17:45:16  <JosefMeier> ThomasWaldmann, dreimark: Do you think that the bug is in xapian itself?
  39 2010-04-02T17:45:24  <ThomasWaldmann> Because regex search uses match for comparison this will only give a result if there is a line or a page title given which starts with "bindi".
  40 2010-04-02T17:45:31  <ThomasWaldmann> that sounds like another bug...
  41 2010-04-02T17:49:57  <ThomasWaldmann> JosefMeier: likely no, our code is just not right
  42 2010-04-02T17:52:33  <ThomasWaldmann> and pumping all wiki content (optionally even all page revisions) into the xapian index doesn't feel right either
  43 2010-04-02T17:52:36  <dreimark> yes, and such stuff you find also only if you look at the code
  44 2010-04-02T17:52:54  <dreimark> but what if this the only one solution?
  45 2010-04-02T17:56:31  <dreimark> case search is selecting MoinSearch
  46 2010-04-02T17:59:37  <dreimark> JosefMeier: i think i have not stopped the refactoring early enough. in the past we did a regex text search by moinsearch now we select the searcher and that searches then
  47 2010-04-02T18:00:54  <dreimark> it tries to make a regex search in content of the xapian database
  48 2010-04-02T18:01:23  <dreimark> but we don't have content there
  49 2010-04-02T18:02:28  <dreimark> so the patch makes a storage backend with duplicated content
  50 2010-04-02T18:03:49  <dreimark> and that can cost some space
  51 2010-04-02T18:11:49  <dreimark> ThomasWaldmann: http://hg.moinmo.in/moin/1.9/annotate/689e2b04bd4d/MoinMoin/search/Xapian/search.py#l78
  52 2010-04-02T18:12:02  <dreimark> i think this is the cause for the case results
  53 2010-04-02T18:24:49  <dreimark> that line just makes not that post-processing it just did a search
  54 2010-04-02T18:28:57  <dreimark> ThomasWaldmann: if we have nothing found by xapian pages is []
  55 2010-04-02T18:29:01  <dreimark> and then http://hg.moinmo.in/moin/1.9/file/689e2b04bd4d/MoinMoin/search/builtin.py#l383
  56 2010-04-02T18:29:17  <dreimark> makes a complete new search by the moinsearch
  57 2010-04-02T18:29:38  <JosefMeier> dreimark: Is it possible to search in attachments also with your patch?
  58 2010-04-02T18:30:56  <dreimark> just asking or found that i broke it or was it broken before too?
  59 2010-04-02T18:34:00  <dreimark> JosefMeier: because the attachment search test looks not broken and does not fail after adding my patch it should work
  60 2010-04-02T18:36:29  * dreimark looks again at the test
  61 2010-04-02T18:39:28  <JosefMeier> dreimark: just asking
  62 2010-04-02T18:41:13  *** amartani has quit IRC
  63 2010-04-02T18:56:01  <dreimark> JosefMeier: if you find an other issue please create a new bug report.
  64 2010-04-02T18:56:31  <dreimark> we don't fix all these bugs in one cs
  65 2010-04-02T19:04:49  <JosefMeier> dreimark: ok. What's the result of your discussion with Thomas about your patch? Does it solve the problem or do you think there has more work to be done?
  66 2010-04-02T19:12:06  <dreimark> if i look at the backlog we don't have a discussion, we both told lines when the other one was not online.
  67 2010-04-02T19:12:56  <dreimark> a smarter solution would be nicer but i doubt that there is one
  68 2010-04-02T19:13:56  <dreimark> also if xapian itselfs could handle regex search terms that would be the best solution, but currently it isn't
  69 2010-04-02T19:14:10  <dreimark> possible
  70 2010-04-02T19:30:34  <ThomasWaldmann> dreimark: space is not the only issue with that approach
  71 2010-04-02T19:31:53  <ThomasWaldmann> but first doing a regex search over all content just to build potentially huge ORed xapian expressions that finally will find the same stuff again...
  72 2010-04-02T19:37:37  <dreimark> that is quite expensive
  73 2010-04-02T19:41:16  <ThomasWaldmann> that is quite insane :)
  74 2010-04-02T19:41:49  <dreimark> but it is supposed to have these xapian expressions
  75 2010-04-02T19:42:14  <ThomasWaldmann> for some rather limited amount of data (like item names), that's doable, but for all content...
  76 2010-04-02T19:42:51  <dreimark> i am sometimes not sure if one has more content by attachments than by pages
  77 2010-04-02T19:43:09  <ThomasWaldmann> that doesn't matter
  78 2010-04-02T19:43:55  <dreimark> that also could have bid xapian expressions
  79 2010-04-02T19:44:13  <dreimark> it just not have the content regex search beforhand
  80 2010-04-02T19:48:22  <ThomasWaldmann> it doesn't matter whether you process huge amounts of data coming from page revisions or from attachments
  81 2010-04-02T19:48:58  * ThomasWaldmann runs py.test to see if we have a failing search test
  82 2010-04-02T19:49:05  <dreimark> now i am confused
  83 2010-04-02T19:49:10  <ThomasWaldmann> if not, there should be one that identifies that problem
  84 2010-04-02T19:50:11  * dreimark has only the test failing which esyr has not fixed
  85 2010-04-02T19:50:46  <dreimark> and a pep8 failure
  86 2010-04-02T19:52:15  *** valeuf has quit IRC
  87 2010-04-02T19:52:31  * ThomasWaldmann fixes some failures
  88 2010-04-02T19:54:28  <CIA-55> Thomas Waldmann <tw AT waldmann-edv DOT de> default * 5640:3cb7a64eaef5 1.9/MoinMoin/formatter/text_docbook.py: fix pep8 failures
  89 2010-04-02T19:54:45  *** valeuf has joined #moin-dev
  90 2010-04-02T20:04:04  <ThomasWaldmann> 2 parser/formatter tests are broken
  91 2010-04-02T20:06:03  <ThomasWaldmann> but all search tests work
  92 2010-04-02T20:07:47  <dreimark> they work because we don't have much content search tests
  93 2010-04-02T20:09:43  <ThomasWaldmann> can you add a simple test that fails for that bug?
  94 2010-04-02T20:10:05  <dreimark> yes
  95 2010-04-02T20:32:11  *** valeuf has quit IRC
  96 2010-04-02T20:33:12  *** grzywacz has joined #moin-dev
  97 2010-04-02T21:13:46  *** valeuf has joined #moin-dev
  98 2010-04-02T21:19:17  <CIA-55> Reimar Bauer <rb.proj AT googlemail DOT com> default * 5641:a0fdc0e059cc 1.9/MoinMoin/search/_tests/test_search.py: test_search: xapian search test for regex content search
  99 2010-04-02T21:22:13  <dreimark> ThomasWaldmann: a l33t string for regex search should not give any collision, the others tests i have had before currently have that problem
 100 2010-04-02T21:24:11  <dreimark> i used a new page because of too much test dependencies of the crap tests
 101 2010-04-02T21:52:55  * ThomasWaldmann simplifies it
 102 2010-04-02T21:54:58  * ThomasWaldmann curses search tests
 103 2010-04-02T22:06:30  <CIA-55> Thomas Waldmann <tw AT waldmann-edv DOT de> default * 5642:226e1b0aa79c 1.9/MoinMoin/search/_tests/test_search.py: search tests: simplify and extend test for regex search
 104 2010-04-02T22:06:36  <ThomasWaldmann> dreimark: look there
 105 2010-04-02T22:08:32  <dreimark> hmm, you killed the l33t text
 106 2010-04-02T22:09:28  <dreimark> me was thinking about converting all of that strings to leet
 107 2010-04-02T22:09:51  <dreimark> because after some time my testwiki is almost all a dictionary
 108 2010-04-02T22:10:00  <dreimark> and i have lots of false hits
 109 2010-04-02T22:11:06  <dreimark> assert [u'MoinMoinBugs/1.6.0XapianWeigthAndSortingOfRe', u'ContentSearchLower'] == ['ContentSearchLower']
 110 2010-04-02T22:11:24  <dreimark> ThomasWaldmann: 4 failed, 154 passed, 1 skipped in 30.69 seconds
 111 2010-04-02T22:26:33  <JosefMeier> Is it necessary to call "getPage" in a multicall session in xmlrpc? Why??? In my c# xmlrpc-lib there is no multicall support?
 112 2010-04-02T22:27:18  <dreimark> yes
 113 2010-04-02T22:27:32  <dreimark> because of security
 114 2010-04-02T22:29:15  <JosefMeier> I used "applyAuthToken" before getPage but in no multicall session. It fails. I assume that is normal behaviour than?
 115 2010-04-02T22:30:00  * ThomasWaldmann kills that len() crap in search tests
 116 2010-04-02T22:30:58  <ThomasWaldmann> JosefMeier: applyAuthToken only works for same call, thus you need multicall
 117 2010-04-02T22:34:17  <JosefMeier> Is there an other way for auth other than applyauthtoken which doesn't need multicalls?
 118 2010-04-02T22:36:21  * dreimark the len may be not the problem besides for extending test, the words make more problems
 119 2010-04-02T22:40:03  <ThomasWaldmann> no
 120 2010-04-02T22:40:24  <ThomasWaldmann> len() is a major problem
 121 2010-04-02T22:41:21  <ThomasWaldmann> if you search for 2 apples and i give you 2 bananas, you are satisfied?
 122 2010-04-02T22:41:29  *** valeuf has quit IRC
 123 2010-04-02T22:42:33  <dreimark> ThomasWaldmann: hehe, depends on I don't like apples
 124 2010-04-02T22:43:14  <dreimark> but the search probably won't know and i get 2 apples and 2 bananas
 125 2010-04-02T22:44:34  *** valeuf has joined #moin-dev
 126 2010-04-02T22:58:44  *** valeuf has quit IRC
 127 2010-04-02T22:59:43  *** JosefMeier has quit IRC
 128 2010-04-02T22:59:58  *** valeuf has joined #moin-dev
 129 2010-04-02T23:10:13  *** grzywacz has quit IRC
 130 2010-04-02T23:14:37  *** JosefMeier has joined #moin-dev
 131 2010-04-02T23:41:26  <CIA-55> Thomas Waldmann <tw AT waldmann-edv DOT de> default * 5643:6282bd44c14c 1.9/MoinMoin/search/_tests/test_search.py: search tests: get rid of most crappy len() based tests
 132 2010-04-02T23:59:37  <CIA-55> Thomas Waldmann <tw AT waldmann-edv DOT de> default * 5644:3100da24a7fa 1.9/MoinMoin/search/_tests/test_search.py: search tests: some cosmetic fixes, improve comments/docstrings

MoinMoin: MoinMoinChat/Logs/moin-dev/2010-04-02 (last edited 2010-04-01 22:15:02 by IrcLogImporter)