MoinMoin: MoinMoinChat/Logs/moin-dev/2006-07-22

2006-07-22T02:14:37  <xorAxAx> ThomasWaldmann: 500 on http://sync.wikiwikiweb.de/
2006-07-22T02:37:01  <xorAxAx> is gone again
2006-07-22T02:37:26  <xorAxAx> nevertheless i would be interested in the error log lines
2006-07-22T02:37:55  <xorAxAx> hehe, the SyncPages action has got its first user visible feature: it shows you the list of new pages in the remote wiki
2006-07-22T11:02:15  <Kepplar> moin
2006-07-22T11:03:06  <ThomasWaldmann> xorAxAx: [Sat Jul 22 02:14:17 2006] [error] [client 89.245.73.51] Premature end of script headers: moin.cgi
2006-07-22T11:03:44  <xorAxAx> very verbose :)
2006-07-22T11:04:21  <ThomasWaldmann> as usual
2006-07-22T11:04:38  <ThomasWaldmann> moin btw
2006-07-22T11:05:04  <Kepplar> StorageEngineManager
2006-07-22T11:05:10  <Kepplar> we need it singlton-like
2006-07-22T11:05:15  <Kepplar> hmm
2006-07-22T11:05:42  <Kepplar> but if it were singleton only one thing can access storage (whereever) At a time
2006-07-22T11:05:50  <Kepplar> I'm playing with a Statelessproxy
2006-07-22T11:05:53  <Kepplar> (aka Borg)
2006-07-22T11:46:19  <Kepplar> hey
2006-07-22T11:46:29  <Kepplar> when you just import an module does its init auto fire
2006-07-22T11:46:36  <Kepplar> as if it was directly called?
2006-07-22T11:47:02  <Kepplar> (this question is like critical btw :) )
2006-07-22T11:56:48  <xorAxAx> you are missing the question word
2006-07-22T12:42:25  <Kepplar> please :)
2006-07-22T12:43:10  <xorAxAx> no, not that one
2006-07-22T12:43:16  <xorAxAx> please reword the question
2006-07-22T12:43:18  <Kepplar> ah
2006-07-22T12:43:22  <Kepplar> When we import a module
2006-07-22T12:43:28  <Kepplar> does its "init" get fired off?
2006-07-22T12:43:33  <Kepplar> as if your directly running the module
2006-07-22T12:43:40  <xorAxAx> do you mean the module global code?
2006-07-22T12:43:49  <xorAxAx> its just run once, of course
2006-07-22T12:44:12  <Kepplar> not global, but you've answered it anyway :)
2006-07-22T12:44:13  <Kepplar> cool
2006-07-22T12:44:28  <Kepplar> i just need a way for storage implimentations to "register" itself
2006-07-22T12:45:54  <xorAxAx> that would go into the global code of course
2006-07-22T12:46:13  <Kepplar> hmm?
2006-07-22T12:46:22  <xorAxAx> module level
2006-07-22T12:46:35  <Kepplar> well it has to register itself against the StorageEngineManager
2006-07-22T12:46:45  <Kepplar> so the SEM can create_engine
2006-07-22T12:47:09  <xorAxAx> the SEM is a global in another module
2006-07-22T12:47:20  <xorAxAx> then your storage implementation has this line:
2006-07-22T12:47:44  <xorAxAx> from MoinMoin.storage import manager; manager.register(FileBasedStorage)
2006-07-22T12:48:49  <Kepplar> does * work across multiple modules, so we can "import" everything from a directory (modules) and then iterated through whats imported and register?
2006-07-22T12:49:08  <Kepplar> -d
2006-07-22T12:49:17  <xorAxAx> Kepplar: see the plugin system
2006-07-22T12:49:23  <Kepplar> plugin.py?
2006-07-22T12:49:29  <xorAxAx> wikiutil.py
2006-07-22T12:49:51  <xorAxAx> or see the SystemInfo macro
2006-07-22T12:50:27  <Kepplar> ah
2006-07-22T12:50:28  <Kepplar> ok
2006-07-22T12:50:35  <Kepplar> so we need a data/plugin/storage
2006-07-22T12:50:49  <Kepplar> er
2006-07-22T12:50:50  <Kepplar> well
2006-07-22T12:50:52  <Kepplar> actually
2006-07-22T12:50:53  <Kepplar> ;/
2006-07-22T12:51:09  <xorAxAx> no, MoinMoin.storage in the normal case
2006-07-22T12:51:54  <Kepplar> isnt the fact plugins are stored on the filesystem like that a paradox to the fact that its stored using flatfile?
2006-07-22T12:52:09  <Kepplar> xorAxAx: ah
2006-07-22T13:00:20  <xorAxAx> checking the code base with python2.5 ... there are many deprecation warnings
2006-07-22T13:01:00  <Kepplar> really?
2006-07-22T13:01:07  <Kepplar> im using old version i admit
2006-07-22T13:01:15  <Kepplar> 2.3.5
2006-07-22T13:03:15  <xorAxAx> not your code :)
2006-07-22T13:03:52  <Kepplar> ohh :)
2006-07-22T13:04:53  <Kepplar> right
2006-07-22T13:04:57  <Kepplar> break for a couple of hours
2006-07-22T13:05:02  <Kepplar> work on my new site design :)
2006-07-22T13:17:56  <xorAxAx> ThomasWaldmann: while fixing 2.5 incompat bugs, i have found the issue that makes wikidicts crash on some systems
2006-07-22T13:18:19  <xorAxAx> ThomasWaldmann: dictbase et al. have an instance attrib called "regex". and this is pickled
2006-07-22T13:18:26  <xorAxAx> and this makes problems in some setups
2006-07-22T13:18:33  <xorAxAx> so i simply convert it to a class attrib
2006-07-22T13:20:58  <xorAxAx> yep, fixed it
2006-07-22T13:21:06  <xorAxAx> should speed up wikidicts a bit as well
2006-07-22T13:22:42  <xorAxAx> ThomasWaldmann: what exactly was broken in copy? did it just affect sre objects? then support.copy shouldnt be necessary anymore after my checkin
2006-07-22T13:33:35  * xorAxAx will have an exam (datenstrukturen und algorithmen) on monday at 9:00 - 12:00 local time and will post the lines today or tomorrow
2006-07-22T13:51:29  <ThomasWaldmann> xorAxAx: some people reported it fixed their problems
2006-07-22T13:51:42  <ThomasWaldmann> Kepplar: do you touch user.py currently?
2006-07-22T14:10:31  <Kepplar> ThomasWaldmann: not really
2006-07-22T14:21:20  <ThomasWaldmann> ok, then i will do some minor cleanup there, too
2006-07-22T14:30:29  <ThomasWaldmann> Kepplar: how about Page*.py?
2006-07-22T14:30:34  <Kepplar> er
2006-07-22T14:30:39  <Kepplar> significant comments
2006-07-22T14:30:56  <Kepplar> migration notes
2006-07-22T14:31:34  <ThomasWaldmann> ok, so i dont touch it now
2006-07-22T14:31:51  <Kepplar> (can always grab mine and sent it to main)
2006-07-22T14:33:21  <xorAxAx> i touched it :)
2006-07-22T14:33:26  <xorAxAx> severely :)
2006-07-22T16:01:18  <fpletz> I think I need some help with this case-sensitivity (and regexp for that matter) issue which would suggest chaining of search engine results. I was thinking about this all afternoon yesterday but every solution would in some way either be not elegant or not performing well (think of "c:bAr and not foo", would possibly yield many pages on both sides -> merging takes a long time)
2006-07-22T16:01:43  <fpletz> when reducing queries to term i.e. xapian can't handle we might run into even more problems.. merging the hits becomes impossible because information is lost
2006-07-22T16:01:58  <xorAxAx> fpletz: give an example for the latter problem
2006-07-22T16:02:20  <fpletz> we need multiple calls of the search engines, for each term once, which would be ineffective
2006-07-22T16:03:04  <fpletz> okay, if we take the query "c:bAr and not foo" and transform the query as suggested
2006-07-22T16:03:13  <fpletz> (easy and stupid now)
2006-07-22T16:03:31  <fpletz> xapian would search for not foo and moinSearch for c:bAr
2006-07-22T16:03:43  <xorAxAx> "c:bAr and not foo" --> "(x(bar) and d(c:bAr)) and not foo" --> x:"bar and not foo", d:"c:bAr" (whereas x==xapian and d==dumb search)
2006-07-22T16:03:49  <xorAxAx> so i dont see a problem there
2006-07-22T16:04:31  <xorAxAx> and d has to be called on the set returned by x
2006-07-22T16:04:37  <fpletz> okay, let's take a nested query
2006-07-22T16:05:06  <fpletz> (c:bAr or not foo) and (c:TeSt and bar)
2006-07-22T16:05:23  <fpletz> in this case we would need to call xapian and dumbsearch two times each
2006-07-22T16:05:38  <fpletz> I think this isn't very effective
2006-07-22T16:06:18  <fpletz> we can't just do: d(c:bAr and c:TeSt) and x(not foo and bar)
2006-07-22T16:06:35  <xorAxAx> you modified the query illegally
2006-07-22T16:06:44  <fpletz> yes ;)
2006-07-22T16:06:49  <fpletz> that's the point
2006-07-22T16:07:04  <fpletz> there's no effective way to transform this query efficiently
2006-07-22T16:07:30  <fpletz> just running _moinSearch on the whole thing would be way faster
2006-07-22T16:07:56  <fpletz> but case-search can be handled more easily with a trick
2006-07-22T16:08:00  <xorAxAx> "(c:bAr or not foo) and (c:TeSt and bar)" --> "((x(test) and d(c:TeSt)) and x(bar) and ((d(c:bAr) and x(bar)) or x(not foo))"
2006-07-22T16:08:11  <fpletz> that's not the problem, just this generel framework
2006-07-22T16:08:18  <fpletz> xorAxAx: right ;)
2006-07-22T16:08:28  <xorAxAx> and then it could transform the query
2006-07-22T16:08:57  <xorAxAx> we should ship a prolog interpreter
2006-07-22T16:09:12  <fpletz> yeah ;)
2006-07-22T16:09:20  <xorAxAx> then we dont have to worry
2006-07-22T16:09:52  <fpletz> well, if you know an algorithm or a method to transform this query, I would be happy to look at it and implement it.. but currently, I've got no real idea how to do this
2006-07-22T16:11:36  <fpletz> I think I should just implement case-sensitive searches with my trick (case-insentive xapian search -> postprecessing) and leave out the chaining completely
2006-07-22T16:12:24  <fpletz> ThomasWaldmann: around? ;)
2006-07-22T16:13:02  <xorAxAx> fpletz: now explain why that is faster than the "chaining"
2006-07-22T16:13:16  <xorAxAx> an and query does not traverse n pages 2*n times
2006-07-22T16:13:24  <xorAxAx> if you didnt break it :)
2006-07-22T16:13:31  <fpletz> it doesn't
2006-07-22T16:13:36  <fpletz> for example..
2006-07-22T16:13:49  <xorAxAx> and this "post processing" is done automatically by the search query engine
2006-07-22T16:13:59  <xorAxAx> but you need to call xapian multiple times, right
2006-07-22T16:14:07  <fpletz> nope ;)
2006-07-22T16:14:26  <xorAxAx> fpletz: and how would you handle regex searches then?=
2006-07-22T16:14:40  <fpletz> on a per-term basis (more later)
2006-07-22T16:15:51  <fpletz> our query "c:bAr and not foo" would instruct xapian to search for "bar and not foo".. we then get all hits in the relevant pages from xapian, all hits which match "bar" will be checked if they are equal to "bAr"
2006-07-22T16:15:55  <fpletz> this is pretty fast
2006-07-22T16:16:10  <fpletz> just one query, and only checking of found terms
2006-07-22T16:17:08  <fpletz> xorAxAx: do you understand what I mean?
2006-07-22T16:17:39  <xorAxAx> umm, yes
2006-07-22T16:17:59  <xorAxAx> that is equal to the following query (and should be implemented like that):
2006-07-22T16:18:29  <xorAxAx> "c:bAr and not foo" -> "x(bar and not foo) and d(c:bAr)"
2006-07-22T16:18:57  <xorAxAx> this will automatically call xapian once (x) and the and query will pass all hits to the normal text term
2006-07-22T16:19:06  <xorAxAx> fpletz: does that make senes?
2006-07-22T16:19:22  <xorAxAx> so how would this work, fpletz?  "(c:bAr or not foo) and (c:TeSt and bar)"
2006-07-22T16:19:40  <fpletz> yes, this also works.. but what about more complex queries?
2006-07-22T16:19:42  <fpletz> yes ;)
2006-07-22T16:19:47  <fpletz> hmm..
2006-07-22T16:19:56  <xorAxAx> fpletz: "also works" - thats exactly what you said
2006-07-22T16:20:23  <xorAxAx> if it is not equivalent to your idea, point to the difference
2006-07-22T16:20:24  <fpletz> its implementation would be a bit different
2006-07-22T16:20:33  <xorAxAx> the implementation is irrelevant :)
2006-07-22T16:20:48  <fpletz> ok ;)
2006-07-22T16:20:56  <fpletz> oh, wait
2006-07-22T16:20:58  <xorAxAx> lets think about evaluation order and terms, in the end you can implement it like you want
2006-07-22T16:21:00  <fpletz> just a second
2006-07-22T16:21:14  <xorAxAx> (similar problems will arise in every implementation)
2006-07-22T16:21:48  <fpletz> we could also do d(c:bAr and not foo) because it would just search in a limited set of pages
2006-07-22T16:21:59  <xorAxAx> yes
2006-07-22T16:22:05  <fpletz> this should be pretty fast and we don't need to transform large and complex queries
2006-07-22T16:22:23  <xorAxAx> now to the query you wrote above
2006-07-22T16:22:30  <xorAxAx> try to "explain" the evaluation order
2006-07-22T16:23:25  <fpletz> you mean "(c:bAr or not foo) and (c:TeSt and bar)"?
2006-07-22T16:23:31  <xorAxAx> yes
2006-07-22T16:24:09  <xorAxAx> next query: (not c:baR and bar)
2006-07-22T16:24:18  <fpletz> with your idea of implementing it, it would be a bit difficult.. I would just query xapian which terms are found in a document and check the query if it's case-sensitive
2006-07-22T16:24:24  <xorAxAx> fpletz: no, your idea
2006-07-22T16:24:32  <xorAxAx> just think about yours :)
2006-07-22T16:24:41  <fpletz> see the last sentence ;)
2006-07-22T16:25:06  <xorAxAx> please be more concrete
2006-07-22T16:25:22  <xorAxAx> how you would call xapian (which query) and how you filter
2006-07-22T16:25:58  <fpletz> okay
2006-07-22T16:26:18  <fpletz> xapian would get the query: "(bar or not foo) and (test and bar)"
2006-07-22T16:26:20  <fpletz> mmh
2006-07-22T16:26:24  <fpletz> oh, wait
2006-07-22T16:26:40  <fpletz> there are two bars
2006-07-22T16:26:40  <fpletz> damn..
2006-07-22T16:27:11  <fpletz> didn't think of that, my idea works not reliably in this case
2006-07-22T16:28:14  <fpletz> do you want the explanation anyway? or was this your point? ;)
2006-07-22T16:28:43  <xorAxAx> this was my "feeling" :)
2006-07-22T16:29:02  <fpletz> okay, thanks
2006-07-22T16:29:14  <fpletz> then straigt fallback to moin ;)
2006-07-22T16:29:31  <fpletz> aehm, moinSearch
2006-07-22T16:29:41  <xorAxAx> and assuming that writing a query optimiser is possible, it could be plugged in later on
2006-07-22T16:30:08  <fpletz> right, no problem at all
2006-07-22T16:30:36  <fpletz> perhaps I'll get some time to do more research on this
2006-07-22T16:30:44  <fpletz> or after soc ;)
2006-07-22T16:30:56  <xorAxAx> hehe
2006-07-22T16:31:26  <xorAxAx> i guess using sites like citeseer might make sense for such tasks :)
2006-07-22T16:32:13  <xorAxAx> umm#
2006-07-22T16:32:42  <fpletz> hehe, yeah
2006-07-22T18:38:15  <Kepplar> hola
2006-07-22T18:46:17  <mitsuhiko> xorAxAx: pingeling
2006-07-22T18:46:33  <xorAxAx> poing, bavaria
2006-07-22T18:46:41  <Ju> Hi all !
2006-07-22T18:46:48  <mitsuhiko> xorAxAx: you know my mokuwiki, don't you?
2006-07-22T18:46:57  <xorAxAx> some parts of it, yes
2006-07-22T18:46:58  <xorAxAx> hi Ju
2006-07-22T18:47:09  <mitsuhiko> jep. it has a very different syntax and so the GUI editor doesn't work
2006-07-22T18:47:23  <mitsuhiko> why is the gui editor hardcoded?
2006-07-22T18:47:30  <mitsuhiko> i havn't found a way to change the syntax of it
2006-07-22T18:47:46  <xorAxAx> mitsuhiko: thats a #moin question
2006-07-22T18:47:59  <mitsuhiko> not really. because if you want i can make it more flexible
2006-07-22T18:48:10  <xorAxAx> ok :)
2006-07-22T18:48:17  <xorAxAx> it is not hardcoded
2006-07-22T18:48:32  <xorAxAx> i mean, you would need to write a converter
2006-07-22T18:48:42  <mitsuhiko> and modify the javascript files
2006-07-22T18:48:44  <xorAxAx> then you can disable the shortcuts re hardcoding
2006-07-22T18:48:53  <xorAxAx> yes, the moin specific ones
2006-07-22T18:49:06  <xorAxAx> but the converter should be most difficult
2006-07-22T18:49:53  <mitsuhiko> hmm
2006-07-22T18:51:00  <mitsuhiko> basically the gui editor uses a special formatter to output simple to parse html and reverse that process after saving?
2006-07-22T18:51:26  <xorAxAx> yes. the formatter is even broken, so it depends on fckeditor  and the browser to convert it to xhtml
2006-07-22T18:51:28  <mitsuhiko> so all i need to implement is a html parser that outputs wiki syntax again
2006-07-22T18:51:40  <xorAxAx> mitsuhiko: yes, look into MoinMoin.converter
2006-07-22T18:55:08  <mitsuhiko> xorAxAx: i thought about something more flexible