2007-07-27T00:00:08 <zenhase> ok
2007-07-27T00:00:31 <grzywacz> Why not?
2007-07-27T00:00:50 <grzywacz> It's bloody template system, the very core of future MoinMoin, no?
2007-07-27T00:00:54 <grzywacz> *It's a
2007-07-27T00:00:54 <zenhase> grzywacz: hard dependency on something that doesn't even work like intended? :>
2007-07-27T00:01:06 <grzywacz> zenhase, is it impossible to make it work? :)
2007-07-27T00:01:49 <zenhase> grzywacz: right now it's a real pain to integrate
2007-07-27T00:02:49 <ThomasWaldmann> we are not talking about genshi integration into moin core, this has been abandoned
2007-07-27T00:03:07 <ThomasWaldmann> we are talking about a theme based genshi usage
2007-07-27T00:03:43 <zenhase> i will refactor back to the first prototype
2007-07-27T00:04:01 <ThomasWaldmann> so if we want to merge the genshi-theme branch into main branch, it must not crash when running it (or the tests) without genshi
2007-07-27T00:04:04 <zenhase> this should remove the deps
2007-07-27T00:05:14 <grzywacz> ah so...
2007-07-27T00:59:09 <ThomasWaldmann> xorAxAx: i bet you didnt test your mod_python "fix" :)
2007-07-27T01:07:01 <xorAxAx> right :)
2007-07-27T01:11:42 <ThomasWaldmann> but i guess after another fix it could work
2007-07-27T09:08:23 <lanius> moin
2007-07-27T09:08:39 <lanius> ThomasWaldmann: in the current code the delete pages appear as empty page with an own revision in the edit-log
2007-07-27T09:08:45 <lanius> ThomasWaldmann: is this a bevhaviour which should be kept
2007-07-27T10:24:27 <ThomasWaldmann> some code relies on it, so either keep or we have to fix those places
2007-07-27T10:24:44 <ThomasWaldmann> cleaner would be to raise some nonexistantrevision exception
2007-07-27T10:32:54 <xorAxAx> hmm, wikisync relies on it i think but i am not sure how often :)
2007-07-27T10:36:18 <lanius> what means it relies on it? what are the alternatives?
2007-07-27T10:37:16 <xorAxAx> well, wikisync assumes that revisions are immutable and that deleted is metadata of a revision
2007-07-27T10:40:09 <lanius> do you think it should be implemented this way?
2007-07-27T10:40:19 <xorAxAx> not sure :)
2007-07-27T10:45:10 <xorAxAx> if 1.7 would change in this behaviour, wikisync would have to support both semantics
2007-07-27T10:45:17 <xorAxAx> s/would change/changed/
2007-07-27T11:13:14 <lanius> ThomasWaldmann: what do you mean with nonexistantrevision exception?
2007-07-27T11:14:47 <ThomasWaldmann> that could be raise if you access some rev that is not there
2007-07-27T11:14:50 <ThomasWaldmann> +d
2007-07-27T11:23:31 <lanius> well another alternative is to not create a revision for deleted pages
2007-07-27T11:23:36 <lanius> so you have rev 1
2007-07-27T11:23:40 <lanius> item metadata = deleted
2007-07-27T11:23:47 <lanius> and if you create a new one you have rev 2
2007-07-27T11:23:51 <lanius> and deleted = false
2007-07-27T11:42:49 <CIA-27> moin: Johannes Berg <johannes AT sipsolutions DOT net> * 2574:7dd5eab0f701 1.7/MoinMoin/request/__init__.py: make generated IDs nicer, qualify iff necessary
2007-07-27T11:42:52 <CIA-27> moin: Johannes Berg <johannes AT sipsolutions DOT net> * 2575:d1ec881c6f86 1.7/MoinMoin/macro/Include.py: rewrite include macro heading link
2007-07-27T11:42:57 <CIA-27> moin: Johannes Berg <johannes AT sipsolutions DOT net> * 2576:f4c371b00fec 1.7/MoinMoin/ (6 files in 5 dirs): refactor the ID generation/writing code
2007-07-27T11:43:03 <CIA-27> moin: Johannes Berg <johannes AT sipsolutions DOT net> * 2577:32bba35902b9 1.7/MoinMoin/formatter/text_python.py: refactor some code in the text_python formatter
2007-07-27T11:43:11 <CIA-27> moin: Johannes Berg <johannes AT sipsolutions DOT net> * 2578:1e5ac92670e6 1.7/MoinMoin/macro/TableOfContents.py: fix TOC html
2007-07-27T11:43:14 <CIA-27> moin: Johannes Berg <johannes AT sipsolutions DOT net> * 2579:9f933413518b 1.7/MoinMoin/macro/ (Action.py _tests/test_Action.py): rewrite action macro. this breaks some pages that'll need to be fixed
2007-07-27T12:14:46 <lanius> ThomasWaldmann:
2007-07-27T12:44:34 <ThomasWaldmann> lanius: if you want to do a diff for a delete op, the current behavior is to diff revN, revN+1
2007-07-27T12:44:48 <ThomasWaldmann> if N+1 is deleted, moin fakes "" as content
2007-07-27T12:44:54 <ThomasWaldmann> so the diff you get is correct
2007-07-27T12:45:17 <ThomasWaldmann> of course we can simulate this behaviour by changing the code...
2007-07-27T12:46:36 <xorAxAx> and wikisync would need to be changed as well
2007-07-27T12:46:56 <xorAxAx> lanius: so whereis the disadvantage of moving the flag to the rev metadata?
2007-07-27T12:47:10 <xorAxAx> lanius: it has the advantage that you dnt need to change the items metadata
2007-07-27T12:47:33 <xorAxAx> which would need more complex kinds of locking etc.
2007-07-27T12:47:36 * johill dug into the cruft that is Page.link_to
2007-07-27T12:47:41 <johill> can we kill that? :)
2007-07-27T12:49:11 <xorAxAx> you mean the 10 different code paths that result in recursive formatter/wikiutil calls starting from linkto? :)
2007-07-27T12:52:45 <johill> yeah
2007-07-27T12:52:48 <johill> all that crap
2007-07-27T12:52:56 <johill> why doesn't it just use page.url() and the formatter?
2007-07-27T12:53:03 <johill> themes would obviously create their own <a> tags
2007-07-27T12:53:25 <johill> I'm tempted to put a deprecation warning into page.link_to
2007-07-27T12:55:12 <xorAxAx> why?
2007-07-27T12:55:29 <johill> to see where else it's called indirectly
2007-07-27T12:55:32 <johill> http://johannes.sipsolutions.net/patches/moin/formatter-linkto.patch
2007-07-27T12:55:59 <xorAxAx> i think most of the semantics can be implemented without that weird code paths
2007-07-27T12:56:35 <xorAxAx> a deprecationwarning only fires once, johill
2007-07-27T12:56:42 <johill> for every caller
2007-07-27T12:56:51 <xorAxAx> are you sure?
2007-07-27T12:56:51 <johill> (if you use the level parameter correctly)
2007-07-27T12:57:45 <johill> try it
2007-07-27T12:57:46 <johill> http://johannes.sipsolutions.net/patches/moin/deprecate-linkto.patch
2007-07-27T12:58:19 <xorAxAx> hmm
2007-07-27T12:58:50 <johill> the big mess in link_to comes from the fact that it allows so many keyword parameters that are pretty bogus to start with
2007-07-27T12:59:03 <johill> but oh well
2007-07-27T12:59:10 <johill> it sort of works ;)
2007-07-27T12:59:26 <xorAxAx> yes, thats the moin paradoxon
2007-07-27T12:59:36 <johill> and as long as nobody draws nice diagrams of how it should work it can't be fixed either
2007-07-27T12:59:43 <xorAxAx> hehe
2007-07-27T12:59:49 <johill> everything's far too entangled
2007-07-27T13:00:24 <johill> or for example if you cache a page
2007-07-27T13:00:29 <johill> it's first formatted into the cache and then executed
2007-07-27T13:00:45 <johill> only by pure luck does that work since everybody modifies the request's state and it's never reset
2007-07-27T13:01:33 <johill> maybe it's time to scrap it and start from scratch
2007-07-27T13:06:11 <johill> anyway, lucnh
2007-07-27T13:06:13 <johill> lunch even
2007-07-27T13:10:29 <xorAxAx> johill: hmm, what exactly do you mean?
2007-07-27T13:10:50 <xorAxAx> i.e. which state
2007-07-27T13:29:24 <johill> well
2007-07-27T13:29:47 <johill> I think that needs a bit of a braindump first ;)
2007-07-27T13:30:05 <xorAxAx> there isnt that much ...
2007-07-27T13:30:09 <johill> I noticed that the cacher inserts things like set language
2007-07-27T13:31:06 <johill> and also that each formatter keeps internal state
2007-07-27T13:31:30 <johill> (and something I don't understand, the wiki parser users formatter state like in_p)
2007-07-27T13:33:34 <johill> so ideally the cacher would just call the formatter's call
2007-07-27T13:33:59 <johill> track what formatter state it modifies and which global state it uses
2007-07-27T13:34:06 <johill> and then insert either the output or the call
2007-07-27T13:34:10 <johill> and send the output to the client
2007-07-27T13:34:20 <johill> (so we dno't need to execute the cache after making it)
2007-07-27T13:34:34 <xorAxAx> what do you mean by "track"?
2007-07-27T13:34:43 <johill> well catch if it writes state
2007-07-27T13:34:49 <xorAxAx> how?
2007-07-27T13:35:10 <xorAxAx> thats not decidable without introducing some API
2007-07-27T13:35:11 <johill> well if formatters wouldn't use formatter.some_state
2007-07-27T13:35:18 <johill> but rather formatter.state.something
2007-07-27T13:35:22 <johill> it could track the state object
2007-07-27T13:35:23 <johill> oh sure
2007-07-27T13:35:26 <johill> I'm dreaming here ;)
2007-07-27T13:35:45 <xorAxAx> even if it used the state, it should be cached as a string sometimes
2007-07-27T13:35:56 <johill> right
2007-07-27T13:36:00 <johill> using state is fine
2007-07-27T13:36:09 <xorAxAx> not always :)
2007-07-27T13:36:12 <johill> updating state means we need to add a state update into the output
2007-07-27T13:36:21 <johill> yes, using formatter state is always ok
2007-07-27T13:36:25 <johill> using global state isn't
2007-07-27T13:36:27 <xorAxAx> ok
2007-07-27T13:36:49 <xorAxAx> generally, the stuff would be much easier if it was designed functionally :)
2007-07-27T13:36:59 <johill> what do you mean by functionally?
2007-07-27T13:37:13 <xorAxAx> returning values (a tree of formatter elements etc.)
2007-07-27T13:37:33 <xorAxAx> instead of modifying state in another component (by calling formatter methods)
2007-07-27T13:37:48 <johill> oh but that isn't always possible
2007-07-27T13:37:50 <johill> is it?
2007-07-27T13:37:56 <johill> think pagelink()
2007-07-27T13:37:59 <xorAxAx> it would be a totally different API
2007-07-27T13:38:01 <xorAxAx> hmm?
2007-07-27T13:38:07 <johill> if you have show question marks it closes the link right away
2007-07-27T13:38:18 <xorAxAx> so?
2007-07-27T13:38:31 <xorAxAx> the parser returns a Pagelink object
2007-07-27T13:38:51 <johill> and then?
2007-07-27T13:38:56 <xorAxAx> and the tree transformations generate HTML from it
2007-07-27T13:39:11 <johill> what would the pagelink object contain?
2007-07-27T13:39:24 <xorAxAx> page name and link text
2007-07-27T13:39:28 <johill> ok
2007-07-27T13:39:37 <johill> still
2007-07-27T13:39:39 <johill> what about a heading
2007-07-27T13:39:41 <xorAxAx> link text being not linked if the ?-mode was enabled
2007-07-27T13:39:51 <xorAxAx> Heading(heading_title)
2007-07-27T13:39:56 <xorAxAx> +depth
2007-07-27T13:40:00 <johill> in text/html you get <h2>jheading_title</h2>
2007-07-27T13:40:03 <johill> but in text/xml you get
2007-07-27T13:40:20 <johill> <s2 title=heading_title> actual contents until next heading </s2>
2007-07-27T13:40:30 <xorAxAx> yes, thats no problem :)
2007-07-27T13:40:40 <xorAxAx> because the formatters would need to continue to track that state
2007-07-27T13:40:49 <xorAxAx> but common transformations could help to factor out that
2007-07-27T13:41:09 <xorAxAx> so you could have a transformation that wraps all blocks into a block with a known heading attribute
2007-07-27T13:41:22 <xorAxAx> and the XML formatter wouldnt have to worry about it anymore
2007-07-27T13:41:40 <johill> not sure I understand the transformation thing
2007-07-27T13:42:09 <xorAxAx> well, imagine a tree: document(heading("foo"), paragraph("foo"), paragraph("foo"))
2007-07-27T13:42:19 <johill> right
2007-07-27T13:42:26 <xorAxAx> well, imagine a tree: document(heading("foo"), paragraph("foo"), paragraph("foo"), heading("bar"), paragraph("barfoo"))
2007-07-27T13:42:44 <xorAxAx> now the xml formatter may optionally run a transformation that changes the tree into:
2007-07-27T13:43:22 <xorAxAx> document(division("foo", paragraph("foo"), paragraph("foo")), division("bar", paragraph("barfoo")))
2007-07-27T13:43:34 <xorAxAx> and thats much easier to output by the xml formatter than the old tree
2007-07-27T13:43:54 <johill> oh right
2007-07-27T13:43:56 <xorAxAx> it would reduce the complexity of the xml parser and in every other parser that wants to have such semantics
2007-07-27T13:44:19 <xorAxAx> s/parser/formatter/g
2007-07-27T13:44:54 <johill> right
2007-07-27T13:45:32 <johill> of course you'd need a much less crappy parser for that
2007-07-27T13:45:40 <johill> <p id=line866> :)
2007-07-27T13:45:51 <xorAxAx> theoretically, it is possible to impleemnt a new rendering system and support the old system by introducing a legacy formatter that only outputs legacy objects
2007-07-27T13:46:12 <xorAxAx> and request.write would spill the objects into some request-wide list
2007-07-27T13:46:29 <xorAxAx> then you can generate a tree from it and feed it into the new system :)
2007-07-27T13:46:37 <johill> no, you can't generate a tree from it
2007-07-27T13:46:41 <xorAxAx> why not?
2007-07-27T13:46:45 <johill> the current parser doesn't create well-formed nestings
2007-07-27T13:46:55 <johill> it sometimes forgets to close a few paragraphs
2007-07-27T13:46:57 <xorAxAx> well, thats a different problem
2007-07-27T13:47:16 <xorAxAx> but you can theoretically shift the whole "fix the damn nesting" logic into the un-flattener
2007-07-27T13:47:17 <johill> I guess you can still fix that up though
2007-07-27T13:48:09 <johill> if you generated objects to start with that wouldn't be a problem
2007-07-27T13:48:29 <johill> you'd have a stack of objects that you're currently appending to
2007-07-27T13:48:45 <xorAxAx> yes, theoretically, you can let all formatters generate objects and everything should work currently
2007-07-27T13:48:47 <johill> and when a new paragraph comes along you drop off the current paragraph and push a new one
2007-07-27T13:48:53 <xorAxAx> they only have to implement __add__
2007-07-27T13:49:01 <johill> nah, ''.join()?
2007-07-27T13:49:05 <xorAxAx> calls add
2007-07-27T13:49:10 <johill> oh ok
2007-07-27T13:49:10 <xorAxAx> hmm, no
2007-07-27T13:49:13 <xorAxAx> that generates a string
2007-07-27T13:49:15 <xorAxAx> indeed
2007-07-27T13:49:19 <xorAxAx> hmm
2007-07-27T13:49:43 <xorAxAx> one issue is that join() doesnt call str()
2007-07-27T13:50:03 <xorAxAx> otherwise one could serialise the compound object and deserialise it on the next occurence in __add__
2007-07-27T13:50:04 <johill> it expects things to be a string to start with
2007-07-27T13:50:07 <xorAxAx> yes
2007-07-27T13:50:19 <johill> I suppose you could always return a serialised object
2007-07-27T13:50:31 <johill> and deserialise it all once you're through
2007-07-27T13:50:53 <johill> ah no
2007-07-27T13:51:04 <johill> no should work
2007-07-27T13:51:12 <johill> you just have to make a few passes over it at the end
2007-07-27T13:51:59 <johill> however
2007-07-27T13:52:05 <johill> that's still just papering over the problem :)
2007-07-27T13:52:49 <xorAxAx> that offers the possibility to generate something more advanced and build a rendering pipeline around that
2007-07-27T13:53:02 <xorAxAx> while reusing old macros and parsers
2007-07-27T13:53:21 <johill> yeah but it'll need a whole bunch of tricks with request
2007-07-27T13:53:32 <johill> the formatter parser thing isn't the biggest problem
2007-07-27T13:53:51 <johill> all the nested calling, formatter re-instantiation etc is much worse I think
2007-07-27T13:54:38 <xorAxAx> hmm, yeah
2007-07-27T13:54:52 <johill> and everybody abuses request for state etc
2007-07-27T13:55:26 <johill> I seriously think that the concept is beyond repair
2007-07-27T13:56:10 <xorAxAx> well, the reinstantiation is not IMHO
2007-07-27T13:56:23 <xorAxAx> the state tracking is maybe but it could be fixed somehow
2007-07-27T13:56:40 <johill> yeah but there's no way to understand it
2007-07-27T13:56:54 <xorAxAx> the instantiation?
2007-07-27T13:57:02 <johill> the whole code path
2007-07-27T13:57:10 <johill> like formatter.page vs. request.page
2007-07-27T13:57:31 <xorAxAx> well, request.page doesnt have any reason to be there IMHO
2007-07-27T13:57:48 <johill> and request.formatter vs. page.formatter
2007-07-27T13:57:59 <xorAxAx> page.formatter sounds weird :)
2007-07-27T13:58:01 <ThomasWaldmann> if you remove request.page, you'll have to fix formatter.page
2007-07-27T13:58:08 <johill> what's wrong with it?
2007-07-27T13:58:21 <xorAxAx> johill: one idea is to rip it out and see what breaks
2007-07-27T13:58:33 <xorAxAx> writing unit tests beforehand simplifies this task
2007-07-27T13:58:36 <ThomasWaldmann> the usual reason to use it is to get the name of the page that is currently process (see url)
2007-07-27T13:58:44 <johill> the thing is it's impossible to write unittests
2007-07-27T13:58:45 <xorAxAx> or writing unit tests when interactive breakage is encountered
2007-07-27T13:58:48 <xorAxAx> johill: why?
2007-07-27T13:59:00 <ThomasWaldmann> but it get's changed by Insert/Include stuff, so the next Insert/Include is borked
2007-07-27T13:59:00 <xorAxAx> johill: how about that xpath approach i presented?
2007-07-27T13:59:05 <johill> because there's no knowing how it should work
2007-07-27T13:59:10 <xorAxAx> johill: thats a good point
2007-07-27T13:59:21 <johill> ThomasWaldmann: but why does it even use the same formatter when you include?
2007-07-27T13:59:32 <johill> actually, include instantiates a new formatter
2007-07-27T13:59:36 <johill> so what's the problem?
2007-07-27T13:59:47 <ThomasWaldmann> it could be either simply broken or not working the other way, no idea
2007-07-27T14:00:09 <johill> xorAxAx: tell me how to take a wiki html string and emit html
2007-07-27T14:00:25 <johill> xorAxAx: there are probably half a dozen ways depending on where you come from
2007-07-27T14:01:30 <xorAxAx> johill: do you want to approach blackbox testing? then using the request api would make sense
2007-07-27T14:01:47 <xorAxAx> johill: there is also paste fixture, which automatically does the whol request handling for you
2007-07-27T14:02:00 <johill> that's beside the point
2007-07-27T14:02:04 <xorAxAx> and you can say body = do_request("FrontPage"); assert body.contains("foo")
2007-07-27T14:02:11 <xorAxAx> johill: you want to test the API?
2007-07-27T14:02:28 <johill> what I'm trying to say is that there is no proper API
2007-07-27T14:02:37 <johill> unless you want to go through request maybe
2007-07-27T14:02:40 <xorAxAx> then you would use send_page with r.w capturing :)
2007-07-27T14:02:52 <johill> I changed it to do that because things broke ;)
2007-07-27T14:03:02 <xorAxAx> hmm?
2007-07-27T14:03:06 <xorAxAx> what broke?
2007-07-27T14:03:21 <johill> some test broke because it was doing tricks like parser.format(...)
2007-07-27T14:03:40 <xorAxAx> and what did you change back?
2007-07-27T14:03:53 <johill> I made it use send_page so that formatter.page is assigned
2007-07-27T14:05:06 <xorAxAx> IMHO the parser should know the formatter
2007-07-27T14:05:26 <xorAxAx> johill: what used send_page?
2007-07-27T14:05:36 <johill> no the test *didn't* use send_page
2007-07-27T14:05:42 <johill> and so the various links between objects weren't set up
2007-07-27T14:06:49 <xorAxAx> which test?
2007-07-27T14:07:01 <johill> the parser tests I think
2007-07-27T14:07:21 <xorAxAx> ah
2007-07-27T14:07:51 <johill> yeah, those
2007-07-27T14:10:46 <johill> actually
2007-07-27T14:11:04 <johill> the fact that the include macro needs to create a new formatter is another instance of the state thing
2007-07-27T14:11:44 <johill> maybe I should try writing a Page.get_parse_tree() function
2007-07-27T14:11:53 <xorAxAx> what would it do?
2007-07-27T14:11:55 <johill> that formats it using a serialising formatter and then deserialises
2007-07-27T14:12:02 <johill> (and fixes the tree too)
2007-07-27T14:12:22 <xorAxAx> using serialised objects like described above?
2007-07-27T14:12:25 <johill> yeah
2007-07-27T14:12:30 <xorAxAx> cool :)
2007-07-27T14:12:39 <johill> then we could rip out all existing formatter code in all other places
2007-07-27T14:12:46 <johill> and replace it by calls to format that dom tree
2007-07-27T14:12:57 <xorAxAx> theoretically, you only need a new formatter that generates those serialised strings
2007-07-27T14:13:01 <johill> yeah
2007-07-27T14:13:07 <xorAxAx> and some deserialising logic and the tree ops
2007-07-27T14:13:09 <johill> easy enough, I'd think
2007-07-27T14:13:29 <johill> I wonder if you can concatenate pickled objects
2007-07-27T14:13:51 <xorAxAx> hmm, i dont think so
2007-07-27T14:13:54 <xorAxAx> but
2007-07-27T14:13:58 <xorAxAx> hmm, you can
2007-07-27T14:14:09 <xorAxAx> pickle has a . operator
2007-07-27T14:14:16 <johill> hm?
2007-07-27T14:14:19 <xorAxAx> if that stops the pickling process, you can just call load() again
2007-07-27T14:14:30 <xorAxAx> and the next pickle loading pass would start from there again
2007-07-27T14:14:43 <xorAxAx> . pops the last object from the stack and stops parsing AFAIK
2007-07-27T14:14:59 <xorAxAx> i guess you have to try it (you will need a stringio object - a string wont work)
2007-07-27T14:15:04 <johill> yeah
2007-07-27T14:15:47 <johill> no it doesn't work
2007-07-27T14:15:51 <johill> oh wait it does
2007-07-27T14:16:14 <johill> guess that'll find all the places that do request.write w/o a formatter call :P
2007-07-27T14:16:35 <johill> so that's easy really
2007-07-27T14:17:17 <johill> class BaseElement:
2007-07-27T14:17:22 <johill> def __str__(self):
2007-07-27T14:17:27 <johill> s = StringIO()
2007-07-27T14:17:35 <johill> pickle.dump(self, s)
2007-07-27T14:17:39 <johill> return s.getvalue()
2007-07-27T14:17:49 <xorAxAx> i meant to say that unpickling needs stringio. pickling doenst
2007-07-27T14:18:35 <johill> oh right
2007-07-27T14:19:01 <johill> class BaseElement:
2007-07-27T14:19:08 <xorAxAx> did you test if you can unpickle two concatted pickling strings?
2007-07-27T14:19:11 <johill> yes
2007-07-27T14:19:21 <johill> that works fine
2007-07-27T14:19:31 <johill> you just have to unpickle as many times as you pickled
2007-07-27T14:20:55 <xorAxAx> unpickle as long as it doesnt raise an eofexception :)
2007-07-27T14:21:05 <johill> something like that
2007-07-27T14:21:24 <johill> EOFError :)
2007-07-27T14:21:30 <xorAxAx> yes
2007-07-27T14:21:51 <johill> and then you have the list
2007-07-27T14:21:57 <johill> and need to reformat it into a tree
2007-07-27T14:25:14 <johill> hm
2007-07-27T14:31:20 <ThomasWaldmann> zenhase: any recfactoring news?
2007-07-27T14:36:54 <johill> it works
2007-07-27T14:37:59 <johill> xorAxAx: http://johannes.sipsolutions.net/files/pickle.py and test_pickle.py
2007-07-27T14:39:49 <xorAxAx> what is other_attributes?
2007-07-27T14:40:13 <johill> not sure
2007-07-27T14:40:22 <johill> I thought things freely set stuff in the formatter
2007-07-27T14:40:38 <xorAxAx> but currently you cannot add anything to that dict
2007-07-27T14:40:48 <johill> oh right
2007-07-27T14:40:53 <johill> yeah doesn't matter anyway I think
2007-07-27T14:41:08 <johill> I replaced it with an attribute error now
2007-07-27T14:41:34 <xorAxAx> ok
2007-07-27T14:42:03 <xorAxAx> does the parser use all those formatter attributes, johill?
2007-07-27T14:42:11 <johill> in_p etc? yeah
2007-07-27T14:42:20 <xorAxAx> a piy
2007-07-27T14:42:21 <xorAxAx> +t
2007-07-27T14:42:32 <johill> yeah I'd almost say crap ;)
2007-07-27T14:42:37 <johill> but I just found a problem
2007-07-27T14:42:39 <johill> I need to call macros
2007-07-27T14:42:44 <johill> and subparsers
2007-07-27T14:43:08 <johill> no wait, I just need insert a macro call tag
2007-07-27T14:43:15 <xorAxAx> try to render SyntaxReference :)
2007-07-27T14:44:47 <johill> just doing that
2007-07-27T14:45:33 <johill> works now
2007-07-27T14:45:39 <johill> uploaded new code
2007-07-27T14:48:08 <johill> so, what now?
2007-07-27T14:48:27 <johill> let's assume for a moment that macros always generate valid nestings
2007-07-27T14:48:36 <johill> or rather
2007-07-27T14:48:44 <johill> that there isn't one macro to start something and another to end it again
2007-07-27T14:49:46 <xorAxAx> you would need kind of barriers
2007-07-27T14:50:04 <xorAxAx> that are inserted before every parser call
2007-07-27T14:50:12 <xorAxAx> but note that macros are sometimes in the paragraph
2007-07-27T14:50:27 <xorAxAx> so you cannot assume that you want to have the paragraph closed beforehand
2007-07-27T14:51:10 <johill> hm
2007-07-27T14:51:58 <johill> that thing opens far too many paragraphs
2007-07-27T14:53:20 <johill> well I guess the question is what we want to use it for
2007-07-27T14:54:25 <xorAxAx> johill: what?
2007-07-27T14:55:01 <johill> the parse tree
2007-07-27T14:55:23 <xorAxAx> the idea is to do transformations on it and to feed it to the formatters
2007-07-27T14:55:38 <johill> right
2007-07-27T14:55:39 <xorAxAx> e.g. you can have a toc transformation :)
2007-07-27T14:56:05 <johill> but we really have multiple stages
2007-07-27T14:56:12 <johill> in the first stage we embed parser and macro calls
2007-07-27T14:56:20 <xorAxAx> that replaces all TOC() objects by tocs that show the items in the current node or any other node beow
2007-07-27T14:56:24 <xorAxAx> +l
2007-07-27T14:56:55 <johill> in the second stage we take the tree, call the macros and replace the macro objects by the result
2007-07-27T14:57:08 <johill> for that we can use the formatter again in its first stage
2007-07-27T14:57:30 <xorAxAx> hmm?
2007-07-27T14:57:39 <xorAxAx> what do you mean by the last line?
2007-07-27T14:58:19 <johill> well
2007-07-27T14:58:25 <johill> look at the output of my test for HelpOnParsers
2007-07-27T14:58:31 <johill> it contains e.g.
2007-07-27T14:58:31 <johill> ('parser', (u'pascal', [u'#!pascal ', u'function TRegEx.Match(const s:string):boolean;', u'var', u' l,i : integer;', u'begin', u' result := MatchPos(s,l,i);', u'end;']), {})
2007-07-27T14:58:45 <xorAxAx> yes
2007-07-27T14:58:57 <johill> now we take this output and make a valid tree out of it
2007-07-27T14:59:07 <johill> that we could even cache
2007-07-27T14:59:13 <johill> (not much use though)
2007-07-27T14:59:28 <xorAxAx> you can have a tranformation that does an activation of all cachable plugins
2007-07-27T14:59:34 <johill> right
2007-07-27T14:59:50 <xorAxAx> but the idea is that those only should use the old formatter interface if they are legacy
2007-07-27T15:00:01 <johill> yeah but that means that the activation of them
2007-07-27T15:00:02 <xorAxAx> otherwise they can generate their tree directly
2007-07-27T15:00:06 <johill> is done by the pickle formatter again
2007-07-27T15:00:12 <xorAxAx> why?
2007-07-27T15:00:22 <johill> if they're legacy
2007-07-27T15:00:47 <johill> eh not by
2007-07-27T15:00:49 <johill> with
2007-07-27T15:00:54 <xorAxAx> ah
2007-07-27T15:00:57 <xorAxAx> umm, sure
2007-07-27T15:01:08 <johill> so we get another tree, make it valid, and insert it instead of the macro node
2007-07-27T15:01:14 <xorAxAx> yes
2007-07-27T15:01:19 <johill> or parser
2007-07-27T15:02:11 <johill> so what do we need?
2007-07-27T15:02:42 <johill> oh
2007-07-27T15:03:00 <johill> and caching is an easy thing too, we just have a transformation that makes to html what can be
2007-07-27T15:04:14 <johill> anyway
2007-07-27T15:05:07 <johill> so how do we make a valid tree out of this?
2007-07-27T15:05:45 * ThomasWaldmann .oO(sound like 1.7 will be rather 2.0 :D )
2007-07-27T15:06:14 <xorAxAx> maybe it will be released before 1.6, who knows
2007-07-27T15:06:30 <xorAxAx> johill: well, the first question is - whats the target format :)
2007-07-27T15:06:47 <johill> well let's say something well-formed ;)
2007-07-27T15:07:01 <johill> I don't think we should fix the target format
2007-07-27T15:07:15 <johill> but we discussed that above, most things are possible with a transformation (remember headings)
2007-07-27T15:07:29 <xorAxAx> yeah, maybe it makes sense to target the old formatter interface first
2007-07-27T15:07:40 <johill> hm?
2007-07-27T15:07:50 <xorAxAx> i mean, the tree needs a meta-moel
2007-07-27T15:07:51 <xorAxAx> +d
2007-07-27T15:07:58 <xorAxAx> what nodes are there etc.
2007-07-27T15:08:00 <johill> that's trivial, just call all the methods in the order we pickled them
2007-07-27T15:08:01 <johill> right
2007-07-27T15:08:06 <johill> but what sort of model do we use?
2007-07-27T15:08:29 <xorAxAx> there is a moinmoin branch which uses the docutils DOM :)
2007-07-27T15:08:46 <xorAxAx> umm, which planned to do so i mean
2007-07-27T15:08:47 <johill> that works?
2007-07-27T15:08:49 <xorAxAx> no
2007-07-27T15:08:56 <xorAxAx> i mean, it was never really started
2007-07-27T15:09:02 <johill> so. I really don't care about things that don't work ;)
2007-07-27T15:09:17 <xorAxAx> well, i havent had a look at their dom
2007-07-27T15:09:35 <xorAxAx> maybe it makes sense to have one that is based on the old formatter semantics
2007-07-27T15:09:37 <johill> hmmm
2007-07-27T15:09:37 <johill> johannes:~/Projects/moin/moin-1.7$ py.test MoinMoin/formatter/_tests/test_pickle.py | grep "'line_anchordef', (7" | wc -l
2007-07-27T15:09:40 <xorAxAx> and that would be amended
2007-07-27T15:09:40 <johill> 142
2007-07-27T15:09:53 <johill> there is no dom on the old semantics
2007-07-27T15:10:03 <johill> you have to rely on all the code doing the right thing
2007-07-27T15:10:16 <xorAxAx> yes, but every method maps to a node type
2007-07-27T15:10:21 <johill> right
2007-07-27T15:10:45 <johill> I'm wondering how to represent this anywya
2007-07-27T15:10:52 <johill> I mean, the dom
2007-07-27T15:10:56 <johill> not the tree later
2007-07-27T15:11:00 <xorAxAx> see the minidom that is shipped with python
2007-07-27T15:11:05 <johill> [how can it output line 7 142 times?]
2007-07-27T15:11:08 <xorAxAx> for some minimal dom-like interface
2007-07-27T15:11:18 <xorAxAx> johill: thats a question for thomas :)
2007-07-27T15:11:43 <johill> xorAxAx: no minidom is just an access to a tree too
2007-07-27T15:11:48 <johill> it doesn't define the model
2007-07-27T15:11:55 <xorAxAx> ah, you still mean the meta-model
2007-07-27T15:12:08 <johill> yes
2007-07-27T15:12:12 <johill> I'd like to be able to declare it somehow
2007-07-27T15:12:13 * xorAxAx checks the docutils
2007-07-27T15:12:25 <xorAxAx> well, using a class hierachy
2007-07-27T15:12:47 <xorAxAx> or do you mean something DTD like? :)
2007-07-27T15:12:55 <johill> yes
2007-07-27T15:13:02 <xorAxAx> ah
2007-07-27T15:13:21 <johill> something that answers the question
2007-07-27T15:13:26 <johill> 'can I open a paragraph now'
2007-07-27T15:13:44 <johill> (well if not we're fucked anyway)
2007-07-27T15:14:20 <johill> I guess it's a bit lik
2007-07-27T15:14:35 <johill> if there's a heading of level N now then close everything to go back to the same level the last heading of that level was on
2007-07-27T15:14:38 <johill> or something like that
2007-07-27T15:14:49 <johill> actually, all headings hsould probably be at the top level of paragraphs
2007-07-27T15:14:59 <xorAxAx> yes, since 1.5 thats a requirement
2007-07-27T15:15:22 <johill> 1.5 of what?
2007-07-27T15:15:23 <johill> moin?
2007-07-27T15:15:24 <johill> hah
2007-07-27T15:15:25 <xorAxAx> yes
2007-07-27T15:15:28 <xorAxAx> or 1.3.x
2007-07-27T15:15:32 <xorAxAx> dont remember exactly
2007-07-27T15:15:47 <johill> doesn't seem to be working though
2007-07-27T15:16:00 <xorAxAx> possible
2007-07-27T15:16:20 <xorAxAx> i mean from the semantical point of view - not the generated html
2007-07-27T15:17:48 <xorAxAx> well, you can encode this somehow in the code of the transformation
2007-07-27T15:17:52 <johill> yeah
2007-07-27T15:17:56 <johill> let me try something
2007-07-27T15:18:01 <xorAxAx> you must just be sure about the rule itself :)
2007-07-27T15:24:36 <johill> hah
2007-07-27T15:24:38 <johill> emphasis (True,) {}
2007-07-27T15:24:38 <johill> paragraph (1,) {'css_class': 'line862'}
2007-07-27T15:24:38 <johill> text (u'wiki',) {}
2007-07-27T15:24:38 <johill> paragraph (1,) {'css_class': 'line867'}
2007-07-27T15:24:40 <johill> paragraph (1,) {'css_class': 'line891'}
2007-07-27T15:24:42 <johill> emphasis (False,) {}
2007-07-27T15:24:45 <johill> expected to close paragraph now, not emphasis
2007-07-27T15:26:56 <ThomasWaldmann> xorAxAx: i doubt that, because it has more open todo
2007-07-27T15:27:06 <johill> wow
2007-07-27T15:27:07 <xorAxAx> ThomasWaldmann: hmm, which ones?
2007-07-27T15:27:14 <johill> the thing opens 76 paragraphs too much
2007-07-27T15:27:19 <xorAxAx> johill: :)
2007-07-27T15:27:24 <xorAxAx> you mean the parser?
2007-07-27T15:28:00 <ThomasWaldmann> xorAxAx: to get 1.7(2.0 :) released has more todo (converter+x) than 1.6 (converter).
2007-07-27T15:29:31 <xorAxAx> if we merge the storage branch, yes
2007-07-27T15:29:43 <xorAxAx> but the 1.6 converter is finished, isnt it, johill?
2007-07-27T15:30:52 <ThomasWaldmann> its finished when it is integrated into the mig framework and has received quite some testing (and is rather complete).
2007-07-27T15:31:31 <johill> yeah
2007-07-27T15:31:37 <johill> it's working
2007-07-27T15:31:42 <johill> well actually
2007-07-27T15:31:45 <johill> I changed the action macro
2007-07-27T15:31:50 <johill> so maybe we should incorporate that
2007-07-27T15:40:00 <johill> hmmm
2007-07-27T15:40:01 <johill> inserted: endContent (False, ) {}
2007-07-27T15:40:01 <johill> nothing on stack, wanted preformatted
2007-07-27T15:40:01 <johill> preformatted (0,) {}
2007-07-27T15:40:01 <johill> endContent () {}
2007-07-27T15:40:07 <johill> it tries to close a preformatted it never opened
2007-07-27T15:42:20 <xorAxAx> the parser is pretty insane :)
2007-07-27T15:43:34 <johill> for sure
2007-07-27T15:43:36 <johill> but I'm getting there
2007-07-27T15:43:48 <johill> this will make for nice tests
2007-07-27T15:44:01 <xorAxAx> yes, functional code is much easier to test
2007-07-27T15:44:10 <xorAxAx> in unittests
2007-07-27T15:47:41 <johill> no I mean use this as a tester
2007-07-27T15:47:48 <johill> anyway, I'm a bit confused
2007-07-27T15:53:35 <johill> how do hah ther
2007-07-27T15:53:39 <johill> xorAxAx: try them again
2007-07-27T15:54:00 <johill> it now checks for good nesting
2007-07-27T15:54:19 <johill> actually, it generates good nesting
2007-07-27T15:56:43 <johill> do headings have to be on the outermost level?
2007-07-27T15:57:13 <xorAxAx> i think html doesnt enforce that, wiki markup does
2007-07-27T15:57:19 <xorAxAx> johill: what should i check again?
2007-07-27T15:57:38 <johill> the two files
2007-07-27T15:57:42 <johill> download them again and run them
2007-07-27T15:57:44 <johill> or do you want the output?
2007-07-27T15:57:52 <xorAxAx> do you mean pickle.py?
2007-07-27T15:58:02 <johill> both really
2007-07-27T15:58:16 <xorAxAx> ah, its in the test file
2007-07-27T15:58:31 <johill> yes
2007-07-27T15:58:37 <xorAxAx> looks like you need the vim7 tabbing feature :)
2007-07-27T15:59:09 <johill> it inserts 74 </p>
2007-07-27T15:59:14 <johill> tabbing? why?
2007-07-27T15:59:33 <xorAxAx> because you prefer editing one file instead of more than one :)
2007-07-27T15:59:40 <johill> oh
2007-07-27T15:59:43 <xorAxAx> hmm, a list called list
2007-07-27T15:59:45 <johill> no, I was just testing
2007-07-27T16:00:09 <johill> phe
2007-07-27T16:00:16 <johill> renamed to nodelist if it makes you hapy ;)
2007-07-27T16:00:57 <xorAxAx> at least its less green in gedit :)
2007-07-27T16:02:01 <xorAxAx> when is if name in expects_close: not true?
2007-07-27T16:02:11 <xorAxAx> ah, for stuff that oesnt have 0 and 1
2007-07-27T16:03:01 <xorAxAx> looks good
2007-07-27T16:03:12 <johill> yup
2007-07-27T16:03:12 <xorAxAx> besides the O(n^2) for the stack.__contains___
2007-07-27T16:04:56 <johill> hm, well
2007-07-27T16:05:11 <johill> could have a dict that keeps a counter
2007-07-27T16:07:37 <johill> now closes everything before a heading
2007-07-27T16:07:49 <xorAxAx> johill: but one question is if tags need to be reopened
2007-07-27T16:07:57 <xorAxAx> johill: but that should be handled by the parser
2007-07-27T16:08:07 <xorAxAx> e.g. '''foo ''bar ''' baz ''
2007-07-27T16:08:33 <xorAxAx> i think it does so already
2007-07-27T16:11:33 <johill> Now O(n) with hashing in the dict
2007-07-27T16:12:47 <xorAxAx> could be made more pythonic :)
2007-07-27T16:13:01 <xorAxAx> by renaming count to __len__, contains to __contains__ etc.
2007-07-27T16:13:39 <johill> nope, it doesn't do that
2007-07-27T16:14:01 <xorAxAx> hmm?
2007-07-27T16:14:28 <johill> well try it
2007-07-27T16:14:46 <xorAxAx> ?
2007-07-27T16:14:54 <johill> it does <strong>foo<emph>bar</strong>baz</emph>
2007-07-27T16:14:59 <xorAxAx> ah
2007-07-27T16:15:12 <johill> and I don't see a way to fix that up
2007-07-27T16:15:33 <johill> well I suppose I could whenever I inserted something insert the opposite after
2007-07-27T16:16:00 <xorAxAx> it works using the current formatter, johill
2007-07-27T16:16:19 <xorAxAx> so the formatter is currently handling that
2007-07-27T16:16:40 <lanius> does pushing work for you guys?
2007-07-27T16:16:56 <xorAxAx> ah, no
2007-07-27T16:16:57 <johill> xorAxAx: not really
2007-07-27T16:17:00 <xorAxAx> johill: sorry :)
2007-07-27T16:17:26 <johill> I can make that work but it'll lead to even more bogus paragraph tags
2007-07-27T16:17:43 <xorAxAx> johill: i guess some tranformation could be written that could fix it
2007-07-27T16:17:47 <xorAxAx> later on
2007-07-27T16:19:59 <johill> but that doesn't work either
2007-07-27T16:20:00 <johill> no
2007-07-27T16:20:02 <johill> see
2007-07-27T16:20:30 <johill> ah no
2007-07-27T16:20:32 <johill> just a slight bug
2007-07-27T16:23:11 <johill> works
2007-07-27T16:23:27 <johill> gotta go
2007-07-27T16:24:06 <johill> ah actually, I have another half hour
2007-07-27T16:24:50 <CIA-27> moin: Heinrich Wendel <h_wendel@cojobo.net> * 2694:17b7b88011ff 1.7-storage-hwendel/MoinMoin/ (7 files in 3 dirs): fixup deleted stuff
2007-07-27T16:25:13 <johill> amazing
2007-07-27T16:25:38 <xorAxAx> lanius: so whats the new semantic?
2007-07-27T16:25:41 <xorAxAx> johill: hmm? :)
2007-07-27T16:26:14 <johill> well
2007-07-27T16:26:46 <johill> it generates 23 calls
2007-07-27T16:26:53 <johill> (for your foo/bar example)
2007-07-27T16:27:10 <johill> I insert another 25 and 13 reopens
2007-07-27T16:27:31 <johill> iow I close 25 and reopen 13
2007-07-27T16:27:36 <xorAxAx> WTF :)
2007-07-27T16:27:44 <johill> well it's the bogus paragraphs
2007-07-27T16:27:49 <johill> I reopen then and end up closing them myself
2007-07-27T16:27:53 <johill> s/then/them
2007-07-27T16:28:16 <xorAxAx> that might be related to the line number handling
2007-07-27T16:28:22 <johill> possibly
2007-07-27T16:28:36 <johill> run the test
2007-07-27T16:28:39 <johill> it's fun to see ;)
2007-07-27T16:28:50 <xorAxAx> i am learning kms, thats more fun :)
2007-07-27T16:29:58 <johill> heh
2007-07-27T16:30:43 <johill> help on parsers
2007-07-27T16:30:48 <johill> generates 360 calls
2007-07-27T16:30:52 <johill> one is killed
2007-07-27T16:31:02 <johill> 775 are closed
2007-07-27T16:31:05 <johill> 701 are reopened
2007-07-27T16:42:10 <johill> ok all works now
2007-07-27T16:43:25 <johill> now I can at least generated a well-nested tree
2007-07-27T16:44:36 <johill> of course you get crap like
2007-07-27T16:44:50 <johill> <heading><p><p>abc</p></p></heading>
2007-07-27T16:44:51 <xorAxAx> <p></p><p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> <p></p> ?
2007-07-27T16:45:05 <xorAxAx> hmm, thats invalid, indeed
2007-07-27T16:45:20 <johill> well it only does well-nestedness
2007-07-27T16:45:26 <johill> and forces headings to the outermost level
2007-07-27T16:45:37 <johill> so if there was a <p> open it gets reopened inside the heading
2007-07-27T16:45:37 <xorAxAx> yeah
2007-07-27T16:45:55 <johill> really
2007-07-27T16:46:02 <johill> heading should never have been heading(on)
2007-07-27T16:46:06 <johill> but rather heading(text)
2007-07-27T16:47:23 <CIA-27> moin: Heinrich Wendel <h_wendel@cojobo.net> * 2696:cd4e3690cced 1.7-storage-hwendel/MoinMoin/storage/ (external.py fs_moin16.py fs_storage.py): separate common file system storage code and special 1.6 code
2007-07-27T16:47:29 <johill> maybe I can rewrite that
2007-07-27T16:48:04 <johill> then again
2007-07-27T16:48:08 <johill> it's trivial to change in the current code
2007-07-27T16:48:25 <xorAxAx> but third party stuff uses the old semantics
2007-07-27T16:48:35 <CIA-27> moin: Heinrich Wendel <h_wendel@cojobo.net> * 2720:f63ec34b4aa7 1.7-storage-hwendel/MoinMoin/ (15 files in 9 dirs): merge main
2007-07-27T16:48:55 <johill> guess it'll have to be updated for 1.7 then
2007-07-27T16:49:33 <johill> or I can rewrite the document tree later
2007-07-27T16:49:40 <johill> and remove all the <p>s inside headings
2007-07-27T16:49:54 <johill> well, really gotta go now
2007-07-27T16:50:28 <xorAxAx> yes, my idea is that we have a sane model where there is Heading(text)
2007-07-27T16:50:38 <xorAxAx> and that the old stuff is fixed by transformations
2007-07-27T18:46:33 <xorAxAx> sigh, he didnt say hi
2007-07-27T19:50:27 <johill> xorAxAx: I forgot formatter.in_p
2007-07-27T19:50:31 <johill> that's why it did so many paragraphs
2007-07-27T19:51:26 <xorAxAx> ah
2007-07-27T19:51:31 <xorAxAx> "forgot"?
2007-07-27T19:54:53 <xorAxAx> johill: maybe you want to create proxying property descriptors for state variables in the proxied formatter :)
2007-07-27T19:55:00 <xorAxAx> hmm
2007-07-27T19:55:02 <xorAxAx> doesnt help ...
2007-07-27T19:55:24 <xorAxAx> johill: can you check how many formatter attrs are accessed by the wiki parser?
2007-07-27T19:55:25 <johill> exactly :)
2007-07-27T19:55:33 <johill> grep? :)
2007-07-27T19:55:46 <xorAxAx> yeah
2007-07-27T19:56:30 * xorAxAx checks
2007-07-27T19:57:20 <xorAxAx> if not (inhibit_p or self.inhibit_p or self.in_pre or self.formatter.in_p or self.no_862):
2007-07-27T19:57:23 <xorAxAx> result.append(self.formatter.paragraph(1, css_class="line862"))
2007-07-27T19:57:26 <xorAxAx> WTF?
2007-07-27T19:57:40 <xorAxAx> johill: only in_p
2007-07-27T19:58:19 <xorAxAx> that makes 14 layering violations that have to be fixed
2007-07-27T19:58:53 <johill> in_pre
2007-07-27T19:59:01 <johill> oh that's self
2007-07-27T19:59:13 <johill> interestingly, formatter has an in_pre variable too
2007-07-27T19:59:16 <johill> anyway
2007-07-27T19:59:31 <johill> it generates pretty good html now
2007-07-27T20:00:04 <johill> after that, a <p><p>...</p></p> -> <p>...</p>
2007-07-27T20:00:04 <johill> filter
2007-07-27T20:00:12 <johill> and an empty paragraph remover
2007-07-27T20:01:18 <johill> I haven't figured out yet how to do empty tag removal correctly
2007-07-27T20:01:24 <johill> <strong></strong> is useless
2007-07-27T20:01:32 <johill> but <span id=...></span> isn't
2007-07-27T20:02:02 <xorAxAx> we are not talking about tags :)
2007-07-27T20:02:14 <johill> node. whatever
2007-07-27T20:02:25 <xorAxAx> in theory, it should be unrelated to html :)
2007-07-27T20:02:37 <johill> exactly
2007-07-27T20:02:45 <johill> hence, it's not easy to do generic empty node rmeoval
2007-07-27T20:02:58 <johill> not possible, even, I guess
2007-07-27T20:03:00 <xorAxAx> well, you know your nodes
2007-07-27T20:03:02 <xorAxAx> of course
2007-07-27T20:03:13 <xorAxAx> esp. because you know your nodes, you may remove them
2007-07-27T20:03:48 <johill> yeah, but who tells me whether the node is useful or not?
2007-07-27T20:03:56 <johill> anyway
2007-07-27T20:03:58 <xorAxAx> and when the model is more formalised (i.e. classes defined), its known what may be used and what not
2007-07-27T20:04:06 <johill> with formatter.in_p it doesn't generate spurious paragraphs
2007-07-27T20:04:11 <johill> so I don't even need that filter
2007-07-27T20:04:26 <xorAxAx> yes
2007-07-27T20:05:30 * xorAxAx has a look into docutils.nodes
2007-07-27T20:05:37 <johill> I uploaded the new files
2007-07-27T20:05:46 <johill> yes, it's all a mess in two files ;)
2007-07-27T20:06:03 <johill> and it writes to /tmp/test.html :)
2007-07-27T20:07:42 <xorAxAx> johill: have a look at docutils.nodes :)
2007-07-27T20:08:24 <xorAxAx> to see how they organised their node tree
2007-07-27T20:08:31 <xorAxAx> or node meta model :)
2007-07-27T20:08:50 <xorAxAx> 217 classes
2007-07-27T20:10:17 <xorAxAx> ok, well, with traversal logic
2007-07-27T20:10:22 * xorAxAx is gone jogging
2007-07-27T20:57:20 <CIA-27> moin: Ville-Pekka Vainio <vpivaini AT cs DOT helsinki DOT fi> * 2719:57e5f9ef94ae 1.7-maninfo-vpv/MoinMoin/script/xmlrpc/manimport.py: fix the bug that caused not all well-formed info pages to be added. also some other cleanups and cleaner debug prints
2007-07-27T20:57:26 <CIA-27> moin: Ville-Pekka Vainio <vpivaini AT cs DOT helsinki DOT fi> * 2727:ee0d3713f3b9 1.7-maninfo-vpv/MoinMoin/ (11 files in 7 dirs): merge from main
2007-07-27T21:02:52 <johill> xorAxAx: 217 classes? uh oh
2007-07-27T21:04:28 <johill> not very pythonic ;)
2007-07-27T21:04:38 <johill> Text node has no __str_ or __unicode__
2007-07-27T21:15:23 <xorAxAx> johill: well
2007-07-27T21:15:29 <xorAxAx> johill: the latter is unpythonic
2007-07-27T21:15:45 <johill> I suppose we could use that DOM
2007-07-27T21:16:01 <xorAxAx> we just need to strip it down
2007-07-27T21:16:37 <johill> question is what we want anyway
2007-07-27T21:16:42 <xorAxAx> yes :)
2007-07-27T21:16:44 <johill> have you seen how I generate html from the tree?
2007-07-27T21:17:04 <xorAxAx> yes
2007-07-27T21:17:40 <johill> I need to fix macro/parser calls though
2007-07-27T21:17:49 <xorAxAx> my idea is to have formatters that may freely deal with the tree
2007-07-27T21:17:58 <xorAxAx> and only have such an approach for legacy formatters
2007-07-27T21:18:02 <johill> sure
2007-07-27T21:18:12 <johill> actually
2007-07-27T21:18:16 <johill> let's not have legacy formatters
2007-07-27T21:18:16 <xorAxAx> beforehand, we need a transformation to the dom and from the dom back to this sequence
2007-07-27T21:18:21 <xorAxAx> hmm?
2007-07-27T21:18:22 <johill> writing formatters isn't hard
2007-07-27T21:18:27 <johill> parsers are much worse
2007-07-27T21:18:45 <xorAxAx> in fact there are not many formatters either
2007-07-27T21:18:50 <johill> yeah
2007-07-27T21:18:59 <johill> so why bother?
2007-07-27T21:19:00 <xorAxAx> indeed its the interface that may be broken at first when thinking about it
2007-07-27T21:19:34 <johill> I think what I did now is simply a backward-compat parser interface
2007-07-27T21:19:42 <johill> basically getting parser stuff andgenerating a dom tree
2007-07-27T21:20:36 <johill> and generating html is trivial
2007-07-27T21:20:55 <xorAxAx> well, currently its trivial because you are using the old interface which you want to drop
2007-07-27T21:21:00 <johill> right
2007-07-27T21:21:08 <johill> but it's even trivial if I just give the thing a node tree
2007-07-27T21:21:26 <johill> but right now I only have well-nestedness
2007-07-27T21:21:36 <johill> no well-formedness or anything
2007-07-27T21:22:02 <johill> (ok well, having headings on the outermost level is almost approaching that partially)
2007-07-27T21:24:27 <johill> so how do we make a valid dom tree with the various nodes?
2007-07-27T21:26:44 <xorAxAx> you mean from the current nestednode structure, johill?
2007-07-27T21:27:34 <johill> yeah
2007-07-27T21:28:06 <johill> or do we just punt
2007-07-27T21:28:16 <johill> and if somebody nests things in a wrong way we raise an exception
2007-07-27T21:28:21 <xorAxAx> punt?
2007-07-27T21:28:40 <xorAxAx> well, one needs to traverse the structure, instantiate the dom classes and build the tree
2007-07-27T21:28:53 <xorAxAx> (recursively)
2007-07-27T21:28:54 <johill> right
2007-07-27T21:29:07 <johill> but if you have <h2><p>...
2007-07-27T21:29:28 <johill> nothing stops you from doing those formatter calls now
2007-07-27T21:29:37 <xorAxAx> yes
2007-07-27T21:29:51 <xorAxAx> so the dom needs to have a heading that is a container class maybe
2007-07-27T21:30:21 <xorAxAx> or you have to headding classes
2007-07-27T21:30:59 <xorAxAx> i think that makes sense - and the cleanup transformation would replace the container heading with a structural heading
2007-07-27T21:31:20 <johill> well that still doesn't help
2007-07-27T21:31:26 <xorAxAx> why?
2007-07-27T21:31:27 <johill> what would it clean it up to if you had
2007-07-27T21:31:50 <johill> <h2><p><h3></h3></p></h2>
2007-07-27T21:32:14 <johill> obviously that's totally bogus
2007-07-27T21:32:17 <xorAxAx> well, no parser or macro generates that
2007-07-27T21:32:18 <johill> but what do we do in that case?
2007-07-27T21:32:23 <johill> so you think ;)
2007-07-27T21:32:24 <xorAxAx> so we dont need to think about it
2007-07-27T21:32:33 <johill> so if we encounter it we raise an exception
2007-07-27T21:32:34 <xorAxAx> it should give an exception in the transformation
2007-07-27T21:32:37 <xorAxAx> yes :)
2007-07-27T21:33:28 <johill> I definitely have <h1><p><p>... now
2007-07-27T21:33:34 <johill> with the wiki parser
2007-07-27T21:34:22 <johill> actually, probably only because I force headings to the outermost level and then reopen the paragraphs myself
2007-07-27T21:34:56 <xorAxAx> btw, i am not sure if <body> <h1> is valid html
2007-07-27T21:38:24 <johill> dunno
2007-07-27T21:39:13 <xorAxAx> validator.w3.org :)
2007-07-27T21:40:23 <johill> it is in xhtml
2007-07-27T21:40:51 <johill> applet | blockquote | body | button | center | dd | del | div | fieldset | form | iframe | ins | li | map | noframes | noscript | object | td | th
2007-07-27T21:41:29 <johill> oh that was 4.01 strict
2007-07-27T21:43:11 <johill> why is it valid in th??
2007-07-27T21:43:13 <johill> or td
2007-07-27T21:48:30 <johill> something's still wrong
2007-07-27T21:48:43 <johill> with" [[macro()]]" in the input Isometimes get a paragraph and sometimes now
2007-07-27T21:51:44 <johill> what's with all the 'ul><li style="list-style-type:none">' it creates?
2007-07-27T21:52:25 <johill> isn't that what paragraphs are for?
2007-07-27T21:52:52 <xorAxAx> no
2007-07-27T21:52:58 <xorAxAx> umm, yes
2007-07-27T21:53:05 <xorAxAx> well, those are indented lists
2007-07-27T21:53:15 <xorAxAx> without bullets
2007-07-27T21:53:25 <xorAxAx> that are neatly composable
2007-07-27T21:53:45 <johill> neatly composable?
2007-07-27T21:53:58 <xorAxAx> well, the indentation increases if you nest such lists
2007-07-27T21:54:04 <xorAxAx> without css, without RTL issues
2007-07-27T21:54:15 <johill> but that works with p/div too, no?
2007-07-27T21:54:19 <xorAxAx> how?
2007-07-27T21:55:11 <johill> text-indent
2007-07-27T21:55:22 <xorAxAx> supported by IE?
2007-07-27T21:55:28 <xorAxAx> and thats CSS
2007-07-27T21:55:34 <xorAxAx> so it wont work in text-only browsers
2007-07-27T21:55:43 <johill> ah no wait
2007-07-27T21:55:43 <xorAxAx> (ok, list-style is css as well :))
2007-07-27T21:56:24 <johill> those lists must look crap in text browsers
2007-07-27T21:56:31 <xorAxAx> :)
2007-07-27T21:56:59 <xorAxAx> the old-school fix is <blockquote> btw
2007-07-27T21:57:07 <xorAxAx> i think there were times when moin spit it out
2007-07-27T21:57:33 <xorAxAx> but its disregarded in such situations because it should not only be a wiki for quotes :)
2007-07-27T21:57:45 <johill> yeah
2007-07-27T21:57:54 <johill> you mean discouraged ;)
2007-07-27T21:57:57 <xorAxAx> ok :)
2007-07-27T21:58:11 <johill> it seems not to be possible. odd
2007-07-27T21:58:21 <johill> but that's something the formatter should worry about
2007-07-27T21:58:22 <johill> not the parser
2007-07-27T21:58:26 <johill> in latex I can easily do that
2007-07-27T21:58:29 <xorAxAx> yes
2007-07-27T21:58:41 <johill> so in the end I don't think what I did makes sense
2007-07-27T21:58:50 <xorAxAx> so p should get some "increase indentation" attr
2007-07-27T21:59:03 <johill> right
2007-07-27T21:59:04 <xorAxAx> or some new class
2007-07-27T21:59:26 <johill> but it means we need to rewrite the parser anyway
2007-07-27T21:59:43 <johill> bbl, phone
2007-07-27T22:07:56 <gufymike> Well I was just surfing http://moinmoin.wikiwikiweb.de/ and it was working a few seconds ago, but I'm getting 500's on the site, this is just an fyi if you didn't know already.
2007-07-27T22:08:53 <xorAxAx> gufymike: we know it already, the admin doesnt :)
2007-07-27T22:09:09 <gufymike> :) ok
2007-07-27T22:43:19 <ThomasWaldmann> lighttpd restarted
2007-07-27T23:03:44 <grzywacz> re
2007-07-27T23:04:43 <xorAxAx> grzywacz: did they say no or yes?
2007-07-27T23:06:22 <grzywacz> xorAxAx, in general: yes, but I'll get a final answer on Tuesday.
2007-07-27T23:07:06 <grzywacz> I'm dead tired now. It was awfully hot, trains sucked and I didn't get any sleep at night, so good night... ;s
2007-07-27T23:07:45 <xorAxAx> have a good night
2007-07-27T23:18:37 <johill> well
2007-07-27T23:18:45 <johill> I don't really see a way forward with the current api
2007-07-27T23:18:51 <xorAxAx> which API?
2007-07-27T23:18:57 <johill> formatter
2007-07-27T23:19:01 <johill> and even parser
2007-07-27T23:19:05 <xorAxAx> why parser?
2007-07-27T23:19:44 <xorAxAx> i dont see any problems continueing this approach
2007-07-27T23:20:23 <johill> well
2007-07-27T23:20:31 <johill> the <ul><li> thing for one
2007-07-27T23:20:40 <johill> it should be in the formatter instead
2007-07-27T23:21:09 <xorAxAx> well
2007-07-27T23:21:40 <xorAxAx> the wiki parser should be converted anyway
2007-07-27T23:21:50 <johill> yeah
2007-07-27T23:21:52 <xorAxAx> so it can use the new sane api
2007-07-27T23:22:02 <johill> so why bother? it's byfar the most complex parser
2007-07-27T23:22:05 <xorAxAx> and other parsers wont see these edge cases very often
2007-07-27T23:22:10 <xorAxAx> johill: ?
2007-07-27T23:22:25 <johill> I mean it'll be easier to convert most other parsers
2007-07-27T23:22:28 <johill> than this one
2007-07-27T23:22:47 <xorAxAx> yes, but most other parsers are not in the distribution
2007-07-27T23:23:26 <johill> and most likely they also generate things that don't fit the tree
2007-07-27T23:23:31 <xorAxAx> why?
2007-07-27T23:23:45 <johill> because they have bugs or whatever
2007-07-27T23:23:54 <xorAxAx> see the parsers on the market, many of them are fine
2007-07-27T23:24:25 <xorAxAx> well, i seriously dont see many issues of having compatiblity here
2007-07-27T23:24:43 <xorAxAx> not much code and not much effort - in fact it makes the transition easier
2007-07-27T23:25:02 <johill> so far it's not much code
2007-07-27T23:25:12 <johill> but we haven't generated a valid dom tree out of this node tree yet
2007-07-27T23:25:13 <xorAxAx> and the next step of thinking about a DOM meta-model is orthogonal to compability questions given that there is a mapping into one direction
2007-07-27T23:25:25 <xorAxAx> :)
2007-07-27T23:25:34 <xorAxAx> i mean, its always possible to map it
2007-07-27T23:25:43 <johill> from the current to dom?
2007-07-27T23:25:46 <xorAxAx> so lets think about the goal structure
2007-07-27T23:25:47 <xorAxAx> yes
2007-07-27T23:25:51 <xorAxAx> dont worry too much
2007-07-27T23:25:55 <johill> heh
2007-07-27T23:26:03 <xorAxAx> it will be less hackery than the current formatter
2007-07-27T23:26:03 <johill> ok I'll just be happy then ;)
2007-07-27T23:26:15 <johill> hah, that may well be true
2007-07-27T23:26:31 <johill> but that's more a function of the current formatter than a function of the hackery involved
2007-07-27T23:26:34 <xorAxAx> larger tasks are the reimplementation of the caching system
2007-07-27T23:26:49 <johill> that actually gets a lot easier
2007-07-27T23:26:55 <xorAxAx> well :)
2007-07-27T23:26:59 <xorAxAx> partly, yes
2007-07-27T23:28:12 <johill> we can attach validity to a node
2007-07-27T23:28:28 <xorAxAx> hmm, macros could return a node that points to a transformation handler :)
2007-07-27T23:28:34 <xorAxAx> (that would be the new TOC macro :))
2007-07-27T23:29:27 <johill> :)
2007-07-27T23:31:38 <johill> actually, that does make caching a bit harder
2007-07-27T23:32:56 <johill> anyway what I was thinking wrt. caching was that the actual formatter returns the cached content, basically
2007-07-27T23:33:02 <xorAxAx> you can pickle functions, they are decomposed into module name and function name
2007-07-27T23:33:22 <johill> you can't pickle instance methods :)
2007-07-27T23:33:29 <xorAxAx> yes
2007-07-27T23:33:58 <johill> well
2007-07-27T23:34:01 <johill> see
2007-07-27T23:34:13 <johill> with caching, what we need to be careful about is what I just did with the IDs
2007-07-27T23:34:14 <xorAxAx> we could just define the name of the handler function as well :)
2007-07-27T23:34:45 <johill> they need to be generated at output time for because of included pages
2007-07-27T23:34:58 <xorAxAx> the generation of ids is a step that runs pretty late
2007-07-27T23:35:28 <xorAxAx> so in the first prototype, it could run after the pickle snapshotting is done, i.e. at rendering time
2007-07-27T23:35:32 <johill> well it could be a tree modifier
2007-07-27T23:35:49 <johill> but you can really pre-render almost everything
2007-07-27T23:35:55 <xorAxAx> in a next version, it could be stored alongside of the tree whether the transformation was aleady executed
2007-07-27T23:36:17 <xorAxAx> that is possible if you have only cachable nodes
2007-07-27T23:36:30 <johill> so what you really want is some way of saying <blabla id=<<insert id for 'xyz'>>
2007-07-27T23:36:39 <johill> with nodes, of course
2007-07-27T23:36:48 <xorAxAx> yes, every node has an id field
2007-07-27T23:37:00 <xorAxAx> well, an optional id field :)
2007-07-27T23:37:07 <johill> yeah
2007-07-27T23:37:33 <johill> however
2007-07-27T23:37:38 <johill> think about the toc formatter again
2007-07-27T23:37:51 <xorAxAx> the one that would be gone?
2007-07-27T23:37:52 <johill> right now it works because we can't cache heading html
2007-07-27T23:38:03 <johill> (due to IDs)
2007-07-27T23:38:58 <johill> oh I guess the easiest is to allow anyone to mark a node as uncacheable
2007-07-27T23:39:09 <johill> then the toc transformation would mark all headings as uncacheable
2007-07-27T23:39:39 <xorAxAx> ?
2007-07-27T23:39:44 <xorAxAx> thats not necessary
2007-07-27T23:39:55 <johill> it is
2007-07-27T23:39:58 <xorAxAx> nope
2007-07-27T23:40:09 <johill> hm, maybe not for the toc
2007-07-27T23:40:17 <johill> it could just insert a node <insert macro toc here>
2007-07-27T23:41:10 <xorAxAx> yes, and the transformation runner sees that there is a pending macro call. the macro evaluates to a node that returns a tree transformation
2007-07-27T23:41:41 <xorAxAx> and that transformation node must have a flag that says "i only want to transform lower levels"
2007-07-27T23:42:09 <johill> sounds complicated to make it refer to the right spot
2007-07-27T23:42:10 <xorAxAx> in that case the transformation driver would run the transform only if all same-level or lower nodes are in their final state
2007-07-27T23:42:21 <xorAxAx> no, not complicated at all :)
2007-07-27T23:42:50 <xorAxAx> if they are not in their final state, the callback is done at rendering time
2007-07-27T23:42:56 <johill> right
2007-07-27T23:43:20 <xorAxAx> so you get the largest static subtrees that are possible
2007-07-27T23:43:56 <johill> it doesn't need to be a subtree if the formatter knows about uncacheable nodes
2007-07-27T23:44:01 <johill> and returns them as-is
2007-07-27T23:44:11 <xorAxAx> hmm?
2007-07-27T23:46:41 <johill> well you could generate the content <div>
2007-07-27T23:47:03 <xorAxAx> give an example
2007-07-27T23:48:18 <johill> <content><macro/></content>
2007-07-27T23:48:35 <xorAxAx> and the macro is cachable?
2007-07-27T23:48:40 <johill> no
2007-07-27T23:48:46 <xorAxAx> argh, car races on the detmolder, like any friday eening
2007-07-27T23:48:50 <johill> but the html formatter could give
2007-07-27T23:49:24 <johill> <raw u'<div clsas=content>'><macro/><raw u'</div>'>
2007-07-27T23:49:59 <xorAxAx> ???
2007-07-27T23:50:09 <xorAxAx> what would be the point?
2007-07-27T23:50:23 <xorAxAx> <macro/> is not html
2007-07-27T23:50:27 <johill> no no
2007-07-27T23:50:36 <johill> I mean it generates 'raw' nodes
2007-07-27T23:50:48 <xorAxAx> why?
2007-07-27T23:50:55 <johill> so you can cahe them
2007-07-27T23:50:57 <johill> cache
2007-07-27T23:51:19 <xorAxAx> and what do you mean by "macro/"?
2007-07-27T23:51:31 <johill> that's the original uncacheable macronode
2007-07-27T23:51:32 <xorAxAx> well, the scheme i presented above yields the best caching results
2007-07-27T23:51:38 <xorAxAx> and you cant get any better than it :)
2007-07-27T23:52:00 <johill> no, you can
2007-07-27T23:52:10 <johill> there's no point to not cache the container of the macro
2007-07-27T23:52:19 <xorAxAx> "container of the macro"?
2007-07-27T23:52:35 <johill> why is that so hard to understand?
2007-07-27T23:52:37 <johill> look
2007-07-27T23:52:40 <xorAxAx> <raw u'<div clsas=content>'><macro/><raw u'</div>'>
2007-07-27T23:52:40 <johill> if you generate a paragraph
2007-07-27T23:52:42 <xorAxAx> is invalid
2007-07-27T23:52:52 <xorAxAx> ah, wait
2007-07-27T23:52:54 <johill> why? it's three nodes
2007-07-27T23:52:58 <xorAxAx> i saw only 1
2007-07-27T23:52:59 <xorAxAx> :-)
2007-07-27T23:53:08 <xorAxAx> ok, now i understand it
2007-07-27T23:53:30 <xorAxAx> yes, you would allow this serialisation transformation only if there is no flag on a callback node that forbids it
2007-07-27T23:53:44 <johill> right
2007-07-27T23:53:48 <johill> or just if it's a callback node
2007-07-27T23:53:54 <johill> macro is just a callback node
2007-07-27T23:54:03 <xorAxAx> a macro is a macro node :)
2007-07-27T23:54:11 <xorAxAx> well, we will see what makes sense :)
2007-07-27T23:54:11 <johill> contained in a call node MoinMoin: MoinMoinChat/Logs/moin-dev/2007-07-27 (last edited 2007-10-29 19:11:02 by localhost)
