2007-07-28T00:04:58 <johill> I think it makes sense to have the 'uncacheable flag' as aa special node type
2007-07-28T00:08:24 <xorAxAx> why is that?
2007-07-28T00:08:50 <xorAxAx> in any case an invalidation checker node would be useful
2007-07-28T00:10:04 <johill> just so aborting the node tree walk generlises better
2007-07-28T00:10:50 <johill> but the macro node could just descend from a Call node
2007-07-28T00:10:56 <xorAxAx> yeah
2007-07-28T00:11:00 <johill> and a call node would mean 'return as is' for a formatter
2007-07-28T00:11:08 <johill> (because otherwise it would alredy be expanded)
2007-07-28T00:14:16 <johill> question is -- the formatter should probably return a list of nodes
2007-07-28T00:14:17 <johill> and not a tree
2007-07-28T00:14:55 <johill> then that list can be pickled for cache
2007-07-28T00:16:02 <xorAxAx> the formatter?
2007-07-28T00:16:14 <johill> yes
2007-07-28T00:16:17 <xorAxAx> hmm
2007-07-28T00:16:23 <xorAxAx> wellll
2007-07-28T00:16:32 <xorAxAx> not a good idea IMHO
2007-07-28T00:16:38 <johill> why not?
2007-07-28T00:16:44 <xorAxAx> because you would need to implement all transformations for 2 datastructures
2007-07-28T00:16:54 <xorAxAx> that are runnable at this late phase
2007-07-28T00:17:04 <xorAxAx> depending in which state you ended up
2007-07-28T00:17:16 <xorAxAx> (if you want to have the fastest system)
2007-07-28T00:17:23 <johill> well then the formatter returns a generic container node that evaulates to nohting
2007-07-28T00:17:33 <xorAxAx> ?
2007-07-28T00:17:38 <xorAxAx> nothing?
2007-07-28T00:17:46 <johill> only it's children
2007-07-28T00:17:50 <johill> its even
2007-07-28T00:18:04 <johill> <raw u'<div clsas=content>'><macro/><raw u'</div>'>
2007-07-28T00:18:06 <johill> is three nodes
2007-07-28T00:18:11 <johill> but what are they contained in?
2007-07-28T00:18:16 <xorAxAx> yes :)
2007-07-28T00:18:18 <xorAxAx> a document node
2007-07-28T00:18:20 <johill> previously, the macro was contained in the content node
2007-07-28T00:19:43 <johill> well
2007-07-28T00:19:54 <johill> then say the formatter evaluates the document node ;)
2007-07-28T00:20:03 <johill> or is that just the generic top-level node then?
2007-07-28T00:20:33 <xorAxAx> yes :)
2007-07-28T00:20:39 <johill> :)
2007-07-28T00:25:21 * johill wonders how to proceed
2007-07-28T00:26:20 <xorAxAx> we need to settle on a DOM meta-model :)
2007-07-28T00:27:42 <xorAxAx> well, some parts of it at least
2007-07-28T00:28:07 <johill> yeah
2007-07-28T00:28:16 <johill> well we already said that we need Call nodes :)
2007-07-28T00:28:17 <johill> oh hm
2007-07-28T00:28:36 <johill> we need two types of call nodes
2007-07-28T00:28:43 <johill> one that is just called and generates new nodes
2007-07-28T00:28:56 <johill> and one that is called with a specific sub-tree after that was called
2007-07-28T00:30:08 <johill> the macro call type is easy
2007-07-28T00:30:10 <xorAxAx> i said above that a flag would be fine
2007-07-28T00:30:16 <johill> it's just a call node that expands to a subtree
2007-07-28T00:30:31 <xorAxAx> but that tree might contain another call node
2007-07-28T00:31:03 <johill> yeah so you walk the subtree it generated again
2007-07-28T00:31:52 <johill> or you can just exapnd that call node when it's created
2007-07-28T00:32:01 <johill> the question is how the toc could be implemented
2007-07-28T00:32:17 <johill> it needs some sort of node that gets a subtree
2007-07-28T00:32:44 <johill> after the subtree is fully evaluated
2007-07-28T00:33:12 <xorAxAx> the TOC macro returns a call node
2007-07-28T00:33:30 <xorAxAx> that refers to a handler and signals that it is only interested in adjacent or sub nodes
2007-07-28T00:33:38 <xorAxAx> (using a flag or class)
2007-07-28T00:34:20 <xorAxAx> maybe it makes sense to store the fact that it is interested in headings only somewhere but i thin kthat leads too far
2007-07-28T00:34:40 <johill> yeah that goes too far since part of the toc can be cached too
2007-07-28T00:34:48 <johill> ah actually, no
2007-07-28T00:34:52 <johill> the IDs are generated later. hrm
2007-07-28T00:35:09 <xorAxAx> well, the ids would be generated before the toc handler is called
2007-07-28T00:35:30 <johill> but that means that we cannot cache any node with an id
2007-07-28T00:35:52 <xorAxAx> no, that means that there needs to be an id-pool in the document node
2007-07-28T00:36:10 <johill> fails with includes
2007-07-28T00:36:16 <xorAxAx> no
2007-07-28T00:36:26 <johill> how would it work?
2007-07-28T00:36:30 <xorAxAx> includes are always done before any id is assigned
2007-07-28T00:36:48 <johill> ok
2007-07-28T00:37:00 <johill> but that means you cannot cache a page with includes
2007-07-28T00:37:07 <xorAxAx> why?
2007-07-28T00:37:07 <johill> or you have to cache the included page with it
2007-07-28T00:37:14 <xorAxAx> why?
2007-07-28T00:37:20 <johill> how else do you assign IDs?
2007-07-28T00:37:34 <johill> you can cache the dom tree of the included page
2007-07-28T00:37:39 <xorAxAx> no
2007-07-28T00:37:40 <johill> but not any generated output
2007-07-28T00:37:44 <xorAxAx> the whole document is cached
2007-07-28T00:37:50 <xorAxAx> without ids if necessary
2007-07-28T00:37:53 <johill> define 'whole document'
2007-07-28T00:37:57 <xorAxAx> the document node
2007-07-28T00:38:01 <johill> right
2007-07-28T00:38:03 <johill> that's easy
2007-07-28T00:38:27 <xorAxAx> if the ids already could be assigned without activating dynamic content, they are cached as well
2007-07-28T00:38:40 <johill> no, they can't
2007-07-28T00:38:46 <xorAxAx> why?
2007-07-28T00:38:51 <johill> if you then included this into another page the ids would conflict
2007-07-28T00:39:01 <xorAxAx> including doesnt use this cache
2007-07-28T00:39:02 <johill> because the cached IDs could conflict
2007-07-28T00:39:37 <xorAxAx> (yes, a rewrite of the include macro would be necessary as well :)
2007-07-28T00:39:51 <johill> but it could use the cache too
2007-07-28T00:40:00 <johill> if you had a way of assigning ids later
2007-07-28T00:40:01 <xorAxAx> there is no point in using it
2007-07-28T00:40:12 <xorAxAx> makes it just more difficult
2007-07-28T00:40:19 <johill> I like challenges ;)
2007-07-28T00:41:44 <xorAxAx> hah!
2007-07-28T00:41:46 <johill> I think it's doable
2007-07-28T00:41:50 <xorAxAx> docutils has a document.ids
2007-07-28T00:41:54 <xorAxAx> that was my idea!
2007-07-28T00:42:06 <xorAxAx> :-)
2007-07-28T00:42:26 <xorAxAx> unpythonicly, their document class is called "document"
2007-07-28T00:42:53 <johill> that's because it adheres to their convention of calling them the same as the xml tag
2007-07-28T00:43:00 <xorAxAx> xml?
2007-07-28T00:43:11 <johill> Classes in lower_case_with_underscores
2007-07-28T00:43:11 <johill> are element classes, matching the XML element generic identifiers in the DTD_.
2007-07-28T00:43:20 <xorAxAx> ah, they even have a DTD
2007-07-28T00:43:54 <johill> I think what I need is refids
2007-07-28T00:45:01 <johill> so a first pass assigns some unique-within-document id
2007-07-28T00:45:14 <xorAxAx> """Holds the whitespace_normalized_name (contains mixed case) of a target.
2007-07-28T00:45:14 <xorAxAx> Required for MoinMoin/reST compatibility."""
2007-07-28T00:45:31 <xorAxAx> weird docstring
2007-07-28T00:45:38 <johill> then the toc comes along and inserts nodes referencing them
2007-07-28T00:46:01 <johill> maybe with something like
2007-07-28T00:46:19 <johill> anchor=concat('#', idof('unique-within-doc'))
2007-07-28T00:46:27 <johill> href, not anchor
2007-07-28T00:47:06 <xorAxAx> why do you need idof if they are unique?
2007-07-28T00:47:13 <johill> only unique within the document
2007-07-28T00:47:21 <xorAxAx> there is only one document
2007-07-28T00:47:26 <johill> but it could include another one
2007-07-28T00:47:27 <xorAxAx> including trees merges them
2007-07-28T00:48:20 <johill> yeah, maybe it's not worth trying to optimise include
2007-07-28T00:48:27 <xorAxAx> ?
2007-07-28T00:48:33 <xorAxAx> i dont see any optimisation
2007-07-28T00:48:51 <johill> well, you could have the html formatter output something like
2007-07-28T00:49:23 <johill> <raw '<h2 id="'><insertid 'something'><raw '">text</h2>"'>
2007-07-28T00:49:29 <johill> (three nodes)
2007-07-28T00:50:13 <johill> then you'd only have to walk that once and assign unique IDs where required and update all idof() statements
2007-07-28T00:50:34 <johill> it'd actually be <insertid 'unique-within-doc'> of course
2007-07-28T00:50:38 <xorAxAx> i am not seeing how this problem is bound to include :)
2007-07-28T00:50:53 <johill> if you don't have anything included then you don't need to generate new IDs
2007-07-28T00:51:02 <johill> there's one ID namespace then
2007-07-28T00:51:20 <johill> with include, you have only one namespace but would really like one for each included doc
2007-07-28T00:52:05 <xorAxAx> ?
2007-07-28T00:52:20 <johill> well let me retry
2007-07-28T00:52:23 <johill> if you have a dom tree
2007-07-28T00:52:32 <johill> then you can go over it and assign unique IDs
2007-07-28T00:52:34 <johill> easily
2007-07-28T00:52:46 <johill> and then format it to html and cache it
2007-07-28T00:52:59 <johill> (along with all IDs you have assigned)
2007-07-28T00:53:08 <xorAxAx> well, the point is that there are only 2 referencing operations
2007-07-28T00:53:15 <xorAxAx> the TOC macro and special links
2007-07-28T00:53:21 <xorAxAx> the former is easy to solve
2007-07-28T00:53:30 <johill> now assume that there is a Call node in that cached tree
2007-07-28T00:53:34 <xorAxAx> so you want to fixup the links as well? sounds like magic a bit
2007-07-28T00:53:59 <xorAxAx> johill: if there is a call node you may not assign IDs
2007-07-28T00:54:15 <johill> actually, you could
2007-07-28T00:54:30 <xorAxAx> yes, by introducing crossreferences
2007-07-28T00:54:34 <johill> and just take care that the nodes the call node generated never reuses ids
2007-07-28T00:54:51 <johill> right
2007-07-28T00:55:51 <xorAxAx> well, thats a further optimisation thats not necessary as a first step i think
2007-07-28T00:56:13 <johill> not sure; we should take it into account
2007-07-28T00:56:31 <xorAxAx> yes
2007-07-28T00:56:35 <xorAxAx> but not implement it :)
2007-07-28T00:56:50 <xorAxAx> as a first step
2007-07-28T00:56:57 <johill> yes
2007-07-28T00:57:05 <johill> but it probably means that all attributes need to be some sort of node too
2007-07-28T00:57:24 <johill> so you can have a href attribute as a crossref(to id) node
2007-07-28T00:58:35 <xorAxAx> well, you would not call it node
2007-07-28T00:58:41 <johill> right
2007-07-28T00:58:46 <johill> but it would have to be possible to wrap it into a node
2007-07-28T00:58:46 <xorAxAx> still an attribute, just not a string
2007-07-28T00:58:54 <xorAxAx> why a node?
2007-07-28T00:58:58 <johill> if the html formatter wants to write it, it needs to create a node if it's not a string
2007-07-28T00:59:15 <xorAxAx> the node is the heading
2007-07-28T00:59:22 <xorAxAx> it has an attribute which is an instance
2007-07-28T00:59:23 <johill> yeah but it could output
2007-07-28T01:00:07 <johill> [raw '<a href="'] [crossref 'to id'] [raw '">link text</a>']
2007-07-28T01:00:11 <johill> (now using [] for nodes)
2007-07-28T01:00:38 <johill> while if the href attribute was a string it could be right there in the raw node
2007-07-28T01:01:02 <johill> hence, it probably should always output a new node for each attribute
2007-07-28T01:01:18 <johill> and then a second step comes along and collects adjacent raw nodes into one
2007-07-28T01:01:27 <xorAxAx> ???
2007-07-28T01:01:32 <xorAxAx> hmm
2007-07-28T01:01:36 <xorAxAx> well
2007-07-28T01:01:56 <xorAxAx> yes, that would be an AttributeNode which is only valid on this lower level
2007-07-28T01:02:01 <johill> yes
2007-07-28T01:02:08 <johill> only as output of a formatter
2007-07-28T01:02:34 <xorAxAx> the formatter has to be able to use its own nodes anyway
2007-07-28T01:02:42 <xorAxAx> for internal state etc.
2007-07-28T01:02:52 <johill> call nodes, I guess
2007-07-28T01:02:58 <xorAxAx> no
2007-07-28T01:03:12 <johill> eh wait
2007-07-28T01:03:14 <johill> why does it need state?
2007-07-28T01:03:28 <johill> stuff it into the node itself
2007-07-28T01:03:31 <xorAxAx> imagine a pdf formatter that is directly outputting the pdf stream
2007-07-28T01:03:43 <xorAxAx> yes, it needs nodes that represent state
2007-07-28T01:03:50 <johill> well, raw nodes containing the pdf stream you mean
2007-07-28T01:04:06 <xorAxAx> no, nodes containing byte positions where stuff needs to be written to
2007-07-28T01:04:10 <xorAxAx> or a relocating table
2007-07-28T01:04:39 <johill> but the pdf formatter shouldn't output the stream itself, it should output raw nodes
2007-07-28T01:04:46 <johill> and then those get concatenated
2007-07-28T01:05:02 <xorAxAx> if you dont offer this possiblity, it cant be cached on this level
2007-07-28T01:05:10 <xorAxAx> but only on the level above
2007-07-28T01:05:20 <xorAxAx> which is fine maybe
2007-07-28T01:05:28 <xorAxAx> (currently moin only uses the bytecode cache for html)
2007-07-28T01:05:41 <johill> ok let's see
2007-07-28T01:05:56 <johill> byte positions might be a problem
2007-07-28T01:06:08 <xorAxAx> yes, this use case is irrelevant
2007-07-28T01:06:12 <xorAxAx> we are not a DMS
2007-07-28T01:06:17 <johill> but it's not hard either
2007-07-28T01:06:25 <johill> just introduce an OffsetOf attribute node
2007-07-28T01:06:31 <xorAxAx> argh :)
2007-07-28T01:06:50 <johill> see
2007-07-28T01:06:58 <johill> if attribute nodes are allowed then we can cache that
2007-07-28T01:07:52 <xorAxAx> welllll
2007-07-28T01:08:04 <xorAxAx> i am not sure if it is enough
2007-07-28T01:08:18 <xorAxAx> lets just drop it, pdf stuff will be fine with caching on the document level
2007-07-28T01:08:34 <johill> but still, why do you think it wouldn't be enough?
2007-07-28T01:12:23 <xorAxAx> well, because you dont know the structure of your document writer
2007-07-28T01:12:30 <xorAxAx> its a layering violation
2007-07-28T01:12:42 <xorAxAx> i dont want to hack reportlab to generate cached pdfs on this level
2007-07-28T01:12:49 <johill> heh
2007-07-28T01:13:11 <johill> but that's because the formatter isn't the lowest level for reportlab
2007-07-28T01:13:23 <xorAxAx> yes
2007-07-28T01:14:14 <johill> so it's not really a problem. however, attribute nodes are still needed for IDs
2007-07-28T01:14:26 <johill> (and it's somewhat tough to make an OffsetOf node work right anyway)
2007-07-28T01:16:08 <johill> and actually, it isn't nice
2007-07-28T01:16:18 <johill> because with attribute nodes, the knowledge of how to evaluate them is built-in
2007-07-28T01:16:33 <johill> 'print the ID of that other object'
2007-07-28T01:17:20 <johill> hm
2007-07-28T01:17:28 <johill> references should probably be just that, references
2007-07-28T01:17:30 <johill> i.e.
2007-07-28T01:18:06 <johill> x = UniqueID(AttributeNode id="something">
2007-07-28T01:18:09 <johill> eh
2007-07-28T01:18:14 <johill> x = UniqueID('something')
2007-07-28T01:18:24 <johill> no hmm
2007-07-28T01:18:39 <johill> it'd be
2007-07-28T01:18:52 <johill> idof('some text')
2007-07-28T01:18:56 <johill> or would it be
2007-07-28T01:18:59 <johill> idof(some_node)
2007-07-28T01:19:37 <johill> text is easier
2007-07-28T01:19:45 <johill> but requires that the text is already unique
2007-07-28T01:20:21 <johill> which I suppose isn't really a constraint since you can give it a document-wide unique id
2007-07-28T01:39:01 <johill> I wonder how much of that should be private to the formatter though
2007-07-28T01:39:31 <johill> basically, anybody linking to somewhere else would create an anchorlink
2007-07-28T01:39:49 <xorAxAx> not private at all
2007-07-28T01:39:50 <johill> containing either the anchor name or an anchor attribute idof object
2007-07-28T01:40:31 <johill> in fact
2007-07-28T01:40:37 <johill> maybe it should just point to the other node
2007-07-28T11:48:56 <grzywacz> moin
2007-07-28T11:49:01 <xorAxAx> moin grzywacz
2007-07-28T12:13:03 <ThomasWaldmann> moin
2007-07-28T12:19:24 <grzywacz> http://www.artima.com/weblogs/viewpost.jsp?thread=211200
2007-07-28T12:21:27 <xorAxAx> moin has many 3.0-incomaptible stuff
2007-07-28T12:23:59 <grzywacz> I wonder how was will it get adopted.
2007-07-28T12:26:19 <xorAxAx> was?
2007-07-28T12:27:32 <ThomasWaldmann> grzywacz: we avoid incompatible stuff that can be done in a 3.0 compatible way with py 2.x
2007-07-28T12:28:29 <ThomasWaldmann> but for the rest, we will have to use the converter and (at some time in the not-so-near future) do a new major moin release that runs on py3k only (moin3k :)
2007-07-28T12:32:31 <ThomasWaldmann> currently we require 2.3 (and obviously we have users using 2.3), it will take some time until we require 2.4, then 2.5, ...
2007-07-28T12:39:05 <grzywacz> moin3k sounds enterprisey ;)
2007-07-28T12:41:39 <ThomasWaldmann> no
2007-07-28T12:42:01 <ThomasWaldmann> to be enterprisey, you need old, broken and bloated stuff.
2007-07-28T12:42:16 <xorAxAx> which is moin :)
2007-07-28T12:42:28 <grzywacz> ;->
2007-07-28T12:42:49 * ThomasWaldmann slaps xorAxAx with a big Python.
2007-07-28T12:44:18 <grzywacz> Tasty snakes!
2007-07-28T12:57:12 * grzywacz has subscribed to planet.python.org and is now spammed with interesting python-related articles ;)
2007-07-28T13:11:24 <johill> xorAxAx: I'm still not really decided how to do the id resolving
2007-07-28T13:13:00 <xorAxAx> i dont see a problem :)
2007-07-28T13:13:18 <johill> say the html formatter inserts an attribute node
2007-07-28T13:13:18 <xorAxAx> and i think that your function is not necessary
2007-07-28T13:13:37 <johill> how does the other code that referenced the heading node know that it now needs to reference that attribute node?
2007-07-28T13:14:08 <xorAxAx> no idea, i suggested something different :)
2007-07-28T13:14:21 <xorAxAx> either the tree contains uncachable nodes or not
2007-07-28T13:14:55 <xorAxAx> in the former case, you cannot transform the stuff into the raw intermediate repr because on that level, it would be too late to run e.g. the TOC callback
2007-07-28T13:15:09 <johill> yeah I guess it's better to just go by nodes
2007-07-28T13:15:27 <johill> though the formatter could still replace the text node within a heading node by a raw node
2007-07-28T13:15:40 <johill> so it only needs to write the heading later
2007-07-28T13:15:47 <xorAxAx> a heading node is not a compound node
2007-07-28T13:20:12 <ThomasWaldmann> http://code.google.com/p/modwsgi/wiki/IntegrationWithMoinMoin
2007-07-28T13:20:31 <johill> xorAxAx: well, it is now, and it is in html
2007-07-28T13:20:58 <xorAxAx> johill: i suggested yesterday to have 2 heading node types
2007-07-28T13:21:52 <xorAxAx> ThomasWaldmann: nothing fancy on that page
2007-07-28T13:21:55 <johill> yeah, not sure I see the point
2007-07-28T13:22:14 <xorAxAx> johill: the idea is that headings should not contain any markup
2007-07-28T13:22:34 <johill> but they can, in html
2007-07-28T13:22:40 <xorAxAx> so?
2007-07-28T13:22:54 <johill> yeah I guess nobody really cares about that
2007-07-28T13:23:04 <johill> so anyway, let's say you have a div, which is a container node
2007-07-28T13:23:08 <johill> we're getting side-tracked again ;)
2007-07-28T13:23:12 <xorAxAx> many people would like to have links in headings
2007-07-28T13:23:21 <xorAxAx> but only because headings are the only way to increase font size
2007-07-28T13:23:24 <johill> that, however, is not allowed afaik
2007-07-28T13:23:39 <xorAxAx> in html? sure
2007-07-28T13:24:05 <ThomasWaldmann> xorAxAx: mod_wsgi is now 1.0rc1
2007-07-28T13:24:38 <johill> oh looks like it is
2007-07-28T13:45:22 <ThomasWaldmann> that mod_wsgi stuff looks much better than everything else we have for apache
2007-07-28T13:45:29 <xorAxAx> lanius: ping
2007-07-28T13:45:36 <xorAxAx> ThomasWaldmann: well
2007-07-28T13:45:42 <xorAxAx> ThomasWaldmann: its still in process, right?
2007-07-28T13:45:51 <xorAxAx> so it has similar problems like mod_py
2007-07-28T13:45:53 <ThomasWaldmann> it supports daemon mode, too
2007-07-28T13:45:59 <xorAxAx> ah
2007-07-28T13:46:25 <ThomasWaldmann> http://code.google.com/p/modwsgi/
2007-07-28T13:50:23 <johill> fun
2007-07-28T13:50:28 <johill> the python 2.5 docs contain
2007-07-28T13:50:29 <johill> raise pickle.UnpicklingError, 'Invalid persistent id'
2007-07-28T13:50:44 <xorAxAx> the docs?
2007-07-28T13:51:04 <johill> as an example
2007-07-28T13:51:07 <xorAxAx> ah
2007-07-28T13:51:14 <johill> well, in an example
2007-07-28T13:51:59 <xorAxAx> and the error doesnt exist?
2007-07-28T13:53:04 <johill> oh, but look at the syntax
2007-07-28T13:54:08 <xorAxAx> hm? thats valid
2007-07-28T13:55:04 <johill> I thought you were supposed to raise instances directly now
2007-07-28T13:55:31 <xorAxAx> no
2007-07-28T13:57:24 <johill> what happens if I pickle something that has circular references?
2007-07-28T13:58:27 <xorAxAx> it makes an internal symbolic id and uses it for the next appearance
2007-07-28T13:58:34 <xorAxAx> i.e. it works
2007-07-28T13:58:43 <johill> oh good so I can just pickle a document node and expect it to work
2007-07-28T14:30:45 <lanius> xorAxAx: pong
2007-07-28T14:31:26 <xorAxAx> lanius: is locking already implemented?
2007-07-28T14:31:43 <lanius> which kind of locking
2007-07-28T14:31:45 <xorAxAx> so that it avoids metadata corruptions
2007-07-28T14:31:51 <xorAxAx> the one on the metadata level
2007-07-28T14:32:09 <lanius> yes
2007-07-28T14:32:18 <xorAxAx> ok
2007-07-28T14:32:25 <lanius> set lock=True, edit metadata, save,set lock=False
2007-07-28T14:32:34 <xorAxAx> then have you had some ideas of what needs to be cached?
2007-07-28T14:32:44 <lanius> not in the wiki yet
2007-07-28T14:32:52 <lanius> currently working on saving the edit-log via the metadata
2007-07-28T14:33:29 <xorAxAx> ok
2007-07-28T14:33:38 <lanius> but i will write them down now
2007-07-28T14:41:08 <lanius> xorAxAx: http://moinmo.in/StorageRefactoring/SOC2007#head-f4edb9071193edeb65dde99867dfbea575cd44be
2007-07-28T14:43:31 <xorAxAx> 250% slower but the measure would only make it 25% faster?
2007-07-28T14:43:46 <lanius> 25% faster then the 1.6
2007-07-28T14:44:31 <xorAxAx> all options are b0rked IMHO
2007-07-28T14:44:40 <lanius> what's your opinion
2007-07-28T14:44:41 <xorAxAx> did you remember how i recommended how to implement the editlog?
2007-07-28T14:44:52 <lanius> which part of the editlog
2007-07-28T14:44:55 <xorAxAx> now you only need to add a timestamp to the name, rev tuple
2007-07-28T14:44:58 <xorAxAx> the global editlog
2007-07-28T14:45:20 <xorAxAx> so you can query the backend for "give me every items that are newer than $timestamp"
2007-07-28T14:45:42 <xorAxAx> easy to implement, fast, concurrency safe
2007-07-28T14:45:59 <lanius> well the timestamp is already there for page
2007-07-28T14:45:59 <lanius> s
2007-07-28T14:46:07 <xorAxAx> ?
2007-07-28T14:46:15 <lanius> the edit-log for pages contains a timestamp
2007-07-28T14:46:15 <xorAxAx> where?
2007-07-28T14:46:22 <lanius> first entry in each line
2007-07-28T14:46:25 <xorAxAx> i am talking about the global edit log
2007-07-28T14:46:28 <xorAxAx> lines?
2007-07-28T14:46:38 <xorAxAx> the storage bcakend doesnt have the notion of lines
2007-07-28T14:46:47 <lanius> i'm talking about the current file format
2007-07-28T14:46:52 <xorAxAx> yes, so
2007-07-28T14:46:54 <xorAxAx> ...?
2007-07-28T14:47:07 <lanius> so for pages there is already an information about the changed timestamp
2007-07-28T14:47:13 <xorAxAx> yes, i know
2007-07-28T14:47:16 <lanius> which can be used
2007-07-28T14:47:19 <lanius> but for users not
2007-07-28T14:47:28 <xorAxAx> yes
2007-07-28T14:47:35 <xorAxAx> thats an implementation detail
2007-07-28T14:47:39 <xorAxAx> i am talking about the API
2007-07-28T14:47:50 <lanius> yes, i'm currently talking about the implementation
2007-07-28T14:47:54 <xorAxAx> thats irrelevant
2007-07-28T14:47:57 <lanius> i can't add information which is currently not available
2007-07-28T14:47:58 <xorAxAx> on that level
2007-07-28T14:48:03 <xorAxAx> ?
2007-07-28T14:48:13 <lanius> the thing must be compatible with 1.6
2007-07-28T14:48:21 <xorAxAx> of course you can add a new file
2007-07-28T14:48:43 <lanius> or i could just call os.getmtime
2007-07-28T14:48:45 <lanius> ok
2007-07-28T14:48:57 <xorAxAx> so did you understand my suggestion?
2007-07-28T14:49:03 <lanius> so you would add a new method to the backend interface?
2007-07-28T14:49:08 <xorAxAx> every backend needs to implement the news operation
2007-07-28T14:49:10 <xorAxAx> yes
2007-07-28T14:49:43 <xorAxAx> btw, can you benchmark the pagelist retrieval in a unittest/code snippet (as in cgi) in 1.6 vs. your branch?
2007-07-28T14:49:57 <lanius> ?
2007-07-28T14:50:00 <xorAxAx> moin caching should show no benefit there and you should see the real performance difference
2007-07-28T14:50:12 <xorAxAx> if it is still slower, then there something else wrong
2007-07-28T14:50:28 <xorAxAx> lanius: i mean you should retrieve the page list exactly once
2007-07-28T14:50:31 <xorAxAx> in the process
2007-07-28T14:50:38 <xorAxAx> you can do that easily on the command line
2007-07-28T14:50:44 <lanius> yes, there is no difference then
2007-07-28T14:50:47 <xorAxAx> ah
2007-07-28T14:50:52 <lanius> i'm starting moin as standalone
2007-07-28T14:51:01 <lanius> the first call of PageList is exactly the same in 1.6 and my branch
2007-07-28T14:51:04 <lanius> the second one is faster in 1.6
2007-07-28T14:53:01 <lanius> xorAxAx: i'm thinking how this can be implemented in a faster manner
2007-07-28T14:53:05 <lanius> the news()
2007-07-28T14:53:32 <xorAxAx> hmm?
2007-07-28T14:53:48 <xorAxAx> you need to write a small file for every modification
2007-07-28T14:54:46 <lanius> doing an mtime() on each item would not be really faster i think
2007-07-28T14:56:49 <xorAxAx> ?
2007-07-28T14:57:01 <xorAxAx> you only need to do it on one file
2007-07-28T14:57:11 <lanius> what do you mean with "a small file"
2007-07-28T14:57:21 <xorAxAx> a file that stores the 3-tuple
2007-07-28T14:57:40 <xorAxAx> and can be searched using bisection
2007-07-28T14:57:50 <lanius> rev is not needed i think
2007-07-28T14:57:55 <lanius> bisection?
2007-07-28T14:58:09 <xorAxAx> binary search
2007-07-28T14:58:13 <ThomasWaldmann> please don't use file mtimes except for caching consistency checks
2007-07-28T14:58:46 <lanius> xorAxAx: ah ok
2007-07-28T14:58:51 <xorAxAx> hmm, you dont even need a binary search
2007-07-28T14:58:57 <lanius> xorAxAx: but the file will grow quite big
2007-07-28T14:59:25 <xorAxAx> as you have single ops in the backend, yes
2007-07-28T14:59:45 <lanius> yes
2007-07-28T15:00:10 <xorAxAx> how many ops do you usually have for a page save?
2007-07-28T15:00:56 <lanius> create_revision(), data.write(), data.close(), metadata.write(), metadta.save()
2007-07-28T15:01:12 <lanius> lock = true, lock = false
2007-07-28T15:01:18 <lanius> so you could do it on lock = False
2007-07-28T15:05:11 <ThomasWaldmann> btw, don't waste too much time in tuning the 1.6 compatible backend
2007-07-28T15:05:50 <ThomasWaldmann> a more complete and better working backend is far more important
2007-07-28T15:06:55 <lanius> i think it would be the same mechanism for a new backend
2007-07-28T15:07:17 <lanius> ThomasWaldmann: i seperated common stuff which can be used for a new file backend in fs_storage.py and left the 1.6 specific code in fs_moin16.py
2007-07-28T15:07:40 <ThomasWaldmann> ok
2007-07-28T15:12:21 <xorAxAx> lanius: and the news operation would be used for the global editlog and synchronisation
2007-07-28T15:12:35 <ThomasWaldmann> and be careful with updating a "file with 3-tuples". as we have recently seen, append mode does not work reliably on nfs.
2007-07-28T15:12:43 <xorAxAx> lanius: for the former, you will need the rev numbers
2007-07-28T15:13:22 <xorAxAx> ThomasWaldmann: one could enable forced locks for such pathologic scenarios (in fact, i tonly means that the nfs version is too old)
2007-07-28T15:13:52 <ThomasWaldmann> or just not have a single file.
2007-07-28T15:14:29 <ThomasWaldmann> and you know it is fixed since which nfs version?
2007-07-28T15:18:51 <ThomasWaldmann> btw, i guess i am repeating myself, but why not use a special item for global edit-log (as we do it for local edit-log).
2007-07-28T15:21:07 <ThomasWaldmann> and the news() stuff could be simply the cache remembering the latest revision (and looking if this is still the same)
2007-07-28T15:21:39 <xorAxAx> ?
2007-07-28T15:21:50 <xorAxAx> hmm, yes
2007-07-28T15:21:56 <xorAxAx> revision numbers instead of timestamps
2007-07-28T15:22:36 <ThomasWaldmann> that means, if your item implementation works on nfs your global edit-log will also work (and vice versa)
2007-07-28T15:23:03 <xorAxAx> but that will generate a lot of small files
2007-07-28T15:23:12 <ThomasWaldmann> we have them anyway
2007-07-28T15:23:15 <xorAxAx> which will slow down the wiki significantly after a short period of time
2007-07-28T15:23:19 <xorAxAx> not really
2007-07-28T15:23:53 <ThomasWaldmann> n*3 vs. n*4
2007-07-28T15:23:58 <xorAxAx> it will generate O(n*r) files while a single item only has at most O(r) files in directory
2007-07-28T15:24:17 <ThomasWaldmann> no, n*2 vs n*3
2007-07-28T15:24:47 <ThomasWaldmann> (and we WILL have purge)
2007-07-28T15:25:04 <ThomasWaldmann> thus it still is about n*2
2007-07-28T15:25:09 <xorAxAx> ???
2007-07-28T15:25:14 <xorAxAx> what is n*2?
2007-07-28T15:25:26 <xorAxAx> well, purging at this point is a bit unfortunate because i want to see old RC items sometimes
2007-07-28T15:25:26 <ThomasWaldmann> n= total amount of item revisions
2007-07-28T15:25:34 <xorAxAx> not really
2007-07-28T15:25:44 <xorAxAx> umm
2007-07-28T15:25:53 <xorAxAx> yes, and that n is much too large
2007-07-28T15:25:56 <xorAxAx> to fit into one folder
2007-07-28T15:26:12 <xorAxAx> well, purging at this point is a bit unfortunate because i want to see old RC items sometimes - older than e.g. 14 days
2007-07-28T15:26:17 <CIA-27> moin: Heinrich Wendel <h_wendel@cojobo.net> * 2721:20432cd4706a 1.7-storage-hwendel/MoinMoin/storage/ (fs_moin16.py fs_storage.py): separate more stuff to global filesystem storage
2007-07-28T15:26:27 <xorAxAx> but you can also say that purging is only necessary for file backends
2007-07-28T15:26:28 <ThomasWaldmann> who wants to see RecentChanges with 10000 entries?
2007-07-28T15:26:32 <xorAxAx> i think that would be fine
2007-07-28T15:26:33 <xorAxAx> me :)
2007-07-28T15:27:08 <ThomasWaldmann> then you have to live with 10000 files in a directory :)
2007-07-28T15:27:26 <xorAxAx> i mean the moinmo.in wiki :)
2007-07-28T15:27:29 <ThomasWaldmann> (which shouldnt be a problem except for fat)
2007-07-28T15:27:48 <xorAxAx> well, the idea is that you can offer a migration path to some more enterprisey backend
2007-07-28T15:27:51 <ThomasWaldmann> and btw, we might do that tgz bundling stuff
2007-07-28T15:27:52 <xorAxAx> in such a case
2007-07-28T15:27:55 <lanius> for databases the edit-log is very simple to implement, just do a query "SELECT * from item where MTIME > mtime" or something like this
2007-07-28T15:28:14 <xorAxAx> lanius: denormalized!
2007-07-28T15:28:20 <xorAxAx> lanius: you mean item_revs :)
2007-07-28T15:28:23 <lanius> right
2007-07-28T15:28:27 <lanius> :-)
2007-07-28T15:29:06 <lanius> the question is how does the interface to the global-edit log look like
2007-07-28T15:29:08 <xorAxAx> and with a db bcakend, you dont need purging
2007-07-28T15:29:15 <lanius> right
2007-07-28T15:29:21 <ThomasWaldmann> so there wont be 10000 files per item, but < 200
2007-07-28T15:30:08 <xorAxAx> lanius: well, you could have some combineditemcache that caches the item revisions of different items of different itembackends
2007-07-28T15:30:22 <xorAxAx> lanius: it would issue news() calls
2007-07-28T15:30:46 <xorAxAx> and the RC macro would use it as a global editlog
2007-07-28T15:31:21 <lanius> yes, but RC is not the only one
2007-07-28T15:33:43 <xorAxAx> you mean you want to preserve the editlog interface?
2007-07-28T15:33:47 <xorAxAx> should be doable
2007-07-28T15:34:28 <lanius> a search for editlog gives 21 results
2007-07-28T15:34:47 <xorAxAx> local or global?
2007-07-28T15:34:50 <lanius> both
2007-07-28T15:34:58 <lanius> more global than local
2007-07-28T15:35:22 * ThomasWaldmann would like to see logfile.* die
2007-07-28T15:35:45 <xorAxAx> lanius: so?
2007-07-28T15:35:48 <xorAxAx> you need to migrate most
2007-07-28T15:35:54 <xorAxAx> same for get_page_path
2007-07-28T15:36:10 <lanius> yes
2007-07-28T15:36:18 <lanius> ThomasWaldmann: there is still this event stuff in logfile.*
2007-07-28T15:36:30 <xorAxAx> event stuff?
2007-07-28T15:36:39 <lanius> logfile.eventlog
2007-07-28T15:38:12 <ThomasWaldmann> you dont need to care about that
2007-07-28T15:38:13 <xorAxAx> well, that has very low priority, i would keep it in the current state currently
2007-07-28T15:38:32 <lanius> yes, just wanted to notice
2007-07-28T15:38:51 <ThomasWaldmann> we either can reimplement it in some easier way or just drop the stats stuff (it also has some other problems as it is)
2007-07-28T15:39:19 <lanius> first drop then reimplement :D
2007-07-28T15:39:35 <xorAxAx> lanius: well, first keep, then replace :)
2007-07-28T15:39:58 <lanius> better not look at old and crufty code but start from scratch :-)
2007-07-28T15:50:40 <CIA-27> moin: Heinrich Wendel <h_wendel@cojobo.net> * 2722:57c3bcec8982 1.7-storage-hwendel/MoinMoin/ (PageEditor.py action/edit.py storage/_tests/__init__.py): remove another getPagePath
2007-07-28T16:00:40 <johill> hm
2007-07-28T16:00:45 <johill> thilo's wikidom is quite complicated
2007-07-28T16:06:03 <johill> but it does raise the question
2007-07-28T16:06:11 <johill> what if you have two tables of contents in one page
2007-07-28T16:06:23 <johill> then both will indicate that they require a complete tree to work on
2007-07-28T16:06:27 <johill> and then you're stuck
2007-07-28T16:07:27 <johill> or you make it an iterative process
2007-07-28T16:07:51 <johill> a TOC macro node always emits what it can at that point, and leaves some call nodes hanging
2007-07-28T16:08:20 <johill> and then you call the new call nodes until none are left
2007-07-28T16:09:43 <johill> obviously, you need to catch runaway situations like a toc macro generating new headings that the other toc macro will try to collect
2007-07-28T16:11:11 <lanius> ThomasWaldmann: does the test_PageEditor.TestExpandPrivateVariables test makes any sense?
2007-07-28T16:12:58 <xorAxAx> johill: well, in fact a TOC doesnt generate headings
2007-07-28T16:13:44 <xorAxAx> johill: it could be signalled in the callback node that the callback itself wont change the structure of the document but only th econtents
2007-07-28T16:14:08 <johill> xorAxAx: I know it doesn't, but ti's only a matter of time until somebody screws that up
2007-07-28T16:14:39 <johill> ah, but then you end up in a trap: what is contents and what is structure?
2007-07-28T16:14:46 <johill> (plus I don't see much point)
2007-07-28T16:14:58 <johill> better just iterate until no call nodes are left or until you've done say 20 steps
2007-07-28T16:16:31 <xorAxAx> johill: well, the only other way is to amend the callback node with subnodes and call it as long as the document is stable
2007-07-28T16:16:39 <xorAxAx> so either flagging or fix point
2007-07-28T16:17:41 <johill> you mean until the document is stable?
2007-07-28T16:18:34 <xorAxAx> "as the document is stable
2007-07-28T16:18:43 <xorAxAx> hmm, yeah, until
2007-07-28T16:18:46 <xorAxAx> %-)
2007-07-28T16:19:05 <johill> yeah but there's no requirement for it to stabilize
2007-07-28T16:19:07 <johill> in theory
2007-07-28T16:19:19 <ThomasWaldmann> lanius: well, it tests the _expand_variables function
2007-07-28T16:19:29 <xorAxAx> then you have a counter and replace the node by an error message
2007-07-28T16:19:35 <johill> yeah
2007-07-28T16:20:40 <ThomasWaldmann> (there could be more tests, of course)
2007-07-28T16:20:59 <xorAxAx> lanius: + if self.getRevList() == []:
2007-07-28T16:21:04 <xorAxAx> lanius: not getRevList()
2007-07-28T16:21:09 <xorAxAx> getRevList is not PEP8
2007-07-28T16:21:32 <ThomasWaldmann> lanius: wait, i looked at the wrong one...
2007-07-28T16:21:57 <lanius> xorAxAx: getRevList() is because it was named this before
2007-07-28T16:22:03 <xorAxAx> lanius: oh
2007-07-28T16:22:07 <xorAxAx> lanius: didnt know that :)
2007-07-28T16:22:13 <lanius> now you know :D
2007-07-28T16:22:16 <xorAxAx> hehe
2007-07-28T16:22:56 <johill> I'm thinking we could also introduce dependencies; like "this call depends on some other node being finalized"
2007-07-28T16:23:39 <johill> then the toc thing could emit a new call node for each heading and for each other call node that depends on that other node
2007-07-28T16:24:01 <johill> so once those are expanded (finalised) the call node there is invoked
2007-07-28T16:24:15 <johill> this helps with IDs because nodes with IDs are only finalised by the ID assigner
2007-07-28T16:25:41 <xorAxAx> hmm
2007-07-28T16:25:55 <lanius> ThomasWaldmann: what about the right one ;)
2007-07-28T16:25:56 <xorAxAx> so you have a handler in the callback node that is used to scan for dependencies
2007-07-28T16:26:11 <xorAxAx> you will have to call it again after you have modified the document, though
2007-07-28T16:26:25 <xorAxAx> (like having substituted a macro call node by its contents
2007-07-28T16:27:33 <xorAxAx> so you need to register a visitor that is called for every tree modification operation
2007-07-28T16:27:36 <johill> well if you have a placeholder node like a macro call then once you finalise it (i.e. expand it) you call everything that depends on it
2007-07-28T16:28:24 <xorAxAx> johill: how does the other stuff signal that it depends on it?
2007-07-28T16:28:25 <johill> generally, it probably makes sense to distinguish between unfinished and placdeholder nodes
2007-07-28T16:28:28 <ThomasWaldmann> lanius: i have problems about finding out why it needs that much code. it seems to only test that expands to the user name.
2007-07-28T16:28:42 <xorAxAx> furthermore, it cant know if it would, because the macro node wasnt activated (i like that term :)) yet
2007-07-28T16:28:45 <johill> well it creates a placeholder node with a dependency to that
2007-07-28T16:28:53 <ThomasWaldmann> (which does not correspond to the test class name)
2007-07-28T16:29:03 <xorAxAx> i dont understand the last sentence, johill
2007-07-28T16:29:09 <johill> ok
2007-07-28T16:29:15 <lanius> ThomasWaldmann: right, that's my problem was well
2007-07-28T16:29:18 <johill> say the toc visitor finds a placeholder node
2007-07-28T16:29:30 <johill> then it inserts a placeholder node that depends on the other one it found
2007-07-28T16:29:40 <xorAxAx> another one?
2007-07-28T16:29:41 <xorAxAx> where?
2007-07-28T16:29:46 <johill> (which internally adds it to a list in the other node)
2007-07-28T16:29:54 <johill> where the toc is included
2007-07-28T16:30:16 <xorAxAx> and then?
2007-07-28T16:30:18 <johill> basically, the idea is to visit the whole tree only once
2007-07-28T16:30:41 <xorAxAx> umm
2007-07-28T16:30:47 <xorAxAx> thats not possible i fear
2007-07-28T16:30:52 <johill> and then, the scanner also adds it to the todo list
2007-07-28T16:31:00 <xorAxAx> as we planned to have a few transformations :)
2007-07-28T16:31:05 <xorAxAx> scanner?
2007-07-28T16:31:12 * xorAxAx is confused
2007-07-28T16:31:27 <johill> let me try to find clear terms we can agree on
2007-07-28T16:34:59 <johill> http://moinmoin.wikiwikiweb.de/JohannesBerg/DOM
2007-07-28T16:35:08 <lanius> ThomasWaldmann: well, i just refactored it to use new page storage stuff
2007-07-28T16:36:24 <xorAxAx> johill: ok
2007-07-28T16:37:36 <xorAxAx> so now describe the transformation that renders the tree into its static final state
2007-07-28T16:39:25 <johill> that would be multiple transformers I guess
2007-07-28T16:39:42 <xorAxAx> yeah, the one that handles the TOC
2007-07-28T16:40:05 <xorAxAx> do you disagree with my loop-until-stable approach?
2007-07-28T16:40:17 <xorAxAx> i think you did by introducing dependency metadata
2007-07-28T16:40:42 <johill> no, I don't
2007-07-28T16:41:16 <johill> I'm just thinking that it should be possible to write a toc as a macro
2007-07-28T16:42:21 <xorAxAx> yes, a toc would be a macro that returns a callback node :)
2007-07-28T16:42:31 <johill> right
2007-07-28T16:42:50 <johill> so then we have an expansion transform
2007-07-28T16:42:53 <xorAxAx> which has a flag that says that it wants to be called with a subtree only
2007-07-28T16:43:09 <johill> a subtree only?
2007-07-28T16:43:22 <johill> oh right
2007-07-28T16:43:26 <xorAxAx> yes, a toc only operates on sisters and children
2007-07-28T16:43:31 <johill> the subtree it is directly contained in I guess
2007-07-28T16:43:41 <xorAxAx> yes
2007-07-28T16:43:44 <johill> well, assuming the wiki parser doesn't embed it into a paragraph
2007-07-28T16:44:05 <johill> bit tricky
2007-07-28T16:44:26 <xorAxAx> hmm, yeah
2007-07-28T16:44:43 <johill> so maybe not subtree but rather next level document up
2007-07-28T16:44:50 <johill> or parseddocument or whatever
2007-07-28T16:45:02 <xorAxAx> hmm
2007-07-28T16:45:08 <xorAxAx> maybe we need a subdocument node
2007-07-28T16:45:12 <johill> for sure
2007-07-28T16:45:19 <xorAxAx> that is used as an anchor point
2007-07-28T16:45:26 <xorAxAx> for sure operations
2007-07-28T16:47:38 <johill> now
2007-07-28T16:47:59 <johill> the toc placeholder node gets called with the subdocument
2007-07-28T16:48:11 <johill> it walks it and finds things it wants
2007-07-28T16:48:29 <johill> if they're final, it simply inserts whatever nodes it needs, but most likely they're not
2007-07-28T16:49:09 <johill> so for any non-final node it inserts a placeholder node
2007-07-28T16:49:20 <johill> any transformer gets notified of tree changes btw
2007-07-28T16:49:34 <johill> so the current activation transformer can keep track of the new placeholder nodes
2007-07-28T16:51:05 <xorAxAx> i think those separate nodes sound a bit artificial
2007-07-28T16:51:12 <xorAxAx> how about a change listener list per node?
2007-07-28T16:52:30 <johill> h
2007-07-28T16:52:32 <johill> hm
2007-07-28T16:53:26 <johill> well
2007-07-28T16:53:30 <johill> thing is
2007-07-28T16:53:47 <johill> it's a bit more complicated anyway
2007-07-28T16:54:00 <johill> when does the activation transform finish?
2007-07-28T16:55:06 <xorAxAx> no idea, my idea was a fix point approach :)
2007-07-28T16:55:13 <xorAxAx> that one works easily :)
2007-07-28T16:55:31 <johill> no
2007-07-28T16:55:40 <xorAxAx> why?
2007-07-28T16:55:50 <johill> because you can have nodes that depend on IDs being assigned
2007-07-28T16:56:01 <johill> but you can't assign IDs until you have activated most placeholders
2007-07-28T16:56:14 <johill> so now you have to do both in one step or something
2007-07-28T16:56:15 <johill> ugh
2007-07-28T16:56:31 <xorAxAx> thats a circular dependency
2007-07-28T16:57:32 <xorAxAx> you have to break it by introducing the notion of flags like "changes the structure" and "depends on the structure" and structure would mean "nodes who are an instance of one of these classes"
2007-07-28T16:58:07 <xorAxAx> then you can introduce an ordering that determines which nodes to transform in which step
2007-07-28T16:58:26 <johill> no
2007-07-28T16:59:04 <johill> if you put the expand functionality into the placeholder node
2007-07-28T16:59:58 <johill> and have dependencies
2007-07-28T17:00:14 <xorAxAx> dependencies?
2007-07-28T17:00:29 <johill> then the id assigner can just walk through and assign ids and thereby finalise nodes
2007-07-28T17:01:07 <johill> all placeholders that depend on a node being finalised will be expanded as part of the finalisation
2007-07-28T17:01:28 <johill> (calling the transform's callback about new nodes)
2007-07-28T17:01:38 <xorAxAx> finalisation?
2007-07-28T17:02:01 <xorAxAx> well, put it onto the wiki page, i will think about it
2007-07-28T17:03:34 <johill> done
2007-07-28T17:05:54 <xorAxAx> "Each placeholder node may have a dependency on an unfinished node or another placeholder node. After an unfinished node is finalised or a placeholder node is activated, the dependent nodes will be activated." - what about circular dependencies?
2007-07-28T17:06:13 <xorAxAx> "This representation can be cached." - i suggested to cache it later
2007-07-28T17:06:26 <xorAxAx> otherwise it will be much slower than the current version
2007-07-28T17:06:53 <xorAxAx> can you amend the process section?
2007-07-28T17:07:14 <johill> reload and read the italic part
2007-07-28T17:07:37 <xorAxAx> ah :)
2007-07-28T17:08:00 <johill> I guess a node must be able to refuse activation
2007-07-28T17:08:15 <xorAxAx> why?
2007-07-28T17:08:35 <johill> say in a pre-cache activation scan for the [[date]] macro
2007-07-28T17:08:57 <xorAxAx> well, that will be determined by a flag
2007-07-28T17:09:04 <johill> ?
2007-07-28T17:09:06 <xorAxAx> or a dependency list like we have it now
2007-07-28T17:09:21 <xorAxAx> so the transformer can know if it is finished with the pre-caching transformations
2007-07-28T17:09:22 <johill> I'd rather actually call them
2007-07-28T17:09:31 <johill> because the date macro if given an actual time-date parameter can output things
2007-07-28T17:09:32 <xorAxAx> yes, i am thinking about a call here
2007-07-28T17:09:58 <xorAxAx> umm
2007-07-28T17:10:03 <xorAxAx> the date macro is always dynamic
2007-07-28T17:10:07 <xorAxAx> regardless of parameters
2007-07-28T17:10:11 <xorAxAx> think about time zones
2007-07-28T17:10:19 <johill> so imho we just activate it and it returns None (refused actiovation) or a list of things to replace it with
2007-07-28T17:10:33 <johill> well, yeah, whatever. think about a macro that isn't always dynamic
2007-07-28T17:10:40 <johill> examples. I'm not into those details yet ;)
2007-07-28T17:10:42 <xorAxAx> umm
2007-07-28T17:10:47 <xorAxAx> i dislike that approach
2007-07-28T17:11:01 <xorAxAx> a plugin should simply return nodes
2007-07-28T17:11:05 <lanius> ThomasWaldmann: can you explainme the attachdir part in caching.needsUpdate please
2007-07-28T17:11:12 <johill> xorAxAx: so it can return itself?
2007-07-28T17:11:13 <xorAxAx> and a macro may return a call node
2007-07-28T17:11:15 <xorAxAx> yes
2007-07-28T17:11:20 <xorAxAx> not itself
2007-07-28T17:11:26 <xorAxAx> another node with more information
2007-07-28T17:11:38 <xorAxAx> the flags etc.
2007-07-28T17:11:41 <johill> flags?
2007-07-28T17:11:48 <xorAxAx> 17:08:57 < xorAxAx> well, that will be determined by a flag
2007-07-28T17:11:53 <xorAxAx> 17:09:05 < xorAxAx> or a dependency list like we have it now
2007-07-28T17:12:04 <johill> but what sort of flags are you thinking about?
2007-07-28T17:12:20 <xorAxAx> depends_on_page_contents = True
2007-07-28T17:12:38 <johill> hm
2007-07-28T17:12:39 <johill> dislike
2007-07-28T17:12:45 <xorAxAx> is_cachable = True
2007-07-28T17:12:49 <johill> too hardcoded
2007-07-28T17:12:57 <xorAxAx> hardcoded?
2007-07-28T17:12:59 <xorAxAx> not at all
2007-07-28T17:13:19 <johill> flags shouldn't be necessary
2007-07-28T17:13:32 <johill> the node is activated and when it replaces itself with itself then fine, can't be cached
2007-07-28T17:13:38 <xorAxAx> *sigh*
2007-07-28T17:13:43 <ThomasWaldmann> lanius: page rendering depends on attachments
2007-07-28T17:13:46 <xorAxAx> how should you know if you should call it again?
2007-07-28T17:13:54 <xorAxAx> thats not a functional approach
2007-07-28T17:14:13 <xorAxAx> and how should the macro know whether it should return itself?
2007-07-28T17:14:28 <lanius> ThomasWaldmann: but why is it only used in one place
2007-07-28T17:14:48 <johill> say whether state is available or not
2007-07-28T17:14:51 <ThomasWaldmann> why not?
2007-07-28T17:15:04 <johill> when outputting we have 'user' state available, but when caching we don't
2007-07-28T17:15:10 <xorAxAx> johill: so you want to call it in every transformation?
2007-07-28T17:15:19 <xorAxAx> that sounds broken
2007-07-28T17:15:51 <johill> no, just for every activation scan
2007-07-28T17:16:01 <xorAxAx> hmm
2007-07-28T17:16:05 <johill> you have a pre-cache activation scan without state
2007-07-28T17:16:08 <johill> and a post-cache activation scan
2007-07-28T17:16:14 <johill> if it's still returning itself then that's a bug
2007-07-28T17:16:47 <johill> that's why I was thinking it may only return itself or new placeholder nodes, but those new placeholder nodes absolutely need dependency info
2007-07-28T17:17:13 <johill> but not sure if that's even necessary
2007-07-28T17:17:13 <xorAxAx> umm
2007-07-28T17:17:29 <johill> probably not
2007-07-28T17:17:42 <johill> so it just returns something
2007-07-28T17:17:54 <johill> and if it's not itself then it is activated in turn
2007-07-28T17:18:39 <xorAxAx> hmm, you could start by describing the transformation in pseudo code :)
2007-07-28T17:18:41 <johill> if (and only if) it returns something with dependency info, that will be activated after the dependency is satisfied
2007-07-28T17:22:12 <CIA-27> moin: Heinrich Wendel <h_wendel@cojobo.net> * 2723:10edc11c0681 1.7-storage-hwendel/MoinMoin/ (9 files in 6 dirs): remove more getPagePath
2007-07-28T17:22:41 <lanius> ThomasWaldmann: got it
2007-07-28T17:25:02 <johill> xorAxAx: ok there
2007-07-28T17:26:20 <xorAxAx> who builds dependent_nodes?
2007-07-28T17:26:39 <johill> oh if you create a new placeholder node you specify the dependency
2007-07-28T17:26:44 <johill> and then it gets added there
2007-07-28T17:27:05 <xorAxAx> lanius: this has to be replaced by item handling:
2007-07-28T17:27:06 <xorAxAx> + from MoinMoin.action.AttachFile import getAttachDir
2007-07-28T17:27:07 <xorAxAx> + attachmentsPath = getAttachDir(self.page_name)
2007-07-28T17:29:04 <lanius> xorAxAx: sure
2007-07-28T17:29:20 <lanius> xorAxAx: the whole attachement stuff has to be replaced by item handling
2007-07-28T17:29:41 <xorAxAx> yes
2007-07-28T17:29:58 <lanius> i'm just narrowing down the getPagePath stuff to different areas
2007-07-28T17:30:02 <xorAxAx> i think the next step of getpagepath replacement is caching.py and then packages.py
2007-07-28T17:30:12 <lanius> left is attachement stuff, edit-log, caching, packages and wikisysnc
2007-07-28T17:30:17 <xorAxAx> packages.py is very easy
2007-07-28T17:30:29 <lanius> is it?
2007-07-28T17:30:34 <xorAxAx> yes, see the code :)
2007-07-28T17:30:50 <lanius> i looked at it but refused to understand it :D
2007-07-28T17:31:01 <xorAxAx> have you read the help page?
2007-07-28T17:31:04 <lanius> no
2007-07-28T17:31:06 <xorAxAx> it explains how it works
2007-07-28T17:31:09 <xorAxAx> HelpOnPackages
2007-07-28T17:31:11 <lanius> sounds good
2007-07-28T17:31:26 <xorAxAx> (from a power user/plugin writer point of view)
2007-07-28T17:31:35 <lanius> that's ok for me :D
2007-07-28T17:32:45 <xorAxAx> and look for 2.0
2007-07-28T17:32:52 <johill> I think oyu mean HelpOnPackageInstaller
2007-07-28T17:32:55 <xorAxAx> yes
2007-07-28T17:33:04 <xorAxAx> @param pagename: Page where the file is attached. Or in 2.0, the file itself.
2007-07-28T17:33:07 <xorAxAx> e.g.
2007-07-28T17:33:21 <xorAxAx> it means that its a trivial function in your branch
2007-07-28T17:33:25 <xorAxAx> which is meant by 2.0
2007-07-28T17:33:44 <lanius> what?
2007-07-28T17:33:50 <xorAxAx> hmm?
2007-07-28T17:34:03 <lanius> don't understand what you wanna say me with this 2.0 stuff
2007-07-28T17:34:05 <xorAxAx> if you see "2.0" somewhere, it means you
2007-07-28T17:34:15 <xorAxAx> and your branch
2007-07-28T17:34:27 <lanius> in the code in the docs?
2007-07-28T17:34:34 <xorAxAx> in the docstrings
2007-07-28T17:34:37 <lanius> ah
2007-07-28T17:34:45 <xorAxAx> btw, the attachment operations are a bit illdefined
2007-07-28T17:34:56 <xorAxAx> i would concatenate the page name and the attachment name and fix the doc string
2007-07-28T17:36:01 <xorAxAx> (reimar/thomas added them)
2007-07-28T17:37:46 <lanius> ok will have a look at this
2007-07-28T17:37:51 <lanius> gotta go now
2007-07-28T17:43:46 <ThomasWaldmann> lanius: -from MoinMoin.parser.text_moin_wiki import Parser
2007-07-28T17:43:46 <ThomasWaldmann> +from MoinMoin.parser.text import Parser
2007-07-28T17:43:47 <ThomasWaldmann> ?
2007-07-28T17:50:22 <johill> xorAxAx: check again
2007-07-28T17:51:29 <xorAxAx> still, who is instantiating the placeholder node?
2007-07-28T17:52:40 <johill> first the parser
2007-07-28T17:54:01 <xorAxAx> the parser? how does it know the dependencies of an opaque callback node?
2007-07-28T17:54:22 <johill> it has none
2007-07-28T17:54:28 <xorAxAx> ?
2007-07-28T17:54:34 <xorAxAx> ok
2007-07-28T17:54:38 <xorAxAx> then how are they added?
2007-07-28T17:55:18 <johill> treeactivator::run
2007-07-28T17:55:23 <johill> it calls it
2007-07-28T17:55:50 <johill> then the macro may walk the subdocument it's given and return a replacement list
2007-07-28T17:55:59 <johill> that replacement list may contain replacement nodes with dependencies
2007-07-28T17:56:10 <ThomasWaldmann> lanius: i am not convinced about moving unquoting from fs code to generic code, because each backend can have different quoting requirements
2007-07-28T17:56:53 <xorAxAx> johill: where is that list in the code?
2007-07-28T17:57:35 <johill> result = self.activation_function(self, dependency, dependency_replacements)
2007-07-28T17:57:38 <johill> it's the result there
2007-07-28T17:59:25 <xorAxAx> ok
2007-07-28T18:10:38 <johill> something like that is better
2007-07-28T18:21:40 <johill> yeah, I think this should worknow
2007-07-28T18:23:41 <johill> what do you think?
2007-07-28T18:25:19 <xorAxAx> no idea
2007-07-28T18:27:50 * johill adds how the toc works
2007-07-28T19:35:22 <johill> xorAxAx: works
2007-07-28T19:35:32 <johill> check out http://git.sipsolutions.net/dom.git/
2007-07-28T19:36:32 <xorAxAx> git!??!
2007-07-28T19:37:03 <xorAxAx> no hgwebdir running? :)
2007-07-28T19:37:04 <grzywacz> ;-)
2007-07-28T19:37:18 <johill> no, why bother? git's fine for all I need :)
2007-07-28T19:37:18 <ThomasWaldmann> i-git :)
2007-07-28T19:37:22 <johill> heh
2007-07-28T19:37:36 <johill> and it can share objects! :P
2007-07-28T19:37:45 <johill> dinner
2007-07-28T19:38:45 <johill> just get a tarball
2007-07-28T19:38:52 <johill> click on a tree and then snapshot
2007-07-28T21:45:19 <johill> no comments? oh well :)
2007-07-28T21:45:57 <xorAxAx> johill: you know ... exams
2007-07-28T21:46:24 <johill> yeah
2007-07-28T21:46:34 <johill> anyway, I'm off for the weekend. later
2007-07-28T21:46:51 <xorAxAx> have fun MoinMoin: MoinMoinChat/Logs/moin-dev/2007-07-28 (last edited 2007-10-29 19:20:39 by localhost)
