MoinMoin: MoinMoinChat/Logs/moin-dev/2007-05-13

2007-05-13T05:07:54  *** irc.freenode.net sets mode: +o ChanServ
2007-05-13T10:14:37  <johill> uh, that html bug I found is really tough to fix
2007-05-13T10:14:50  <johill> apparently a similar bug was there before even without the Verbatim stuff
2007-05-13T10:14:59  <johill> and was fixed with a bunch of lookahead assertions
2007-05-13T10:15:07  <johill> which isn't right since anything might be in some macro...
2007-05-13T11:21:50  <ThomasWaldmann> moin
2007-05-13T11:40:31  <lanius> helo
2007-05-13T11:42:16  <xorAxAx> hi lanius
2007-05-13T11:45:25  <lanius> xorAxAx: edit-log is not saved directly but can be recontructed from the metadata of the revisions
2007-05-13T11:46:03  <xorAxAx> lanius: thats insane and not bearable :)
2007-05-13T11:46:20  <lanius> insane sounds good
2007-05-13T11:46:23  <xorAxAx> lanius: because its O(n) :)
2007-05-13T11:47:11  <lanius> then a layer above must cache it
2007-05-13T11:48:08  <ThomasWaldmann> xorAxAx: of course it would be cached
2007-05-13T11:48:37  <ThomasWaldmann> that would make it faster and more solid
2007-05-13T11:50:24  <xorAxAx> ThomasWaldmann: how do you invalidate it?
2007-05-13T11:52:21  <lanius> i think it should be cached on page.py level and this level knows when the page changes
2007-05-13T11:52:31  <ThomasWaldmann> i mean a disk cache using caching module
2007-05-13T11:53:15  <xorAxAx> ThomasWaldmann: sure, but how is the data invalidated?
2007-05-13T11:53:29  <ThomasWaldmann> it is not invalidated, but kept valid
2007-05-13T11:53:54  <xorAxAx> IMHO this is a layerin violation :)
2007-05-13T11:54:09  <lanius> why
2007-05-13T11:54:12  <xorAxAx> because the upper levels dont know anything about history besides the list of revisions
2007-05-13T11:54:21  <xorAxAx> and now you have to map that into some internal cache
2007-05-13T11:54:30  <lanius> no the other way around
2007-05-13T11:54:39  <ThomasWaldmann> and it would contain much less information than data/edit-log has now
2007-05-13T11:54:43  <ThomasWaldmann> just pointers
2007-05-13T11:54:44  <lanius> the lower level does not know anyhitng about history, it only knows about data and metadata
2007-05-13T11:55:17  <xorAxAx> lanius: sure, it knows about an ordered sequence of revisions, thats a lot of history
2007-05-13T11:55:51  <lanius> but that's all, and that's only a small part of the edit-log
2007-05-13T11:56:05  <xorAxAx> hmm, ok
2007-05-13T11:56:32  <xorAxAx> so you want to keep caching files on the drive, still keeping up the necessarity for fs storage in any case?
2007-05-13T11:57:03  <lanius> it's not a full fs storage, but a simple cache
2007-05-13T11:57:20  <xorAxAx> its essential
2007-05-13T11:57:27  <xorAxAx> not the primary source of data, though
2007-05-13T11:57:31  <lanius> yes
2007-05-13T11:57:45  <xorAxAx> well
2007-05-13T11:57:54  <xorAxAx> but now this is the invalidaton issue:
2007-05-13T11:58:13  <xorAxAx> if you have just one backend but multiple cache dirs, how do you invalidate the "cache" in the other dir?
2007-05-13T11:58:30  <lanius> why would you want multiple cache dirs
2007-05-13T11:58:31  <xorAxAx> it doesnt make sense to introduce the restriction of having a global cache dir
2007-05-13T11:58:41  <xorAxAx> because you may want to have a cluster setup?
2007-05-13T11:58:50  <xorAxAx> thats why people need db storage
2007-05-13T11:58:57  <xorAxAx> because its easy to replicate
2007-05-13T11:59:06  <xorAxAx> IMHO this is a serious flaw
2007-05-13T11:59:14  <lanius> you can check the last modified metadata of the revision against the date of the cache file
2007-05-13T11:59:17  <xorAxAx> (and would be avoided by mapping "caching" to the backend)
2007-05-13T11:59:27  <xorAxAx> lanius: which revision?
2007-05-13T11:59:31  <lanius> the last
2007-05-13T11:59:56  <xorAxAx> of which item?
2007-05-13T12:00:02  <xorAxAx> there are n items ...
2007-05-13T12:00:04  <lanius> of the one you wan't the edit-log from
2007-05-13T12:00:17  <xorAxAx> we are talking about the global one
2007-05-13T12:00:28  <lanius> which global
2007-05-13T12:00:57  <lanius> every item in the cache must be allocated to one storage item
2007-05-13T12:01:07  <xorAxAx> the one that is used for RecentChanges
2007-05-13T12:01:11  <xorAxAx> the most important one
2007-05-13T12:01:24  <xorAxAx> the one that is more often fetched than any other edit-log
2007-05-13T12:01:33  <lanius> aah, i see
2007-05-13T12:04:21  <lanius> what is your suggestion
2007-05-13T12:04:51  <xorAxAx> well, if one accepts the above described issue, the solution would be to map all these caches to the backend
2007-05-13T12:08:02  <lanius> there are two possibilities to do that, map only the cache or map the functionality of the edit-log to the backend
2007-05-13T12:08:40  <lanius> where the second one is a layering violation again, so i prefer the first one
2007-05-13T12:14:39  <ThomasWaldmann> if we like to use the normal item backend for caching, we need some possibility to switch off revisioning
2007-05-13T12:14:54  <ThomasWaldmann> (btw, that is needed for misc other applications, too)
2007-05-13T12:15:09  <lanius> i think the backend must provide additional methods for caching
2007-05-13T12:15:20  <lanius> and revisioning isn't done on the backend level but on the item level
2007-05-13T12:16:01  <ThomasWaldmann> ok, then we still need items without revisioning
2007-05-13T12:16:21  <lanius> that's possible since the new revisions are created by page.py
2007-05-13T12:20:02  <lanius> xorAxAx: any comments ;)
2007-05-13T12:20:12  <ThomasWaldmann> Item.__getitem__ needs some fixes
2007-05-13T12:20:34  <lanius> yes?
2007-05-13T12:21:36  <ThomasWaldmann> s/name/rev/, s/=/==/, currev?
2007-05-13T12:22:08  <ThomasWaldmann> (or revno)
2007-05-13T12:22:32  <lanius> yes
2007-05-13T12:22:37  <lanius> done
2007-05-13T12:24:02  <ThomasWaldmann> no
2007-05-13T12:24:30  <ThomasWaldmann> the last 2 items are still wrong/unclear
2007-05-13T12:25:06  <lanius> the last two?
2007-05-13T12:25:15  <ThomasWaldmann> and PEP8 tells not to but spaces around = for named arguments
2007-05-13T12:25:26  <ThomasWaldmann> s/but/put/ :D
2007-05-13T12:27:47  <ThomasWaldmann> s/realize/implement/
2007-05-13T12:27:47  <lanius> now satisfied?
2007-05-13T12:28:23  <xorAxAx> lanius: yeah, it makes sense to have a different API to some kind in order to not mix namespaces and keep locking requirements
2007-05-13T12:29:29  <lanius> xorAxAx: this api would also satisfie your wiki-wide cache question i guess
2007-05-13T12:29:33  <ThomasWaldmann> lanius: much better :)
2007-05-13T12:32:39  <lanius> xorAxAx: what is wiki-wide meta data
2007-05-13T12:35:15  <xorAxAx> lanius: well, look into a moin data directory :)
2007-05-13T12:36:41  <xorAxAx> we have sister site caches, i18n caches, the IWID, chart caches, drafts, surgeprotection status, user lookup caches, wikidict caches
2007-05-13T12:36:51  <xorAxAx> not all of those but most depend on actual data
2007-05-13T12:38:10  <lanius> sounds like a lot
2007-05-13T12:38:15  <xorAxAx> yes :)
2007-05-13T12:38:25  <lanius> so what do you want to say with that
2007-05-13T12:38:47  <xorAxAx> it has to be checked what is saved howm when and why
2007-05-13T12:38:52  <xorAxAx> s/wm/w,/
2007-05-13T12:39:09  <lanius> but this must not be checked by the storage layer
2007-05-13T12:39:14  <lanius> but by the coresponding layers
2007-05-13T12:39:32  <xorAxAx> why? we have the same problems of invalidation here
2007-05-13T12:39:46  <xorAxAx> so you say that everything goes to the backend via the caching api? ok
2007-05-13T12:40:26  <lanius> you can't put everything into the storage layer
2007-05-13T12:44:02  * ThomasWaldmann fixed some more stuff
2007-05-13T12:44:22  <lanius> xorAxAx: the wiki wide metadata seems more like configuration stuff
2007-05-13T12:45:18  <lanius> :-)
2007-05-13T12:49:38  <ThomasWaldmann> btw, that caching stuff could be done as a second step
2007-05-13T12:50:30  <xorAxAx> ThomasWaldmann: well, the design must be sound
2007-05-13T12:50:34  <ThomasWaldmann> same for the edit-log
2007-05-13T12:50:43  <ThomasWaldmann> sure
2007-05-13T12:50:51  <xorAxAx> lanius: not really
2007-05-13T12:51:46  <lanius> :q
2007-05-13T12:52:29  <lanius> xorAxAx: then be more detailed please
2007-05-13T12:52:52  <xorAxAx> well, which of the above items is unclear?
2007-05-13T12:54:14  <lanius> what metadata do you mean
2007-05-13T12:54:47  <xorAxAx> 12:36:41 < xorAxAx> we have sister site caches, i18n caches, the IWID, chart caches, drafts, surgeprotection status, user lookup caches, wikidict caches
2007-05-13T12:55:28  <xorAxAx> everything that does not say "cache" is meta-data is suppose :)
2007-05-13T12:55:29  <lanius> that is caches not metadata?
2007-05-13T12:55:33  <lanius> ok
2007-05-13T12:55:45  <xorAxAx> even though its handled by the caching system currently
2007-05-13T12:55:46  <lanius> drafts and sugeprotection status and iwid
2007-05-13T12:55:57  <xorAxAx> except for the iwid
2007-05-13T12:55:57  <lanius> whatever this three things are
2007-05-13T12:56:09  <xorAxAx> iwid is a unique identifier for the wiki instance
2007-05-13T12:56:16  <lanius> configuration
2007-05-13T12:56:19  <xorAxAx> nope
2007-05-13T12:56:21  <lanius> why not
2007-05-13T12:56:24  <xorAxAx> it can and may not be configured
2007-05-13T12:56:29  <xorAxAx> because it is internal
2007-05-13T12:56:37  <xorAxAx> the interwiki name can be configured
2007-05-13T12:56:43  <xorAxAx> unfortunately, it is not unique
2007-05-13T12:57:21  <lanius> url of the wiki is uniqu
2007-05-13T12:57:23  <lanius> e
2007-05-13T12:59:04  <xorAxAx> nope
2007-05-13T13:00:43  <lanius> never have seen two wiki with the same url
2007-05-13T13:02:40  <xorAxAx> but a wiki might have multiple URLs
2007-05-13T13:02:47  <xorAxAx> and you may want to change the URL
2007-05-13T13:02:59  <xorAxAx> snibril mitigated resulting issues by using purls
2007-05-13T13:04:29  <lanius> how is the iwid generated
2007-05-13T13:07:03  <xorAxAx> timestamp, some hashes etc .
2007-05-13T13:07:08  <xorAxAx> its basically a hash
2007-05-13T13:07:32  <ThomasWaldmann> lanius: http://localhost:8080/ :)
2007-05-13T13:07:49  <lanius> localhost doesn't count
2007-05-13T13:08:10  <lanius> what is surgeprotection status
2007-05-13T13:08:36  <ThomasWaldmann> a list of misbehaving clients
2007-05-13T13:08:58  <lanius> and where are drafts used
2007-05-13T13:09:14  <xorAxAx> sure does it count, lanius
2007-05-13T13:09:21  <xorAxAx> pretty much even
2007-05-13T13:09:24  <xorAxAx> think about MMDE
2007-05-13T13:09:25  <lanius> no sine localhost will never be reachable from the www
2007-05-13T13:09:27  <ThomasWaldmann> drafts are the 1.6 replacement for /MoinEditorBackup pages
2007-05-13T13:09:39  <xorAxAx> lanius: thats irrelevant, wikisync works in both directions
2007-05-13T13:09:49  <xorAxAx> and the IWID is essential for this
2007-05-13T13:10:02  <xorAxAx> ThomasWaldmann: but why are they in the cache folder?
2007-05-13T13:10:48  <lanius> ok three posibilities for wiki metadata
2007-05-13T13:10:54  <lanius> first: save it in the cache again
2007-05-13T13:11:03  <xorAxAx> it may not go into the cache
2007-05-13T13:11:12  <lanius> second: specify a page, e.g. "/" where this metadata will be saved
2007-05-13T13:11:14  <xorAxAx> because it is primary metadata (at least the iwid)
2007-05-13T13:11:19  <lanius> third: add special methods to the backend
2007-05-13T13:11:26  <xorAxAx> second sounds nice
2007-05-13T13:11:35  <xorAxAx> and i think drafts should go somewhere else
2007-05-13T13:11:41  <xorAxAx> but no idea where
2007-05-13T13:12:12  <lanius> haven't understood the concept yet
2007-05-13T13:13:00  <ThomasWaldmann> xorAxAx: the drafts are only temporarily interesting and it is good not to see them in the page namespace
2007-05-13T13:13:13  <lanius> ok
2007-05-13T13:13:16  <xorAxAx> ThomasWaldmann: they could go into the page metadata
2007-05-13T13:13:29  <xorAxAx> moin saves the current draft while you are editing and pressing cancel/preview, lanius
2007-05-13T13:13:36  <lanius> item-caches like the pagelinks should be stored in the metadata and generated by page.py
2007-05-13T13:13:39  <xorAxAx> ThomasWaldmann: usually you want a draft per page, not one per user
2007-05-13T13:14:28  <ThomasWaldmann> xorAxAx: so you want to add 130kb to metadata if someone edits MoinMoinQuestions?
2007-05-13T13:14:37  <xorAxAx> ThomasWaldmann: no, not metadata
2007-05-13T13:15:21  <ThomasWaldmann> ?
2007-05-13T13:16:08  <xorAxAx> who is talking about metadata? its not metadata :)
2007-05-13T13:18:13  <ThomasWaldmann> (13:13) <   xorAxAx> ThomasWaldmann: they could go into the page metadata
2007-05-13T13:18:27  <xorAxAx> hmm, ok :-)
2007-05-13T13:18:53  <xorAxAx> well, they could go "with the page"
2007-05-13T13:18:57  <ThomasWaldmann> maybe it was your other self :)
2007-05-13T13:19:30  <xorAxAx> yes, the one that advises me to write rubrik cube dict data structures
2007-05-13T13:19:39  <ThomasWaldmann> i see no reason why not to store them with caching
2007-05-13T13:19:47  <ThomasWaldmann> it is simply and it works
2007-05-13T13:19:49  <ThomasWaldmann> e
2007-05-13T13:20:01  <xorAxAx> the editor backup page had revisions, right?
2007-05-13T13:20:09  <ThomasWaldmann> no
2007-05-13T13:20:12  <xorAxAx> hmm, ok
2007-05-13T13:20:25  <ThomasWaldmann> and it needed special treatment in pagelist
2007-05-13T13:23:44  <ThomasWaldmann> lanius: you dont have a wiki homepage yet
2007-05-13T13:26:46  <lanius> yes, important things first ;)
2007-05-13T13:27:00  <lanius> item-caches like the pagelinks should be stored in the metadata and generated by page.py
2007-05-13T13:28:51  <xorAxAx> you already said that, yes
2007-05-13T13:29:27  <lanius> i repeat it until i get the feeling that you read it
2007-05-13T13:29:33  <xorAxAx> :-)
2007-05-13T13:29:59  <xorAxAx> hmm, i forgot - do you want to make single metadata keys queryable in the backend or all at once?
2007-05-13T13:30:00  <lanius> and user profiles are stored as metadata of the user pages
2007-05-13T13:30:11  <lanius> single
2007-05-13T13:30:16  <xorAxAx> ok
2007-05-13T13:30:35  <lanius> when they are migrated to the new storage
2007-05-13T13:31:04  <xorAxAx> well, they have to appear like that even with the old storage - but only read-only
2007-05-13T13:31:20  <xorAxAx> that should not very difficult if you share the metadata keys on every level, though
2007-05-13T13:32:26  <lanius> ok, we didn't finish the locking discussion last time
2007-05-13T13:32:34  <xorAxAx> yes
2007-05-13T13:32:37  <ThomasWaldmann> lanius: "user pages" == "user's wiki homepage"?
2007-05-13T13:32:46  <lanius> why not
2007-05-13T13:32:57  <ThomasWaldmann> you dont have one :)
2007-05-13T13:33:28  <lanius> but you could automatically create one for new users
2007-05-13T13:33:50  <ThomasWaldmann> and if your wiki has 10000 users, you dont want to have all those pages
2007-05-13T13:34:01  <xorAxAx> yeah, many users are only lurking
2007-05-13T13:34:15  <ThomasWaldmann> this is why user_homewiki exists
2007-05-13T13:34:46  <xorAxAx> but maybe this is a behavourial question - how about writing the user settings to a deleted revision's metadata?
2007-05-13T13:34:51  <ThomasWaldmann> otoh, that could be used with the new old interwiki auth :)
2007-05-13T13:34:54  <lanius> then they will just have mimetype=User and the pagelist does not show it and
2007-05-13T13:35:13  <xorAxAx> lanius: but then they must have a different name from the real user home pages
2007-05-13T13:35:16  <lanius> but this is a question that is independent from the storage, since the storage supports whatever you want here
2007-05-13T13:36:41  <xorAxAx> its independent from the backend, but not the whole storage subsystem
2007-05-13T13:36:51  <xorAxAx> if you think it as the M in MVC :)
2007-05-13T13:37:17  <lanius> it's not indepentent, but i think the current design supports whatever you want there
2007-05-13T13:38:06  <xorAxAx> the point is that it has to be fixed in the context of designing the M
2007-05-13T13:38:14  <xorAxAx> but possibly later than earlier, yes
2007-05-13T13:39:31  <ThomasWaldmann> the user/acl thing is not in the pseudocode yet
2007-05-13T13:40:19  <lanius> no it is only in the description
2007-05-13T13:40:41  <lanius> so where do get the user from
2007-05-13T13:41:03  <lanius> if it is not always request
2007-05-13T13:42:57  <ThomasWaldmann> as a named parameter with default None
2007-05-13T13:43:04  <lanius> where
2007-05-13T13:43:23  <xorAxAx> lanius: you need an itemcollection facada
2007-05-13T13:43:24  <xorAxAx> facade
2007-05-13T13:43:45  <ThomasWaldmann> that has still to be defined :) maybe same place you want to check the acls.
2007-05-13T13:43:57  <lanius> itemcollection facade?
2007-05-13T13:44:00  <xorAxAx> lanius: ye
2007-05-13T13:44:01  <xorAxAx> s
2007-05-13T13:44:08  <lanius> how would it work
2007-05-13T13:44:14  <lanius> ThomasWaldmann: no, somewhere more global
2007-05-13T13:45:06  <xorAxAx> lanius: request.storage["FrontPage"].author would access the attrib request.storage which is a facade. it proxies to the instance wiki itemcollection. it is instantiated on every request to add the current user as an instance attribute to the facade
2007-05-13T13:45:32  <xorAxAx> thats like the currying lambda which is currently used for may
2007-05-13T13:45:47  <xorAxAx> but as we have multiple methods in the "itemcollection", this is a simpler approach
2007-05-13T13:48:32  <lanius> so the user will be in the init of itemcollection
2007-05-13T13:48:37  <xorAxAx> no
2007-05-13T13:48:41  <xorAxAx> umm
2007-05-13T13:48:51  <xorAxAx> no, there should be only one itemcollection per instance
2007-05-13T13:48:56  <xorAxAx> wiki instance
2007-05-13T13:49:16  <xorAxAx> because it might need to do some init works
2007-05-13T13:49:46  <lanius> yes it must
2007-05-13T13:50:09  <xorAxAx> the user or request would be passed for every method
2007-05-13T13:50:22  <lanius> every method of item or every method of itemcollection
2007-05-13T13:52:30  <ThomasWaldmann> bbl
2007-05-13T13:53:41  <lanius> ?
2007-05-13T13:53:48  <xorAxAx> lanius: of item collection
2007-05-13T13:54:09  <xorAxAx> and items could be per-request singletons
2007-05-13T13:54:26  <lanius> ok
2007-05-13T13:54:36  <lanius> sooo lockin
2007-05-13T13:54:37  <lanius> g
2007-05-13T13:56:22  <lanius> we didn't agree on a solutoin yet
2007-05-13T13:56:33  <xorAxAx> there are different requirements
2007-05-13T13:57:03  <xorAxAx> first, the atomic modification of certain values like the data file or metadata keys
2007-05-13T13:57:22  <xorAxAx> then, transactional locks that lock page modifications or metadata modifications while metadata is modified
2007-05-13T13:57:47  <xorAxAx> the first ones would need to be realised on the backend level
2007-05-13T13:58:02  <xorAxAx> mainly for metadata in case of file backends
2007-05-13T13:59:17  <xorAxAx> currently its used so that multiple threads/processes dont corrupt files because of concurrent modification
2007-05-13T13:59:27  <xorAxAx> the second locks and their reasons are more important
2007-05-13T13:59:36  <xorAxAx> in terms of design
2007-05-13T14:00:29  <lanius> how are conflicts handled
2007-05-13T14:00:43  <xorAxAx> there are no conflicts because there are locks
2007-05-13T14:00:48  <xorAxAx> which conflicts do you mean? :)
2007-05-13T14:01:10  <xorAxAx> page editing locks?
2007-05-13T14:01:17  <xorAxAx> they are on a wholly different level
2007-05-13T14:01:31  <xorAxAx> those are the only locks that a user can see
2007-05-13T14:01:52  <lanius> but how does user a notice that user b locked a page
2007-05-13T14:02:16  <xorAxAx> thats done via editlocks which is page metadata
2007-05-13T14:02:30  <lanius> but both already requested an item
2007-05-13T14:02:35  <xorAxAx> thats a few levels above
2007-05-13T14:02:38  <xorAxAx> "requested"?
2007-05-13T14:02:44  <lanius> from the itemcollection
2007-05-13T14:02:52  <xorAxAx> thats a different level
2007-05-13T14:02:59  <xorAxAx> about which level are we talking right now? :)
2007-05-13T14:03:18  <lanius> they have to work together, don't they
2007-05-13T14:03:18  <xorAxAx> lets talk about requirements:
2007-05-13T14:03:37  <xorAxAx> a) the user wants to see that somebody else is editing a page (editlocks)
2007-05-13T14:03:58  <xorAxAx> b) atomic file modifications must occur without corruptions in case of concurrent write accesses
2007-05-13T14:04:18  <xorAxAx> c) transactional file modifications must occur without losing any data
2007-05-13T14:04:27  <lanius> c?
2007-05-13T14:04:30  <xorAxAx> which requirement do you want to talk about? :)
2007-05-13T14:04:46  <xorAxAx> do you wonder what it is or do you want to talk about it? :)
2007-05-13T14:04:55  <lanius> wonder
2007-05-13T14:05:29  <xorAxAx> nowadays - metadata is append only
2007-05-13T14:05:40  <xorAxAx> but in your model - the following sequence is perfectly valid:
2007-05-13T14:06:06  <lanius> your model sounds very negative
2007-05-13T14:06:12  <xorAxAx> sorry :-)
2007-05-13T14:06:22  <xorAxAx> there are threads A and B, A fetchs an item with its metadata, B as well, A modifies the metadata, B as well, A commits, B as well, B overwrites A
2007-05-13T14:06:29  <xorAxAx> its our completly broken model
2007-05-13T14:06:31  <xorAxAx> :-)
2007-05-13T14:06:50  <lanius> yes that what i meant with conflicts
2007-05-13T14:06:58  <xorAxAx> johill suggested to bring some immutablitly aspects into the API
2007-05-13T14:07:20  <xorAxAx> so that it recommends "do not modify this as long as you do not have a wiki-wide lock" or something like that
2007-05-13T14:07:50  <xorAxAx> so that the "conversion" requirement can be solved but in the average case, nothing is affected by this flexibilty
2007-05-13T14:08:14  <xorAxAx> in fact, real metadata of a revisions selmdoly changes
2007-05-13T14:08:29  <lanius> wiki-wide lock? a lock won't help here, you won't notice the changes
2007-05-13T14:08:40  <xorAxAx> "the changes"?
2007-05-13T14:08:52  <lanius> the changes a commts and b overwrites
2007-05-13T14:09:13  <xorAxAx> well, that would not happen
2007-05-13T14:09:22  <xorAxAx> the idea is that modifications are reduced to a minimum
2007-05-13T14:09:34  <xorAxAx> currently, real metadata is read-only
2007-05-13T14:09:41  <xorAxAx> only caches are readable
2007-05-13T14:09:46  <xorAxAx> (and there are a few exceptions)
2007-05-13T14:09:55  <lanius> so the lock would be made when requeting the item at the collection
2007-05-13T14:10:04  <lanius> and the second one would only get a readable item
2007-05-13T14:10:06  <lanius> ?
2007-05-13T14:10:15  <xorAxAx> hmm
2007-05-13T14:10:28  <lanius> otherwise it won't work
2007-05-13T14:10:34  <xorAxAx> that would be very slow for a few number of use cases
2007-05-13T14:10:45  <xorAxAx> well, there are different hazards
2007-05-13T14:10:53  <xorAxAx> often, you dont need read and write locks
2007-05-13T14:11:12  <xorAxAx> because you dont mind the seldom issue of incoherent metadata
2007-05-13T14:11:23  <lanius> i don't understand how your use case can be fixed otherwise
2007-05-13T14:11:43  <lanius> and isn't that the same as a.)
2007-05-13T14:11:46  <xorAxAx> well, you would only take the lock if you plan to modify the metadata
2007-05-13T14:11:57  <xorAxAx> no, a) is completly different
2007-05-13T14:12:06  <lanius> ok, take the lock, modify it, save it, release the lock
2007-05-13T14:12:11  <xorAxAx> editlocks are softlocks, on saving the merge, the real merging happens
2007-05-13T14:12:21  <xorAxAx> lanius: moin allows concurrent edits
2007-05-13T14:12:25  <lanius> the other one takes the lock, modifies it and saves it, overwriteing the other changes
2007-05-13T14:12:39  <xorAxAx> no, that has been handled for decades in moin :)
2007-05-13T14:12:42  <xorAxAx> moin merges your changes
2007-05-13T14:12:54  <xorAxAx> s/the merge/the page/
2007-05-13T14:13:01  <lanius> that was your use case
2007-05-13T14:13:05  <xorAxAx> hmm?
2007-05-13T14:13:20  <lanius> i don't see the difference in your use case with and without locking
2007-05-13T14:13:23  <xorAxAx> i just wanted to point out that there are at least 3 different requirements to the overall system
2007-05-13T14:13:41  <xorAxAx> a) is completly unrelated to the lower level locking
2007-05-13T14:15:07  <lanius> but a must also use backend level locking code how else would the other thread notice that the first thread locked the item
2007-05-13T14:15:15  <lanius> soft-locked
2007-05-13T14:15:16  <xorAxAx> lanius: sure
2007-05-13T14:15:22  <xorAxAx> soft-locked?
2007-05-13T14:15:34  <xorAxAx> well, there are different layers and levels
2007-05-13T14:15:57  <xorAxAx> everything needs to be locked in the same realm as where it is located itself, yes
2007-05-13T14:16:13  <xorAxAx> but the best idea is to avoid locking
2007-05-13T14:16:14  <lanius> all locknig must be synchronized with the backend
2007-05-13T14:16:17  <xorAxAx> yes
2007-05-13T14:16:19  <lanius> if you do locking
2007-05-13T14:16:29  <xorAxAx> thats what it makes very slow
2007-05-13T14:16:35  <lanius> yes
2007-05-13T14:16:42  <xorAxAx> esp. if you plan to lock every item on a full text search e.g.
2007-05-13T14:16:47  <xorAxAx> not sensible
2007-05-13T14:16:47  <lanius> b.) is the easiest one, it can be implemented on backend level
2007-05-13T14:16:55  <lanius> each backend can take care of it
2007-05-13T14:17:05  <xorAxAx> yes
2007-05-13T14:17:10  <lanius> internally
2007-05-13T14:17:20  <xorAxAx> and its important that read locks arent necessary at most places
2007-05-13T14:17:29  <xorAxAx> so the backend shouldnt use read locks everywhere
2007-05-13T14:17:39  <xorAxAx> but when are they necessary?
2007-05-13T14:18:06  <lanius> they are not necessary on level b.) the read operation can just wait for the save to finish
2007-05-13T14:18:13  <xorAxAx> "wait"?
2007-05-13T14:18:28  <xorAxAx> in order to wait, you need a semaphore which needs locks
2007-05-13T14:18:39  <lanius> ok, then it's a global lock
2007-05-13T14:18:47  <xorAxAx> a global lock for what?
2007-05-13T14:18:48  <lanius> global item lock on backend level
2007-05-13T14:18:58  <lanius> you lock it, write, release
2007-05-13T14:19:00  <xorAxAx> for every read? as pointed out above, that wont scale
2007-05-13T14:19:04  <lanius> during the wrinting every other operation must wait
2007-05-13T14:19:05  <xorAxAx> i am talking about reading :)
2007-05-13T14:19:26  <lanius> that can be even a read operation, since you don't want to read half-written data
2007-05-13T14:19:35  <xorAxAx> how can they wait without struggling for the lock?
2007-05-13T14:19:48  <xorAxAx> see, so you want to lock for every read operation?
2007-05-13T14:19:49  <lanius> yes, they must struggle with it
2007-05-13T14:19:55  <lanius> it depends on the backend
2007-05-13T14:19:55  <xorAxAx> that would be very sloow
2007-05-13T14:20:06  <xorAxAx> well, it does not
2007-05-13T14:20:08  <lanius> if the backend does not support atomic operations what do you wan't to do
2007-05-13T14:20:24  <xorAxAx> hmm, atomic operations on which data?
2007-05-13T14:20:52  <lanius> on the data that is written
2007-05-13T14:21:58  <lanius> we agreed that b.) happens internally for each backend, so the backend must take care that there are is data inconsitency
2007-05-13T14:22:14  <lanius> s/are is/is no/
2007-05-13T14:22:51  <lanius> maybe you write your locking ideas down?
2007-05-13T14:23:21  <xorAxAx> well, the first question is - what will be concurrently modified
2007-05-13T14:23:26  <xorAxAx> then we think about what to lock
2007-05-13T14:23:37  <xorAxAx> can you make a list of possible scenarios?
2007-05-13T14:23:54  <lanius> i think you have a better overview over the scenarios
2007-05-13T14:23:57  <xorAxAx> e.g. we do not aim for concurrent modification of the data part
2007-05-13T14:24:00  <xorAxAx> hmm
2007-05-13T14:29:42  <xorAxAx> i added a short section
2007-05-13T14:29:46  <xorAxAx> feel free to extend :)
2007-05-13T14:32:04  <lanius> what aboure your a,b,c
2007-05-13T14:32:56  <xorAxAx> a is not relevant, thats "modify metadata"
2007-05-13T14:33:20  <xorAxAx> ok, b is missing
2007-05-13T14:35:20  <xorAxAx> added
2007-05-13T14:36:33  <lanius> if you say adding a revision needs a lock because of the global edit-log then creating an item needs this as well
2007-05-13T14:38:28  <xorAxAx> creating an item is not an operation that would be logged :)
2007-05-13T14:38:36  <xorAxAx> creating the first revision, is
2007-05-13T14:38:41  <lanius> ok
2007-05-13T14:39:09  <lanius> and modifying data needs no lock because of automatically merging which is not possible on metadata, right?
2007-05-13T14:39:30  <xorAxAx> noooo
2007-05-13T14:39:44  <xorAxAx> data modification is not possible without a global wiki lock
2007-05-13T14:39:55  <xorAxAx> lanius: the merged result goes into a new revision
2007-05-13T14:40:08  <xorAxAx> its handled on the controller level, not in the data model
2007-05-13T14:42:00  <lanius> then that is missing in the list
2007-05-13T14:43:30  <xorAxAx> no
2007-05-13T14:43:34  <xorAxAx> it is backend-agnostic
2007-05-13T14:43:51  <lanius> but merging is not done on backend level
2007-05-13T14:43:55  <xorAxAx> it would only go there to say that it is an issue that doesnt need to be thought about
2007-05-13T14:44:14  <xorAxAx> yes, thats why it is not directly relevant :)
2007-05-13T14:44:28  <lanius> so whats the difference between data and metadata locking
2007-05-13T14:45:56  <xorAxAx> data doesnt need to be locked
2007-05-13T14:46:14  <xorAxAx> but again, you may want to specify the level we are talking about
2007-05-13T14:46:38  <lanius> and why do we need to lock metadat athen
2007-05-13T14:47:56  <xorAxAx> because metadata may be modified
2007-05-13T14:48:02  <lanius> and data not?
2007-05-13T14:48:08  <xorAxAx> data is immutable, yes
2007-05-13T14:48:15  <lanius> since when
2007-05-13T14:48:20  <xorAxAx> didnt we say so?
2007-05-13T14:48:32  <lanius> i can't read anything about that in your locking comments
2007-05-13T14:48:43  <lanius> and how do you change data if it is immutable
2007-05-13T14:48:44  <xorAxAx> i will add it
2007-05-13T14:48:55  <lanius> and why is metadata not immutable then
2007-05-13T14:49:25  <xorAxAx> because metadata needs to be modified
2007-05-13T14:49:43  <lanius> and data not?
2007-05-13T14:49:43  <xorAxAx> some of it at least
2007-05-13T14:49:46  <xorAxAx> no, data not
2007-05-13T14:49:50  <lanius> i don't understand
2007-05-13T14:50:34  <xorAxAx> the idea is that data is immutable after a page has been saved
2007-05-13T14:50:38  <xorAxAx> umm, revision
2007-05-13T14:51:11  <lanius> admins must be able to change everything
2007-05-13T14:51:14  <xorAxAx> so data modification is synchronized by the same measures as revision creation
2007-05-13T14:51:25  <xorAxAx> lanius: see the page, that is done via a global lock
2007-05-13T14:51:51  <lanius> a global lock is a real performance killer, isn't it
2007-05-13T14:51:58  <xorAxAx> why?
2007-05-13T14:52:14  <lanius> because during the lock nobody can save anything
2007-05-13T14:52:27  <xorAxAx> well, the lock is not locked, its only checked for
2007-05-13T14:52:48  <lanius> do you need a lock then
2007-05-13T14:52:50  <xorAxAx> these soft semantics fulfill the requirements well
2007-05-13T14:53:21  <xorAxAx> indeed, it should not be necessary. if the admin should do those edits, he may want to take down the wiki for a short moment
2007-05-13T14:53:28  <xorAxAx> or just do them like he does now
2007-05-13T14:53:30  <lanius> can you try to write it down more detailed, comparing with how it is done currently?
2007-05-13T14:53:38  <xorAxAx> what exactly?
2007-05-13T14:53:45  <lanius> everything locking related
2007-05-13T14:53:53  <xorAxAx> hmm, ask your mentor :)
2007-05-13T14:54:01  <lanius> why
2007-05-13T14:55:03  <xorAxAx> because i dont like doing this task :)
2007-05-13T14:55:17  <xorAxAx> and i have other things on my todo list :)
2007-05-13T14:55:35  <lanius> but you have the idee what is needed
2007-05-13T14:55:46  <lanius> and i just don't understand it
2007-05-13T14:55:55  <xorAxAx> well, currently, data is immutable
2007-05-13T14:56:13  <xorAxAx> having mutable data would be a new requirement. and its not very sane to model the concurrent modification checks around it
2007-05-13T14:56:42  <lanius> i don't see a difference if metadata is immutable, should be the same then
2007-05-13T14:56:55  <xorAxAx> but metadata is not mutable currently
2007-05-13T14:57:01  <xorAxAx> umm, not immutable
2007-05-13T14:57:21  <lanius> i don't see a difference if metadata is mutable, should be the same then
2007-05-13T14:57:35  <lanius> you could apply the same locking strategy of the metadat to data then
2007-05-13T14:57:56  <xorAxAx> but the first goal is to avoid locking at all
2007-05-13T14:58:00  <xorAxAx> because of speed reasons
2007-05-13T14:58:09  <xorAxAx> but indeed, that was possible, yes
2007-05-13T15:06:48  <lanius> will think about it...
2007-05-13T15:42:04  <johill> hm, irssi highlighted here, but I don't know why :)
2007-05-13T15:42:16  <johill> did anyone want something from me?
2007-05-13T15:42:46  <xorAxAx> johill: not enough backlog?
2007-05-13T15:42:55  <xorAxAx> i said that you said that data immutability has advantages
2007-05-13T15:45:24  <johill> I didn't find it in the backlog ;)
2007-05-13T15:45:49  <xorAxAx> /lastlog johill
2007-05-13T15:45:59  <johill> hah
2007-05-13T15:46:00  <johill> :)
2007-05-13T15:46:20  <johill> yeah, I think data immutability would be good in places, especially if you were to say use git for revisioning the wiki
2007-05-13T15:47:03  <johill> s/use git/use a git-like storage system/
2007-05-13T15:48:43  <johill> anyway, I can't elaborate right now, I have to go. I can give it a bit of thought and maybe add some comments tomorrow
2007-05-13T16:19:09  <ThomasWaldmann> dreimark: I try to explain http://moinmoin.wikiwikiweb.de/MoinMoinBugs/SetupCheckFails, but I cant see why it should fail.
2007-05-13T16:26:57  <dreimark> the module is imported by e.g. text_html_text_moin_wiki.py and if the python installation on Suse is different then it fails
2007-05-13T16:33:19  <ThomasWaldmann> dreimark: but minidom should be there, so why does the import fail?
2007-05-13T16:34:30  <dreimark> because its not installed on Suse on default its an additional rpm package: python-xml
2007-05-13T16:34:49  <ThomasWaldmann> but it is part of stdlib, too
2007-05-13T16:41:08  <dreimark> it should be but they have definitly this python-xml package in 10.0. with /usr/lib64/python2.4/xml/dom/minidom.py and others
2007-05-13T16:45:38  <ThomasWaldmann> i am not interested in python-xml in that case because it works with it. i am interested why it doesnt work without.
2007-05-13T16:46:14  <ThomasWaldmann> and why it gives the import error when the minidom module is there.
2007-05-13T16:47:26  <dreimark> ok, I will check that monday or tuesday when I do install a new system
2007-05-13T17:49:25  <ThomasWaldmann> dreimark: no need to test. i looked at the suse rpms, they are simply incomplete.
2007-05-13T17:50:19  <dreimark> ThomasWaldmann: thats ugly
2007-05-13T19:05:52  <dreimark> bbl
2007-05-13T22:02:18  <neagulm> hello
2007-05-13T22:02:30  <xorAxAx> hi neagulm
2007-05-13T23:21:29  * ThomasWaldmann .oO(luckily my dsl still works, lightning strike <100m away)
2007-05-13T23:28:03  <starshine> owie.
2007-05-13T23:45:56  <neagulm> I have a 'small' question ? Lets say that I am a MoinMoin wiki administrator and I want to subscribe all users to a page (Eg. UsagePolicy) how could I achieve this ?
2007-05-13T23:46:02  <neagulm> the simplest way
2007-05-13T23:47:29  <xorAxAx> neagulm: there is a subscribeuser action. you would need to enter all usernames there
2007-05-13T23:54:48  <neagulm> xorAxAx, thank you