MoinMoin: MoinMoinChat/Logs/moin-dev/2007-06-08

2007-06-08T00:03:59  <ThomasWaldmann> sure, never mix
2007-06-08T00:04:05  <xorAxAx> zenhase: have you seen the redirection helpers in th request class?
2007-06-08T00:04:12  <xorAxAx> zenhase: you dont need to reinvent the wheel :)
2007-06-08T00:05:19  <ThomasWaldmann> zenhase: +        if hasattr(self.cfg, 'charset'): ?
2007-06-08T00:06:51  <zenhase> ThomasWaldmann: it wasn't there when i called ./moin cli show
2007-06-08T00:07:43  <zenhase> xorAxAx: *cough* thanks ... will look there
2007-06-08T00:11:04  <zenhase> hmm ... is it guaranteed, that request.redirect() does not get called between calls to theme.send_title() and theme.send_footer()?
2007-06-08T00:14:47  <xorAxAx> zenhase: its called very often
2007-06-08T00:14:52  <xorAxAx> in that timespan
2007-06-08T00:15:48  <zenhase> ah, i see that it works stack-like
2007-06-08T00:16:15  <zenhase> so it won't pose as a problem i suppose
2007-06-08T00:30:49  <zenhase> hmm, the redirection helpers are buggy/inconsistant it seems
2007-06-08T00:31:52  <zenhase> request.write (in CGI.py for example) has *data as parameter
2007-06-08T00:32:49  <zenhase> but the .write of file-like objects only accepts one parameter
2007-06-08T00:36:51  <xorAxAx> request.write is a redirection helper? :)
2007-06-08T00:37:01  <xorAxAx> zenhase: try to find places that need the *data syntax and unify it
2007-06-08T00:37:15  <xorAxAx> not to use it of course
2007-06-08T00:37:43  <ThomasWaldmann> all methods use *data
2007-06-08T00:38:11  <zenhase> hmm ...
2007-06-08T00:39:15  <xorAxAx> ThomasWaldmann: do you know why?
2007-06-08T00:39:40  <ThomasWaldmann> it is like that for long
2007-06-08T00:39:57  <ThomasWaldmann> and if you have to output multiple small strings, it is more efficient
2007-06-08T00:39:59  <zenhase> you mean, noone ever 'fixed' it :>
2007-06-08T00:42:42  <zenhase> at least in themebase and other themes they almost always aggregated output by themselves and did a request.write(''.join(outputlist)) when they were ready
2007-06-08T00:43:48  <ThomasWaldmann> what exactly is the problem with it?
2007-06-08T00:44:25  <zenhase> my problem with request.write accepting *data is that it seems so inconsistent :>
2007-06-08T00:44:59  <zenhase> outputting to request should act like a file-like object imo
2007-06-08T00:45:22  <zenhase> s/file-like/io-like
2007-06-08T00:46:08  <ThomasWaldmann> well, it does. just some other file like object might not behave like our write() :)
2007-06-08T00:46:27  <zenhase> but it's more about my sense of POLS in oo-design
2007-06-08T00:47:28  <zenhase> it's mostly cosmetic too me right now, i can work around it if there are problems
2007-06-08T00:47:52  <zenhase> tho the problem with redirectedOutput and StringIO remains
2007-06-08T00:48:45  <zenhase> output i want to capture and which expects .write(*data) will fail with the redirection to StringIO
2007-06-08T00:50:19  <ThomasWaldmann> ok, so it is a bug
2007-06-08T00:51:24  <zenhase> question is: where to fix it? fix it in redirectedOutput/redirect or fix it everywhere by making .write file-like in whole moin
2007-06-08T00:51:27  <zenhase> ?
2007-06-08T00:52:04  <ThomasWaldmann> maybe check the callers first
2007-06-08T00:52:12  <zenhase> could be many
2007-06-08T00:52:19  <zenhase> hmm
2007-06-08T00:52:28  <ThomasWaldmann> if most stuff has only one argument anyway, we can fix the write() methods
2007-06-08T00:53:08  <zenhase> i really would like to have a tool for tracing callers of a function
2007-06-08T00:54:25  <zenhase> perhaps i looked in the wrong places, but i didn't found anything on the web
2007-06-08T00:54:46  <zenhase> a decorator that could do that would be awesome :>
2007-06-08T00:55:22  <ThomasWaldmann> well, if you just change it, you will notice non-matching calls rather fast X)
2007-06-08T00:55:51  <zenhase> trace-thru-exception :>
2007-06-08T00:57:50  <ThomasWaldmann> btw, when refactoring that, encode() will get rather trivial. I would suggest removing the call to it and embedding the 2 lines of code directly into write()
2007-06-08T00:59:08  <zenhase> yeah
2007-06-08T01:03:54  <zenhase> erm ... bugfix into my repo?
2007-06-08T01:04:52  <ThomasWaldmann> try to make a clean changeset
2007-06-08T01:05:45  <zenhase> yep unintrusive
2007-06-08T01:06:18  <xorAxAx> zenhase: just grep for write before changning the sig
2007-06-08T01:06:22  <xorAxAx> zenhase: dont do it like ThomasWaldmann :-p
2007-06-08T01:06:50  <zenhase> xorAxAx: that's why i want a callee-tracer :>
2007-06-08T01:07:21  <xorAxAx> zenhase: well, we would need better code coverage to have that make sense
2007-06-08T01:08:06  <zenhase> xorAxAx: how do you mean that?
2007-06-08T01:08:51  <xorAxAx> zenhase: in order to trace calls, you need to actually do those calls. but in order to have all callersites do those, you need unit tests that cover all of the code
2007-06-08T01:10:23  <zenhase> xorAxAx: i see :>
2007-06-08T01:11:00  <xorAxAx> what did you expect? :)
2007-06-08T01:11:07  <xorAxAx> this is not java :)
2007-06-08T01:12:06  <zenhase> i know ... that's why unit-testing is more important
2007-06-08T01:12:30  <zenhase> tho i have to admit that i've never been a real good unit-test-writing coder myself
2007-06-08T01:12:57  <xorAxAx> life long learning :)
2007-06-08T01:13:49  <xorAxAx> good night
2007-06-08T01:13:54  <zenhase> sleep well xorAxAx
2007-06-08T01:30:19  <grzywacz> night xorAxAx
2007-06-08T01:32:31  <zenhase> hi grzywacz :>
2007-06-08T02:20:46  <zenhase> i call it a day for now
2007-06-08T02:21:04  <zenhase> next hacking session tomorrow
2007-06-08T02:21:12  <zenhase> gn8
2007-06-08T02:26:55  <grzywacz> ;]
2007-06-08T02:41:46  <CIA-27> moin: Karol 'grzywacz' Nowak <grzywacz@sul.uni.lodz.pl> * 2102:1ede132a9d38 1.7-jabber-knowak/ (5 files in 2 dirs): Suggested fixes.
2007-06-08T02:53:55  <grzywacz> ThomasWaldmann, around? 8)
2007-06-08T03:43:57  <CIA-27> moin: Karol 'grzywacz' Nowak <grzywacz@sul.uni.lodz.pl> * 2103:a3377b242e21 1.7-jabber-knowak/MoinMoin/events/ (JabberNotification.py messages.py): First try to reduce code duplication. More tomorrow after some sleep...
2007-06-08T08:14:56  <ThomasWaldmann> moin
2007-06-08T11:35:27  <zenhase> moin
2007-06-08T13:18:43  <grzywacz> moin
2007-06-08T15:09:08  <vpv> have a nice weekend, I'll be away until Sunday evening probably. real coding hopefully next week...
2007-06-08T15:10:23  <zenhase> have a nice weekend
2007-06-08T17:27:19  <lanius> xorAxAx: the data stuff we discussed is not very performant for databases since they can't just do a move operation but have to read the whole file again and save it to the db then
2007-06-08T17:27:41  <xorAxAx> lanius: what is unperformant about it?
2007-06-08T17:27:59  <lanius> first saving all to the file
2007-06-08T17:28:00  <xorAxAx> lanius: the problem is that not all databases offer chunked write
2007-06-08T17:28:01  <lanius> then reading the file again
2007-06-08T17:28:07  <lanius> and then inserting it to the db
2007-06-08T17:28:20  <xorAxAx> well, its the best scheme i can come up with
2007-06-08T17:28:41  <xorAxAx> and the copying latency is rather irrelevant
2007-06-08T17:29:13  <lanius> if they don't offer chunked write they won't offer chunked read, will they
2007-06-08T17:30:38  <lanius> so for reading you will first download the content to a file again and read it from there?
2007-06-08T17:31:02  <xorAxAx> yes
2007-06-08T17:31:32  <xorAxAx> btw!
2007-06-08T17:31:41  <xorAxAx> the interface doesnt mandate if temporary files should be used
2007-06-08T17:32:08  <xorAxAx> so i dont see a problem
2007-06-08T17:33:07  <lanius> i'm currently thinking about the interface
2007-06-08T17:33:19  <lanius> is there a tmp dir for moin?
2007-06-08T17:34:10  <xorAxAx> you mean in the current data tree?
2007-06-08T17:35:43  <lanius> yes
2007-06-08T17:36:56  <xorAxAx> no, moin creates temporary files in the direcotries where it needs them
2007-06-08T17:37:20  <lanius> ok
2007-06-08T17:38:02  <lanius> my current design that you have a global Data class which acts as Proxy for the DataBackend does not make sense i think, because when both implement DataBackend you cannot do that move operation
2007-06-08T17:41:01  <lanius> do you understand what i mean?
2007-06-08T17:41:37  <xorAxAx> yes, so my initial hint was to implement this on a lower level
2007-06-08T17:41:56  <lanius> yes, so no proxy object, just the DataBackend
2007-06-08T17:42:19  <xorAxAx> maybe with some abstract base class, yes
2007-06-08T17:42:36  <lanius> but what about the following case
2007-06-08T17:42:39  <lanius> you create a new item
2007-06-08T17:42:54  <xorAxAx> yes
2007-06-08T17:43:10  <lanius> the create operation is only performed on save(), until then the item is only in memory
2007-06-08T17:43:14  <lanius> then you call get_data_backend for that item
2007-06-08T17:43:19  <lanius> but the item is not yet there
2007-06-08T17:44:27  <xorAxAx> well, then you need some temporary data record on the backend side ... in a db this is trivial
2007-06-08T17:44:33  <xorAxAx> for files as well (move operation)
2007-06-08T17:44:53  <xorAxAx> hmm, but thats a layering violation
2007-06-08T17:45:11  <lanius> so the write operations are done to a temporary file and the move is performend on close() which is called by save() and with a check if the item exists now
2007-06-08T17:45:27  <xorAxAx> how about requiring save() before actual data operations?
2007-06-08T17:45:35  <xorAxAx> so save() is only restricted to metadata
2007-06-08T17:46:38  <lanius> what about performing all operations directly? what was the problem with that?
2007-06-08T17:47:26  <lanius> since we have no transactions...
2007-06-08T17:47:38  <xorAxAx> which operations?
2007-06-08T17:47:47  <xorAxAx> the data operations?
2007-06-08T17:47:51  <lanius> creating an item, creating a revision, changing metadata
2007-06-08T17:48:26  <xorAxAx> so it reads+writes the metadata file for every single  changed metadata key?
2007-06-08T17:48:43  <lanius> ok not efficient for metadata, what about the rest
2007-06-08T17:49:10  <xorAxAx> as long as you use locks ...
2007-06-08T17:49:21  <xorAxAx> it should be ok
2007-06-08T17:50:08  <lanius> locks on what
2007-06-08T17:52:08  <xorAxAx> well, you need two phase write locking to avoid transaction lost update issues on concurrent metadata modifications
2007-06-08T17:52:16  <xorAxAx> as we discussed earlier
2007-06-08T17:52:38  <lanius> what about two threads
2007-06-08T17:52:47  <lanius> the first one reads the list of revisions
2007-06-08T17:52:53  <lanius> the second one deletes the last revision after that
2007-06-08T17:53:02  <lanius> the first one tries to  access the last revision which does not exist anymore
2007-06-08T17:53:30  <lanius> does not happen very often so ignore? or read locks? or what
2007-06-08T17:55:53  <xorAxAx> lanius: i think that can be ignored
2007-06-08T18:02:31  <lanius> what do you mean by two phase
2007-06-08T18:04:52  <xorAxAx> lanius: every transaction has a lock aquiring and a lock releasement stage, see the slides that i linked some days ago
2007-06-08T18:08:11  <lanius> i didn't really understand the slides
2007-06-08T18:12:21  <xorAxAx> :)
2007-06-08T18:17:12  <lanius> what concurrent metadata modification have we again? for the wiki sync we decided to lock the whole page
2007-06-08T18:19:27  <xorAxAx> well, locking the whole page is fine
2007-06-08T18:19:45  <xorAxAx> but you need to lock before you start reading the page
2007-06-08T18:19:51  <xorAxAx> in case of any modification
2007-06-08T18:20:16  <xorAxAx> i imagine that users will be able to modify the metadata like they can do now
2007-06-08T18:20:38  <xorAxAx> the question is if it makes sense to write a new data revision for that
2007-06-08T18:22:08  <xorAxAx> imagine users changing the ACL
2007-06-08T18:45:39  <lanius> currently it will create a new revision
2007-06-08T18:51:19  <xorAxAx> so it duplicates the data?
2007-06-08T18:58:30  <lanius> for the moin1.6 backnd yes
2007-06-08T19:01:10  <xorAxAx> there is no other way :)
2007-06-08T19:01:13  <xorAxAx> for 1.6
2007-06-08T19:13:14  <lanius> yes
2007-06-08T19:51:40  <ThomasWaldmann> moin
2007-06-08T20:01:00  <ThomasWaldmann> that's an interesting topic for bigger files
2007-06-08T20:02:25  <ThomasWaldmann> on posix we could use hardlinks, on win32 we're fscked
2007-06-08T20:02:45  <xorAxAx> ThomasWaldmann: ?
2007-06-08T20:02:58  <xorAxAx> ah, you mean metadata changes?
2007-06-08T20:03:18  <ThomasWaldmann> if only metadata changes, and data = 4.7gb iso image
2007-06-08T20:05:25  <xorAxAx> yes
2007-06-08T20:06:17  <ThomasWaldmann> that somehow kills my plans for next fs backend 8(
2007-06-08T20:11:23  <starshine> mswin has its own wacky edition of hardlinks
2007-06-08T20:11:34  <starshine> which cygwin honors
2007-06-08T20:12:07  <starshine> they're called.. uh... re___ points though I can't remember what the something is
2007-06-08T20:12:32  <starshine> act a little more like bind mounts
2007-06-08T20:12:37  <ThomasWaldmann> http://win32.mvps.org/ntfs/lnw.cpp X)
2007-06-08T20:17:36  <ThomasWaldmann> we could also point to some data revision from the metadata
2007-06-08T20:18:22  <ThomasWaldmann> if we realize exists() somehow else, that will do no harm, except making stuff a bit more complicated
2007-06-08T20:24:20  <ThomasWaldmann> win32file.CreateHardLink
2007-06-08T20:33:03  <xorAxAx> well, i dont see the point
2007-06-08T20:33:32  <xorAxAx> the metadata file could have some attrib that specifies where to find the data
2007-06-08T20:33:43  <xorAxAx> without any speed impact (you need to read the metadata anyway)
2007-06-08T20:35:09  <ThomasWaldmann> if you don't do it with hardlinks, you have to refcount in case you want to purge some
2007-06-08T20:35:38  <xorAxAx> not really :)
2007-06-08T20:35:57  <xorAxAx> you just read all metadata entries on purge
2007-06-08T20:36:03  <xorAxAx> much cheaper
2007-06-08T20:36:28  <ThomasWaldmann> that's what you have to do for refcounting, yes
2007-06-08T20:37:29  <xorAxAx> yes, but not in realtime, only on puge
2007-06-08T20:37:31  <xorAxAx> purge
2007-06-08T20:38:07  <ThomasWaldmann> yes, I said that :)
2007-06-08T20:38:43  <xorAxAx> i just wanted to clarify my last sentence :)
2007-06-08T20:40:07  <ThomasWaldmann> but now we have 2 "current revision" things. one for metadata revision and one for data revision (stored in the metadata)
2007-06-08T20:41:22  <xorAxAx> so? :)
2007-06-08T20:44:02  <ThomasWaldmann> we have to check the api about that :)
2007-06-08T20:48:09  <ThomasWaldmann> lanius:
2007-06-08T22:23:06  <ThomasWaldmann> hi rhoe :)
2007-06-08T22:24:03  <rhoe> hi thomas!
2007-06-08T22:24:18  <rhoe> thanks for setting up a new accessibility branch
2007-06-08T22:25:57  <ThomasWaldmann> yeah, your and nico's ssh keys are set (as it was in the old branch)
2007-06-08T22:26:35  <rhoe> ok. thanks. now we do it step by step to keep also the code accessible
2007-06-08T22:26:43  <rhoe> ;-)
2007-06-08T22:29:04  <xorAxAx> wee, accessible code :)
2007-06-08T22:41:16  * ThomasWaldmann reviews cs 1920
2007-06-08T22:41:52  <xorAxAx> grzywacz: PEP8 says "Modules should have short, all-lowercase names. ", please rename *Notification.py (This message has been postponed.)
2007-06-08T22:41:55  <xorAxAx> grzywacz: still a star import ... from MoinMoin.events import * (This message has been postponed.)
2007-06-08T22:41:58  <xorAxAx> grzywacz: stilla a relative import ... +from config import BotConfig (This message has been postponed.)
2007-06-08T22:42:00  <grzywacz> evening
2007-06-08T22:42:18  <xorAxAx> hi grzywacz
2007-06-08T22:42:36  <grzywacz> The train was 1.5h late -_-;
2007-06-08T22:42:46  <xorAxAx> ThomasWaldmann: using numbers instead of hashes is confusing and pointless :)
2007-06-08T22:45:08  <zenhase> *phew*
2007-06-08T22:45:56  <zenhase> looked through all request.write calls and rewrote them to a single parameter
2007-06-08T22:46:19  <ThomasWaldmann> i use the revision numbers of the official repo
2007-06-08T22:47:08  <ThomasWaldmann> zenhase: did you count? %)
2007-06-08T22:47:36  <xorAxAx> ThomasWaldmann: even then, you didnt say the branch
2007-06-08T22:48:08  <ThomasWaldmann> of course it is 1.6 accessibility :) man context :))
2007-06-08T22:48:50  <rhoe> yeah, cs1920 is also my favorite one ;-) "Added all changes until now"..
2007-06-08T22:50:23  <zenhase> ThomasWaldmann: not really
2007-06-08T22:50:53  <zenhase> i can look at the diff
2007-06-08T22:52:01  <zenhase> hmm, i would guess about 10 occurances
2007-06-08T23:00:27  <xorAxAx> how many callers?
2007-06-08T23:02:31  <zenhase> 10 calls in 4 callers
2007-06-08T23:02:56  <zenhase> wait a sec
2007-06-08T23:11:53  <CIA-27> moin: Florian Krupicka <florian.krupicka@googlemail.com> * 2049:688e7a207163 1.7-template-fkrupicka/MoinMoin/ (12 files in 3 dirs): made request.write more file-like (only one str-parameter)
2007-06-08T23:31:19  <CIA-27> moin: Thomas Waldmann <tw AT waldmann-edv DOT de> * 2050:ef2e894abab1 1.7-accessibility/ (8 files in 2 dirs): implement table_header formatter method
2007-06-08T23:32:23  <ThomasWaldmann> zenhase: str parameter?
2007-06-08T23:35:13  <ThomasWaldmann> ok, the parameter is either str or unicode, just the commit comment is misleading
2007-06-08T23:39:12  <zenhase> hmm yeah