1 2009-12-17T00:08:11 <dreimark> may be CheckTranslation should not only show pages
2 2009-12-17T00:08:30 <dreimark> i think showing the messages could be nice too
3 2009-12-17T00:14:02 *** johill
4 2009-12-17T00:14:02 *** dimazest
5 2009-12-17T00:14:02 *** vpv
6 2009-12-17T00:14:02 *** xorAxAx
7 2009-12-17T00:14:02 *** ser
8 2009-12-17T00:18:01 *** dimazest
9 2009-12-17T00:18:01 *** johill
10 2009-12-17T00:18:01 *** ser
11 2009-12-17T00:18:01 *** vpv
12 2009-12-17T00:18:01 *** xorAxAx
13 2009-12-17T00:20:52 *** johill
14 2009-12-17T00:20:52 *** dimazest
15 2009-12-17T00:20:52 *** vpv
16 2009-12-17T00:20:52 *** xorAxAx
17 2009-12-17T00:20:52 *** ser
18 2009-12-17T00:21:00 *** dimazest
19 2009-12-17T00:21:00 *** johill
20 2009-12-17T00:21:00 *** ser
21 2009-12-17T00:21:00 *** vpv
22 2009-12-17T00:21:00 *** xorAxAx
23 2009-12-17T00:31:07 *** johill
24 2009-12-17T00:31:07 *** dimazest
25 2009-12-17T00:31:07 *** vpv
26 2009-12-17T00:31:07 *** xorAxAx
27 2009-12-17T00:31:07 *** ser
28 2009-12-17T00:31:12 *** vpv
29 2009-12-17T00:31:16 *** dimazest
30 2009-12-17T00:31:19 *** ser
31 2009-12-17T00:31:20 *** xorAxAx
32 2009-12-17T00:32:40 *** johill
33 2009-12-17T01:01:54 *** grzywacz
34 2009-12-17T01:16:53 <waldi> ThomasWaldmann: the new moin and creole module needs about 800ms for the regexp parsing
35 2009-12-17T01:19:11 <waldi> ups, no
36 2009-12-17T01:19:58 <waldi> wikiutils ...
37 2009-12-17T01:25:57 <waldi> ThomasWaldmann: i just did a import of MoinMoin.converter2, and it scrolls with all the dependencies ...
38 2009-12-17T01:35:26 <waldi> config loading needs 150ms in re
39 2009-12-17T01:36:51 <waldi> this all needs a consequent cleanup
40 2009-12-17T01:46:24 *** dennda
41 2009-12-17T03:48:38 *** franklin_
42 2009-12-17T03:58:53 *** franklin
43 2009-12-17T04:13:05 *** TheSheep_
44 2009-12-17T04:19:21 *** TheSheep
45 2009-12-17T08:24:29 <waldi> ThomasWaldmann: we should read the documentation exactly. \s and so one are dependant on the unicode database, which themself does not depend on the locale. (that the differences between the locale-definitions are not worth the trouble anyway is a different matter)
46 2009-12-17T08:25:55 <waldi> but there is something missing, hrm
47 2009-12-17T08:26:00 <waldi> okay, drop that whole shit
48 2009-12-17T08:36:05 <waldi> hmm, that sre module supports more than documented ...
49 2009-12-17T09:00:31 <waldi> wtf?
50 2009-12-17T09:00:59 <waldi> werkzeug includes validators for function arguments?
51 2009-12-17T09:02:51 *** vpv
52 2009-12-17T09:04:14 <waldi> including werkzeug needs about 200ms of its own, which is about 1/10 of the startup cost
53 2009-12-17T09:19:26 *** TheSheep_
54 2009-12-17T09:23:11 <ThomasWaldmann> waldi: i just read the py 2.6 re docs and wondered whether that unicode stuff was ever like documented there
55 2009-12-17T09:31:56 <ThomasWaldmann> hmm, there is no upper/lower stuff
56 2009-12-17T09:36:49 <waldi> yes, python re lacks the [::] character classes all the other implementations features
57 2009-12-17T09:38:06 <waldi> okay, points for optimization: calculate the large regexps of the converters on the first use; cut out all other uses of the upper/lower thing; don't include heavy modules on module level
58 2009-12-17T09:39:32 <waldi> kill down the swiss army knife wikiutils
59 2009-12-17T09:39:52 <waldi> s/s$//
60 2009-12-17T09:43:53 <waldi> replace regexps where possible: split_regex could be replaced by a static lookup table for unicode.translate
61 2009-12-17T09:46:07 <johill> don't open all pages ;)
62 2009-12-17T09:48:20 <ThomasWaldmann> johill: btw, we did a debugger step through up to send_title / edit_bar yesterday
63 2009-12-17T09:49:09 <ThomasWaldmann> and the only thing eating quite some (cpu) time was compiling the huge regexes
64 2009-12-17T09:50:26 <waldi> replace regexps where possible: split_regex could be replaced by a static lookup table for unicode.translate if we use the undocumented fact that invalid mappings only raises errors if the characters are found during the process
65 2009-12-17T09:50:50 <waldi> okay, away
66 2009-12-17T09:52:06 <johill> ThomasWaldmann: I don't think it's really CPU, but if you open/stat a lot of files (all of underlay) it depends on your disk speed
67 2009-12-17T09:52:12 <ThomasWaldmann> i don't think we need to run around and replace regexes. we should just get rid of those insanely big regexes needed for camelcase stuff.
68 2009-12-17T09:52:14 <johill> fwiw, I'm disk limited, not cpu limited
69 2009-12-17T09:52:48 <ThomasWaldmann> i didn't see it yet opening all pages
70 2009-12-17T09:59:19 <johill> dunno, I straced it and it definitely did
71 2009-12-17T09:59:25 <johill> might not have been at startup, but at the first request
72 2009-12-17T09:59:33 <johill> or so
73 2009-12-17T09:59:45 <johill> anyway, bbl
74 2009-12-17T10:02:06 <ThomasWaldmann> i does open SOME underlay pages
75 2009-12-17T10:02:30 <ThomasWaldmann> that is due to navibar or the page your are looking at referencing them
76 2009-12-17T10:02:43 <ThomasWaldmann> it checks whether those pages exist
77 2009-12-17T10:02:52 <johill> nah it was way more than that
78 2009-12-17T10:03:02 <johill> like some weird script ones that I can't even read with (xxxxx)
79 2009-12-17T10:03:56 <ThomasWaldmann> ok, i'll unpack some weird script stuff and run strace again
80 2009-12-17T10:04:57 <johill> and it also opened the editlog twice for every page it opened or something
81 2009-12-17T10:05:20 <johill> which is probably ok (at least on linux since it'd be cached by the kernel)
82 2009-12-17T10:07:53 <ThomasWaldmann> i straced wikiserver.py, requested LanguageSetup - no accesses to non-german non-english pages
83 2009-12-17T10:08:19 <ThomasWaldmann> (i unpacked japanese and lithuanin pages in a previous run)
84 2009-12-17T10:08:23 <johill> hm
85 2009-12-17T10:08:48 <johill> well not sure, this was on an existing installation
86 2009-12-17T10:08:55 <ThomasWaldmann> $ strace -o strace.log -f -e trace=file python wikiserver.py
87 2009-12-17T10:08:55 <johill> I can take a look later, for now I really have to go
88 2009-12-17T10:09:13 <ThomasWaldmann> ok, cul8er
89 2009-12-17T10:10:47 <ThomasWaldmann> bbl
90 2009-12-17T10:16:02 *** dreimark
91 2009-12-17T10:16:02 *** ThomasWaldmann
92 2009-12-17T10:19:28 *** ThomasWaldmann
93 2009-12-17T10:19:33 *** dreimark
94 2009-12-17T11:19:49 <ThomasWaldmann> moin
95 2009-12-17T11:22:53 *** dennda
96 2009-12-17T12:59:38 *** grzywacz
97 2009-12-17T16:44:57 <dreimark> moin
98 2009-12-17T17:03:46 <ThomasWaldmann> hi dreimark
99 2009-12-17T17:08:53 <dreimark> i don't see many wiki/data/page requests
100 2009-12-17T17:09:30 <dreimark> or underlay
101 2009-12-17T17:10:44 <dreimark> but this one is often requested workspace/moin-1.9/wiki/underlay/pages/FehlendeSeite/revisions/00000001
102 2009-12-17T17:12:47 <dreimark> i wonder because i have only viewed on RC and some other pages
103 2009-12-17T17:12:56 <dreimark> but didn't create a new one
104 2009-12-17T17:26:55 <ThomasWaldmann> johill: could the underlay accesses you see be related to some extensions (theme, whatever) you use?
105 2009-12-17T18:22:34 *** vpv
106 2009-12-17T18:30:48 *** lotekthirteen
107 2009-12-17T18:52:14 <dreimark> bbl
108 2009-12-17T20:23:07 *** lotekthirteen
109 2009-12-17T21:36:47 *** franklin_
110 2009-12-17T21:38:25 *** franklin
111 2009-12-17T21:39:53 <johill> ThomasWaldmann: not sure, I'll have to look at it in more detail but have a lot of other stuff on my plate right now
112 2009-12-17T21:39:58 <johill> I'll figure it out eventually
113 2009-12-17T23:31:17 *** lotekthirteen
114 2009-12-17T23:51:40 <ThomasWaldmann> re
