1 2011-06-30T00:00:16 <dreimark> ThomasWaldmann: is the time from shell correct?
2 2011-06-30T00:00:37 <dreimark> it looks to me it is 2mins in the future
3 2011-06-30T00:04:20 <moin7> thanks
4 2011-06-30T00:17:53 <dreimark> moin7: http://pypi.python.org/pypi/TinyUrl/
5 2011-06-30T00:20:10 <dreimark> RogerHaase: forget line 42 question, i had the + in the new line, that confused me
6 2011-06-30T00:27:06 <RogerHaase> dreimark: I am confused by line 32 comment. Are you suggesting I replace "href" with "url"?
7 2011-06-30T00:28:03 <dreimark> yes because href = remove_overlay_prefixes(href) but the parameter of thar function is url
8 2011-06-30T00:28:46 <dreimark> later on i saw that code below also mix it with href
9 2011-06-30T00:31:15 *** m4k3r
10 2011-06-30T00:39:04 <dreimark> gn
11 2011-06-30T00:49:35 *** m4k3r
12 2011-06-30T00:55:54 *** grzywacz
13 2011-06-30T02:10:17 *** MattMaker
14 2011-06-30T03:34:23 *** Marchael
15 2011-06-30T04:42:53 *** RogerHaase
16 2011-06-30T06:55:57 *** m4k3r
17 2011-06-30T07:14:09 *** Marchael
18 2011-06-30T07:14:30 <Marchael> moin
19 2011-06-30T08:06:44 *** m4k3r
20 2011-06-30T08:28:07 <dreimark> ThomasWaldmann: what is ?action=raw in 2.0?
21 2011-06-30T08:28:39 <dreimark> I often use that for spam defending because I don't want the content rendered
22 2011-06-30T08:43:48 *** MattMaker
23 2011-06-30T08:55:45 *** sinha
24 2011-06-30T09:59:03 *** mkerrin
25 2011-06-30T09:59:29 *** mkerrin
26 2011-06-30T10:07:45 *** m4k3r
27 2011-06-30T10:08:59 *** humbu
28 2011-06-30T10:10:20 <humbu> Hello, I red the page http://moinmo.in/JabberSupport and i want to implement it on my wiki. Where can i find further Documentation for this? Up to now is doesn't work
29 2011-06-30T10:11:32 *** humbu
30 2011-06-30T10:28:18 *** mkerrin
31 2011-06-30T10:51:51 *** mkerrin
32 2011-06-30T10:56:41 *** mkerrin
33 2011-06-30T11:24:46 *** greg_f
34 2011-06-30T11:26:08 *** sinha
35 2011-06-30T11:26:13 *** sinha1
36 2011-06-30T11:30:18 *** TheSheep
37 2011-06-30T11:30:53 *** TheSheep
38 2011-06-30T12:03:50 *** Marchael
39 2011-06-30T12:07:26 *** sinha1
40 2011-06-30T12:12:32 *** Marchael
41 2011-06-30T12:15:42 <ThomasWaldmann> re
42 2011-06-30T12:16:19 <Marchael> hi ThomasWaldmann
43 2011-06-30T12:17:46 <ThomasWaldmann> dreimark: until recently one could use the "download" view, because it showed some stuff in the browser
44 2011-06-30T12:25:15 <Marchael> ThomasWaldmann: can I use one flask script from another?
45 2011-06-30T12:26:04 <ThomasWaldmann> you do not need
46 2011-06-30T12:26:16 <ThomasWaldmann> AFAIK
47 2011-06-30T12:26:31 <Marchael> i think what I can call update_index script from build_index
48 2011-06-30T12:26:48 <Marchael> build script wil take params for update script and call update script
49 2011-06-30T12:27:05 <ThomasWaldmann> you do that on the python level
50 2011-06-30T12:27:33 <ThomasWaldmann> and I already advised you yesterday to just use one script called "index.py" and do all index build/update/etc. operations from there
51 2011-06-30T12:28:21 <Marchael> but then it was huge and less readable script
52 2011-06-30T12:36:00 <ThomasWaldmann> source gets readable by writing readable source (using good names, functions, being consistent, ...), not by splitting related stuff it into multiple files
53 2011-06-30T12:36:14 <ThomasWaldmann> -it
54 2011-06-30T12:37:04 <Marchael> ok
55 2011-06-30T12:57:21 <dreimark> ThomasWaldmann: ah, new issue
56 2011-06-30T12:58:06 <dreimark> have you reviewed rogers transclusions?
57 2011-06-30T12:59:22 <dreimark> http://codereview.appspot.com/4535086
58 2011-06-30T13:04:36 *** sinha
59 2011-06-30T13:04:53 <ThomasWaldmann> dreimark: not yet, rather busy with server stuff
60 2011-06-30T13:15:59 *** MattMaker
61 2011-06-30T14:12:34 *** m4k3r
62 2011-06-30T14:12:51 *** m4k3r
63 2011-06-30T14:34:55 *** sinha
64 2011-06-30T14:35:06 *** sinha
65 2011-06-30T14:37:46 *** moin7
66 2011-06-30T15:55:55 <sinha> dreimark: Regarding putting the list of letters on top, you said to put the first letters of item names which exist in wiki, suppose no item has name starting from letter P, R or S, so we wont display the letter P, R or S there to select? And i am also stuck in on more place, like how to put links for non-english chars, in general we are doing +index/startsWith=A, we can easily send the english letters as param but how to send chinese/russian ?
67 2011-06-30T15:57:18 <sinha> dreimark: One solution for all this above could be putting a small input textbox there, where a user can enter its input and data can easily be posted to backend .
68 2011-06-30T16:24:49 <Marchael> ThomasWaldmann: please, review http://codereview.appspot.com/4662071
69 2011-06-30T16:26:54 *** m4k3r
70 2011-06-30T16:27:18 *** m4k3r
71 2011-06-30T17:13:46 *** m4k3r
72 2011-06-30T17:14:09 *** m4k3r
73 2011-06-30T17:14:17 <Marchael> ThomasWaldmann: ping
74 2011-06-30T17:20:18 <ThomasWaldmann> Marchael: still at work
75 2011-06-30T17:24:15 <Marchael> ok, I'm now reading doc to indexing,py
76 2011-06-30T17:25:07 <dreimark> sinha: no input box please
77 2011-06-30T17:25:22 <sinha> okay
78 2011-06-30T17:25:28 <sinha> so how to put letters on top ?
79 2011-06-30T17:25:49 <sinha> because reading first letters of items isn't giving all the alphabets
80 2011-06-30T17:25:57 <dreimark> sinha: and yes if there is no item with P, R or S then also not a letter for that
81 2011-06-30T17:26:10 <sinha> it looks a bit weird
82 2011-06-30T17:26:24 <sinha> i mean there will be only 5-6 letters on top
83 2011-06-30T17:26:28 <sinha> in some cases
84 2011-06-30T17:26:51 <dreimark> you need a function which collects the first char of each item
85 2011-06-30T17:27:06 <dreimark> then you have also those you can't type on the keyboard
86 2011-06-30T17:27:35 <sinha> yes i have created a function to extract first letter of items
87 2011-06-30T17:28:05 <sinha> but they are getting returned as unicode as u'A', u'B'....like this
88 2011-06-30T17:28:25 <sinha> so when i try to put a link as +index/startsWith=letter, there error comes
89 2011-06-30T17:28:40 <sinha> so should i get unicodes for these letters
90 2011-06-30T17:28:48 <sinha> for linking purpose ?
91 2011-06-30T17:29:50 <dreimark> exact error please
92 2011-06-30T17:30:16 <dreimark> +index/startsWith=A works?
93 2011-06-30T17:32:16 <sinha> yes
94 2011-06-30T17:39:03 *** sinha
95 2011-06-30T17:39:14 *** sinha
96 2011-06-30T17:40:52 <dreimark> sinha: can you show the traceback of the error please
97 2011-06-30T17:41:59 <dreimark> xorAxAx: waldi ronny ThomasWaldmann do you know if you attend pyconde (http://de.pycon.org)?
98 2011-06-30T17:43:16 <sinha> dreimark: yes was doing that only, here http://paste.pocoo.org/show/424883/
99 2011-06-30T17:43:58 <sinha> here the letter which has been fetched is a unicode
100 2011-06-30T17:48:01 <ronny> dreimark: not yet
101 2011-06-30T18:02:06 <dreimark> sinha: query strings needs I guess the right encoding. if all would be only ascii a str() would solve that
102 2011-06-30T18:02:19 *** m4k3r
103 2011-06-30T18:03:03 *** m4k3r
104 2011-06-30T18:03:52 <sinha> dreimark: will it solve if the letters is in other language, in russina/chinese ?
105 2011-06-30T18:05:48 <dreimark> str likly not. For now implement it only for english. I have to look at that in more detail or we have to wait until thomas has time.
106 2011-06-30T18:07:18 <dreimark> after we know how we can pass unicode to the querystring this is just one little change
107 2011-06-30T18:11:17 <sinha> dreimark: i did str() in python code, it isn't working, outputting error as "'ascii' codec can't encode character u'\u041c' in position 0: ordinal not in range(128)"
108 2011-06-30T18:12:02 <sinha> dreimark: and i was also navigating few wikis, they are also providing this kind of selection for english only, for other languages they are simply skipping this part
109 2011-06-30T18:19:46 *** m4k3r
110 2011-06-30T18:40:38 <dreimark> sinha: well we don't need to do the same mistakes than others
111 2011-06-30T18:41:29 <dreimark> ronny: do you know what the name of this % hex expression for hex chars of the browser is
112 2011-06-30T18:41:57 <dreimark> http://test.moinmo.in/%C3%AE%C3%B8%C3%AE
113 2011-06-30T18:43:06 <dreimark> sinha: you must only find a way to send the data to the python program, this can be done by some encoding
114 2011-06-30T18:43:20 <dreimark> e.g. you can do base64 encoding
115 2011-06-30T18:43:53 <dreimark> because then the python program can decode it back and search with that
116 2011-06-30T18:44:08 <dreimark> we have really no reason to skip this functionallity
117 2011-06-30T18:57:13 <dreimark> sinha: ^
118 2011-06-30T19:03:43 *** m4k3r
119 2011-06-30T19:12:38 *** MattMaker
120 2011-06-30T19:24:01 *** mkerrin
121 2011-06-30T19:29:24 <sinha> dreimark: rather then sending it as GET, by sending it through POST may solve the problem ?
122 2011-06-30T19:36:55 <dreimark> sinha: try it
123 2011-06-30T19:37:26 <sinha> trying the same now
124 2011-06-30T19:42:27 <dreimark> sinha: Because URLs cannot contain non ASCII data you will always get
125 2011-06-30T19:42:27 <dreimark> bytestrings back. Non ASCII characters are urlencoded with the
126 2011-06-30T19:42:27 <dreimark> charset defined on the map instance.
127 2011-06-30T19:42:47 <dreimark> that is from routing.py
128 2011-06-30T19:42:53 <dreimark> so just encode it
129 2011-06-30T19:44:04 <sinha> use urlencode function of that urllib ? or something else
130 2011-06-30T19:45:01 <dreimark> please read werkzeug.routing.py
131 2011-06-30T19:45:34 <sinha> okay
132 2011-06-30T19:49:05 <Marchael> ThomasWaldmann: are you still busy?
133 2011-06-30T19:55:14 <dreimark> sinha: look at from werkzeug.urls import url_encode
134 2011-06-30T19:55:26 <dreimark> url_encode({'test':u'î'})
135 2011-06-30T19:55:55 <sinha> yes, looked at it, how abour url_quote_plus
136 2011-06-30T19:56:05 <sinha> s/abour/about
137 2011-06-30T19:57:05 <dreimark> that's it because you don't need the key
138 2011-06-30T19:57:59 <dreimark> please use the one from werkzeug
139 2011-06-30T19:58:28 <dreimark> sinha: ^
140 2011-06-30T20:01:12 <Marchael> ThomasWaldmann: sometimes I think what it was wrong to write WhooshIndex(), because that stuff should be located in MoinMoin.storage.backends.indexing
141 2011-06-30T20:01:34 <Marchael> also
142 2011-06-30T20:02:04 <Marchael> script should import udpate/clean/build functions from that
143 2011-06-30T20:02:12 <Marchael> *scripts
144 2011-06-30T20:02:39 <dreimark> bbl
145 2011-06-30T20:02:45 <Marchael> because when I'll edit MoinMoin.storage.backends.indexing I'll need same functions
146 2011-06-30T20:04:16 <Marchael> I guess you was right, I should move all stuff from MoinMoin.search.indexing to MoinMoin.storage.backends.indexing, and in additional move functions from building script
147 2011-06-30T20:05:01 <Marchael> otherwise I'll duplicate myself or write unconsistent code
148 2011-06-30T20:06:20 *** greg_f
149 2011-06-30T20:09:05 <ThomasWaldmann> re
150 2011-06-30T20:10:37 <ThomasWaldmann> Marchael: the stuff you need at moin runtime is a bit different.
151 2011-06-30T20:10:58 <ThomasWaldmann> you don't have that big looploop, but just index a single revision
152 2011-06-30T20:11:10 <ThomasWaldmann> and you need to think about concurrency
153 2011-06-30T20:13:35 <Marchael> I mean may be move my stuff from MoinMoin.search.indexing to MoinMoin.storage.backends.indexing.ItemIndex?
154 2011-06-30T20:15:29 <Marchael> also, please make review
155 2011-06-30T20:23:15 <Marchael> ThomasWaldmann: ^
156 2011-06-30T20:43:44 *** grzywacz
157 2011-06-30T20:45:02 <ThomasWaldmann> Marchael: did you just copy that big function?
158 2011-06-30T20:45:44 <Marchael> commonly yes
159 2011-06-30T20:46:11 <Marchael> but add some stuff to check for index name
160 2011-06-30T20:46:19 <Marchael> s/for//
161 2011-06-30T20:51:48 <Marchael> ThomasWaldmann: I guess BufferedWriter will be fine for working with single transactions
162 2011-06-30T20:52:20 <Marchael> because we don't plan to add bactches of documents there
163 2011-06-30T20:56:30 <Marchael> but then we need to find a way to keep in open all the time what moin working
164 2011-06-30T20:56:40 <Marchael> s/in/it/
165 2011-06-30T20:58:01 <Marchael> may be put it to flaskg.storage and call each time when commit happens?
166 2011-06-30T21:18:06 <ThomasWaldmann> Marchael: review done
167 2011-06-30T21:18:34 <ThomasWaldmann> i wrote some thought about these special writers on EP, just read it
168 2011-06-30T21:18:46 <Marchael> ok
169 2011-06-30T21:19:47 <Marchael> am, only 181?
170 2011-06-30T21:34:46 <ThomasWaldmann> yes
171 2011-06-30T21:38:12 *** MattMaker
172 2011-06-30T21:38:33 *** RogerHaase
173 2011-06-30T21:47:38 <Marchael> ThomasWaldmann: please, see my comments
174 2011-06-30T21:52:05 *** sinha
175 2011-06-30T21:53:33 <Marchael> see ep 183
176 2011-06-30T22:17:55 <Marchael> ThomasWaldmann: but if someone destroys revision then it'll be removed from backend
177 2011-06-30T22:18:21 <Marchael> and we can see it in delete_indexes
178 2011-06-30T22:18:56 <ThomasWaldmann> the errors you are catching there are usually due to race conditions, otherwise you would not see them, right?
179 2011-06-30T22:20:02 <Marchael> But I don't want to catch unexpected excetpions
180 2011-06-30T22:20:42 <ThomasWaldmann> why do you catch NonExistentItem then?
181 2011-06-30T22:21:21 <Marchael> mm? I catch NoSuchItemError
182 2011-06-30T22:21:44 <ThomasWaldmann> yeah, that is what i meant
183 2011-06-30T22:22:06 <Marchael> because item marked as trashed
184 2011-06-30T22:22:15 <Marchael> it don't removes from backend
185 2011-06-30T22:22:23 <Marchael> but I can't access to it
186 2011-06-30T22:23:04 <ThomasWaldmann> there are no "items marked as trash" and you can access all items
187 2011-06-30T22:23:25 <ThomasWaldmann> if a items gets trashed, it just gets renamed to Trash/...
188 2011-06-30T22:24:04 <Marchael> I iterate over all revisions in this item and can't get access
189 2011-06-30T22:24:24 <Marchael> that's why I catching this exception
190 2011-06-30T22:26:02 <ThomasWaldmann> i guess without more specific information or questions from your side, we could discuss this for hours
191 2011-06-30T22:27:05 <Marchael> ok, usually error raises from storage_rev = backend.get_item(name).get_revision(rev_no)
192 2011-06-30T22:27:08 <ThomasWaldmann> and if you expect items not being there any more, you should also expect revisions not being there any more, that is all what I said
193 2011-06-30T22:27:51 <Marchael> > name
194 2011-06-30T22:28:16 <Marchael> seems like I use old name for this item
195 2011-06-30T22:29:00 <Marchael> perhaps I should use "Trash" + name
196 2011-06-30T22:29:07 <Marchael> "Trash/" + name
197 2011-06-30T22:29:14 <ThomasWaldmann> ?
198 2011-06-30T22:29:40 <ThomasWaldmann> maybe first get clear about what you are doing
199 2011-06-30T22:29:54 <ThomasWaldmann> then, if there are problems, think about a good question
200 2011-06-30T22:30:00 <ThomasWaldmann> and then ask me
201 2011-06-30T22:39:57 <Marchael> ThomasWaldmann: I don't understand why item change own name. First when I move (name, rev_no) to create_documents it works perfect, but when I try to get access to other revision meteadata with
202 2011-06-30T22:39:57 <Marchael> > storage_rev = backend.get_item(name).get_revision(rev_no)
203 2011-06-30T22:39:57 <Marchael> it raises NoSuchItemError
204 2011-06-30T22:42:55 <ThomasWaldmann> btw, how often did I tell already that one needs a traceback for a good diagnosis?
205 2011-06-30T22:45:49 <Marchael> ThomasWaldmann: there is http://paste.pocoo.org/show/425158/
206 2011-06-30T22:52:35 <ThomasWaldmann> name = item.get_revision(0)[NAME]
207 2011-06-30T22:52:44 <ThomasWaldmann> i think it is because of this
208 2011-06-30T22:52:56 <Marchael> then how I should get item name?
209 2011-06-30T22:53:15 <ThomasWaldmann> try -1 instead of 0
210 2011-06-30T22:53:31 <ThomasWaldmann> (or current rev)
211 2011-06-30T22:55:10 <Marchael> ok, working, thx
212 2011-06-30T22:55:37 <Marchael> but why I'm need trashed items?
213 2011-06-30T22:55:41 <Marchael> in indexes
214 2011-06-30T22:56:14 <ThomasWaldmann> you don't need to care about this right now
215 2011-06-30T22:56:25 <Marchael> ok
216 2011-06-30T22:56:26 <ThomasWaldmann> we can fine tune that later, if needed
217 2011-06-30T22:57:08 <Marchael> now switching to writers
218 2011-06-30T22:57:41 <Marchael> why BufferedWriter is bad for indexing at moin runtime?
219 2011-06-30T22:57:53 <Marchael> I read docs, I looks pretty fine
220 2011-06-30T22:58:04 <Marchael> but it should be open all the time
221 2011-06-30T22:58:15 <Marchael> *it looks
222 2011-06-30T23:02:56 <ThomasWaldmann> you buffer stuff and then moin gets terminated
223 2011-06-30T23:03:12 <ThomasWaldmann> what's with your buffered stuff then?
224 2011-06-30T23:03:26 <Marchael> why moin terminated?
225 2011-06-30T23:03:48 <Marchael> it depends from buffer size?
226 2011-06-30T23:03:50 <ThomasWaldmann> it doesn't matter why, it happens
227 2011-06-30T23:04:30 <ThomasWaldmann> (one reason can be that mod-wsgi decides to kill the moin process because it has reached n requests)
228 2011-06-30T23:05:35 <Marchael> hm, but it can flush information on disk
229 2011-06-30T23:05:57 <ThomasWaldmann> how? when?
230 2011-06-30T23:06:23 <Marchael> http://packages.python.org/Whoosh/api/writing.html#whoosh.writing.BufferedWriter
231 2011-06-30T23:06:32 <Marchael> > BufferedWriter(myindex, period=120, limit=100)
232 2011-06-30T23:07:09 <Marchael> it flushing on disk each 2 mins
233 2011-06-30T23:07:57 <ThomasWaldmann> that does not solve the problem
234 2011-06-30T23:11:07 <Marchael> then AsyncWriter
235 2011-06-30T23:12:28 <Marchael> may be add safe option?
236 2011-06-30T23:12:44 <Marchael> so wiki admin can choose whick writer he'll use
237 2011-06-30T23:13:06 <ThomasWaldmann> how does AsyncWriter fix the problem?
238 2011-06-30T23:13:13 <Marchael> fast and chance to loose some indexes, or safe but a bit slow
239 2011-06-30T23:14:04 <Marchael> s/loose/lose/
240 2011-06-30T23:14:19 <ThomasWaldmann> how does AsyncWriter fix the problem?
241 2011-06-30T23:15:56 <Marchael> If moin will terminate accidentally, then wiki lose not so many documents as with BufferedWriter
242 2011-06-30T23:16:19 <Marchael> or you suggest me to writer own index writer?
243 2011-06-30T23:17:06 <Marchael> *write
244 2011-06-30T23:17:13 <ThomasWaldmann> termination is not an accident, it happens regularly
245 2011-06-30T23:20:07 <Marchael> then we should use normal wrier and alghorithm for lock monitoring
246 2011-06-30T23:20:19 <Marchael> *algorithm
247 2011-06-30T23:20:47 <RogerHaase> ThomasWaldmann: I updated transclusions patch again: http://codereview.appspot.com/4535086
248 2011-06-30T23:21:13 <ThomasWaldmann> RogerHaase: I'll try to review it tommorrow
249 2011-06-30T23:22:05 <ThomasWaldmann> Marchael: design that algorithm on EP
250 2011-06-30T23:22:23 <Marchael> ok, but first I sleep some hours
251 2011-06-30T23:22:35 * Marchael bbl/gn
252 2011-06-30T23:23:41 <ThomasWaldmann> gn Marchael
253 2011-06-30T23:25:49 <RogerHaase> ThomasWaldmann: OK. btw, avoid doing a diff between 8 and 9 -- I did 8 on a windows PC and it was obviously at a different level so a couple of other changesets show up.
254 2011-06-30T23:29:29 <ThomasWaldmann> ok
255 2011-06-30T23:31:33 *** grzywacz
256
