1 2011-06-23T01:38:31 *** waldbaer
2 2011-06-23T01:38:34 <waldbaer> hi
3 2011-06-23T01:42:59 *** MattMaker
4 2011-06-23T01:49:42 *** balu
5 2011-06-23T01:51:42 *** waldbaer
6 2011-06-23T03:59:30 *** MattMaker
7 2011-06-23T04:14:48 *** waldbaer
8 2011-06-23T04:17:15 *** balu
9 2011-06-23T05:26:29 <Marchael> moin
10 2011-06-23T05:39:05 <Marchael> ThomasWaldmann: I've found why this issue appears
11 2011-06-23T05:40:06 <Marchael> For documet with name "Home" at first uuid is "Home", but then he changing to "352138cdc6704fcda9e2cdfccb070111"
12 2011-06-23T05:40:17 <Marchael> uuid is changed
13 2011-06-23T05:40:43 <Marchael> that's why I can't found revision in backend
14 2011-06-23T05:40:58 <Marchael> I assume what I've one uuid for all revision of doc
15 2011-06-23T05:42:21 <Marchael> May use some immutable for all revisions, line "name"?
16 2011-06-23T05:42:28 <Marchael> *like
17 2011-06-23T05:43:04 <Marchael> s/some/something
18 2011-06-23T06:45:55 *** xjjk
19 2011-06-23T06:46:01 *** xjjk
20 2011-06-23T07:45:01 <Marchael> seems now work
21 2011-06-23T07:45:28 <Marchael> at least it won't add documents what already existed :D
22 2011-06-23T08:28:58 <dreimark> Marchael: uuid should never change
23 2011-06-23T08:29:09 <dreimark> please file a bug report to the issue tracker
24 2011-06-23T08:29:14 <dreimark> of m-2.0
25 2011-06-23T08:29:52 <dreimark> that may be because it was serialized some time ago
26 2011-06-23T08:31:07 <Marchael> dreimark: I mean what revisions of one document may have different uuid, uuid of revision is never change
27 2011-06-23T08:31:49 <Marchael> It's appears only for default documents what shipped with moin2
28 2011-06-23T08:32:01 <Marchael> like "Home","rst", "creole" etc.
29 2011-06-23T08:32:13 <Marchael> they have uuid as same as name
30 2011-06-23T08:32:26 <Marchael> but when I change it, uuid is changed too
31 2011-06-23T08:32:36 <dreimark> Marchael: as i said they are installed from some old serialized items
32 2011-06-23T08:32:49 <dreimark> that is wrong
33 2011-06-23T08:33:33 <Marchael> ok, I report issue
34 2011-06-23T08:36:06 <dreimark> the bug is that it never should change the uuid.
35 2011-06-23T08:36:15 <dreimark> but it did for that items
36 2011-06-23T08:37:29 <dreimark> Marchael: ^
37 2011-06-23T08:37:38 <Marchael> ok
38 2011-06-23T08:43:01 <Marchael> https://bitbucket.org/thomaswaldmann/moin-2.0/issue/33/uuid-is-changing-for-default-documents
39 2011-06-23T08:43:20 <Marchael> If I can improve it, just say what need to do
40 2011-06-23T08:49:33 *** sinha
41 2011-06-23T08:56:12 <Marchael> dreimark: content of reverted revisions is duplicating?
42 2011-06-23T09:02:40 <ThomasWaldmann> moin
43 2011-06-23T09:02:53 <Marchael> hi ThomasWaldmann
44 2011-06-23T09:03:00 <ThomasWaldmann> Marchael: you don't have to care about that
45 2011-06-23T09:03:58 <Marchael> For now I using name for creating revisions list
46 2011-06-23T09:04:04 <Marchael> *name field
47 2011-06-23T09:05:11 <ThomasWaldmann> yes, so?
48 2011-06-23T09:05:51 <Marchael> script is working for some cases now, but I'm forgot what revision can be reverted.
49 2011-06-23T09:06:09 <Marchael> and we have same document with new rev_no
50 2011-06-23T09:06:16 <Marchael> s/document/revision/
51 2011-06-23T09:07:16 <ThomasWaldmann> we have to think about timestamps i think
52 2011-06-23T09:07:28 <Marchael> but my script checking only mtime, and if he assumes what different revisions with same mtime are equal
53 2011-06-23T09:07:31 <ThomasWaldmann> ehrm, or just use something different
54 2011-06-23T09:07:46 <Marchael> s/if//
55 2011-06-23T09:07:48 <ThomasWaldmann> because you are using timestamps to detect change
56 2011-06-23T09:07:55 <Marchael> нуы
57 2011-06-23T09:07:56 <Marchael> yes
58 2011-06-23T09:08:06 <ThomasWaldmann> you can also detect change by looking at the hash
59 2011-06-23T09:08:24 <Marchael> but revision hasn't hash field
60 2011-06-23T09:08:29 <Marchael> *schema
61 2011-06-23T09:08:49 <ThomasWaldmann> or revno
62 2011-06-23T09:09:02 <ThomasWaldmann> we have to thoroughly think this through
63 2011-06-23T09:09:12 <Marchael> so I should check for mtime and rev_no?
64 2011-06-23T09:09:16 <ThomasWaldmann> timestamp is a bad idea, that i can tell you right now
65 2011-06-23T09:09:43 <Marchael> hash is good, but then we need to extend schema
66 2011-06-23T09:10:26 <ThomasWaldmann> yup
67 2011-06-23T09:10:37 <ThomasWaldmann> no, hash alone is not enough
68 2011-06-23T09:10:48 <ThomasWaldmann> because it is a data hash, not covering meta
69 2011-06-23T09:12:13 <Marchael> hash + rev_no?
70 2011-06-23T09:13:09 <Marchael> or script should calculate revision hash and put it to index
71 2011-06-23T09:13:51 <Marchael> and next we checking, calcute this for item and compare with hash in index_revision
72 2011-06-23T09:16:32 <dreimark> we need a hash for the whole item
73 2011-06-23T09:17:00 <ThomasWaldmann> no
74 2011-06-23T09:17:00 <dreimark> because you also can only change meta which creates a new revision
75 2011-06-23T09:18:13 <Marchael> calcultaing hash for large amount of documents may be not good idea
76 2011-06-23T09:19:27 <ThomasWaldmann> Marchael: use revno for now
77 2011-06-23T09:19:39 <Marchael> ok
78 2011-06-23T09:19:48 <dreimark> we can use the hash we have already now
79 2011-06-23T09:20:00 <dreimark> we need only to combine it with a hash for meta
80 2011-06-23T09:20:14 <ThomasWaldmann> dreimark: where do you want to store the hash for meta?
81 2011-06-23T09:20:22 <ThomasWaldmann> metameta? :)
82 2011-06-23T09:21:58 <ThomasWaldmann> at runtime and at indexing time, we can rely on new stuff getting a new, different (increasing) revno
83 2011-06-23T09:22:21 <ThomasWaldmann> the only exception is if we renumber revisions, but we can require index rebuild after such rare occasions
84 2011-06-23T09:27:45 <dreimark> hmm can't we use the object identifier of the item itselfs?
85 2011-06-23T09:29:12 <ThomasWaldmann> dreimark: you mean uuid?
86 2011-06-23T09:30:12 <dreimark> id(item)
87 2011-06-23T09:30:22 <ThomasWaldmann> ???
88 2011-06-23T09:30:57 <dreimark> i am thinking about if we can have always using object ids for items
89 2011-06-23T09:36:58 <dreimark> Return the “identity” of an object. This is an integer (or long integer) which is guaranteed to be unique and constant for this object during its lifetime.
90 2011-06-23T09:37:40 <dreimark> that means we would need to announce all already existing ids after a restart
91 2011-06-23T09:40:17 <dreimark> so that we don't get overlap.
92 2011-06-23T09:48:39 <ThomasWaldmann> id() is dealing with python objects in memory
93 2011-06-23T09:53:05 <dreimark> .oO online updates
94 2011-06-23T09:55:46 *** MattMaker
95 2011-06-23T09:57:20 *** MattMaker
96 2011-06-23T09:57:30 <ThomasWaldmann> dreimark: i think I'll wait some more days until upgrading to ff5
97 2011-06-23T10:27:26 <Marchael> whoosh has strage searching alg
98 2011-06-23T10:27:29 <Marchael> http://paste.pocoo.org/show/417330/
99 2011-06-23T10:28:06 <Marchael> there is iterating over all revisions
100 2011-06-23T10:28:29 <Marchael> and I don't understand why 8 at top, and 9 at bottom
101 2011-06-23T10:31:30 <Marchael> ThomasWaldmann: can you advice me a way how to solve it? sorting bt rev_no?
102 2011-06-23T10:31:52 <Marchael> *by
103 2011-06-23T10:40:13 <dreimark> that id idea is problematic if the number space could not be selected and it can become evil
104 2011-06-23T10:40:42 <dreimark> for a moment i thought that would be very easy
105 2011-06-23T11:01:36 <Marchael> ThomasWaldmann: I'm done with script(at least it working for my tests), please review http://codereview.appspot.com/4539114/diff/52005/MoinMoin/script/maint/update_indexes.py
106 2011-06-23T11:02:19 *** greg_f
107 2011-06-23T11:02:56 <Marchael> btw, codereview shows diff, but I commtied almost code before pasting.
108 2011-06-23T11:04:23 <Marchael> btw, there is full version http://paste.pocoo.org/show/417343/
109 2011-06-23T11:20:15 <ThomasWaldmann> Marchael: you should not assume order of search results, except if you made something to create order
110 2011-06-23T11:20:40 <ThomasWaldmann> (same is true for using some other iteration over indexed stuff)
111 2011-06-23T11:21:11 <Marchael> that's why I use sorted()
112 2011-06-23T11:21:37 <ThomasWaldmann> for?
113 2011-06-23T11:22:21 <Marchael> do you see http://paste.pocoo.org/show/417330/ ?
114 2011-06-23T11:22:43 <ThomasWaldmann> that is just showing some result
115 2011-06-23T11:22:46 <Marchael> If order will be random I can't compair revisions
116 2011-06-23T11:25:20 <ThomasWaldmann> hmm, maybe you could just compute a set difference of revno list storage minus revno list index?
117 2011-06-23T11:27:06 <Marchael> mmm, but how we can consider revisions order?
118 2011-06-23T11:27:54 <ThomasWaldmann> you would not consider order then
119 2011-06-23T11:28:25 <ThomasWaldmann> but you basically just want to know what you need to add and remove from the index, right?
120 2011-06-23T11:28:35 <Marchael> yup
121 2011-06-23T11:31:07 <ThomasWaldmann> and for the revno list, you do not even need to "get" the revision, you just need to get the list of the revnos
122 2011-06-23T11:31:25 <Marchael> If I'll using your way, then how I'll know which documents was deleted?
123 2011-06-23T11:32:40 <ThomasWaldmann> you mean destroyed revisions?
124 2011-06-23T11:32:45 <Marchael> yes
125 2011-06-23T11:33:16 <ThomasWaldmann> then the revno won't be in the storage revno list, but in the index revno list
126 2011-06-23T11:33:24 <ThomasWaldmann> s/list/set/
127 2011-06-23T11:33:35 <ThomasWaldmann> so it is a set difference also
128 2011-06-23T11:34:19 <ThomasWaldmann> btw, same thing for item names, you could also use set differences there
129 2011-06-23T11:56:36 *** MattMaker
130 2011-06-23T12:53:02 *** pkumar
131 2011-06-23T13:19:37 *** xorAxAx
132 2011-06-23T13:20:18 *** xorAxAx
133 2011-06-23T14:08:04 *** ThomasWaldmann
134 2011-06-23T14:08:54 *** ThomasWaldmann
135 2011-06-23T14:08:54 *** ThomasWaldmann
136 2011-06-23T14:11:22 *** sinha
137 2011-06-23T14:43:56 *** pkumar
138 2011-06-23T14:54:09 * Marchael dreaming about unique id for each revision in indexes
139 2011-06-23T14:55:46 <Marchael> for identifying document, whoosh using id which storest in backend.
140 2011-06-23T14:56:02 <Marchael> *stores
141 2011-06-23T14:56:41 <Marchael> if I can put that id to schema, I could remove documents without searching in backend
142 2011-06-23T14:57:15 <Marchael> ThomasWaldmann: what ypu think?
143 2011-06-23T15:22:53 <Marchael> if we'll use multi writer for removing revisions then we need to choose a function what will do it. delete_by_term() accept only one field, delete_by_quey() allows queries, but I played with whoosh Multi Parser and can say what it not that thing what we want
144 2011-06-23T15:25:39 <Marchael> and delete_document() removes documents by internal id, but I'm not sure what multi writer suppor that.
145 2011-06-23T15:30:03 <Marchael> yes, it doesn't support
146 2011-06-23T16:27:45 *** m4k3r
147 2011-06-23T17:03:30 *** Marchael
148 2011-06-23T18:02:23 *** xorAxAx
149 2011-06-23T18:04:03 *** xorAxAx
150 2011-06-23T18:04:42 *** greg_f
151 2011-06-23T18:05:39 *** greg_f
152 2011-06-23T18:47:16 *** m4k3r
153 2011-06-23T18:48:20 *** m4k3r
154 2011-06-23T19:33:31 *** pkumar
155 2011-06-23T19:37:55 <pkumar> ThomasWaldmann : hi, I was just wondering if you could suggest me, which parts of current tests may possibly have complex structure/code and should be simplified or can be done in a more efficient way.
156 2011-06-23T20:14:13 *** MattMaker
157 2011-06-23T20:16:32 *** greg_f
158 2011-06-23T20:19:17 *** m4k3r
159 2011-06-23T22:50:50 *** pkumar
160 2011-06-23T23:21:15 *** brunomartin
161
