Description
If you generate Table-of-contents for Headings generated by Include, then the links in the TOC and the anchors of Headings mismatch.
And it seems, if you have Includes in your included pages, then the headings of the included pages of the included pages doesn't appear in the TOC.
Example
TOC:
Contents
/IncludedPageWhichIncludesAnotherPage :
Heading with mismatching anchor
text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text.
text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text.
Heading which didn't appear in TOC
text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text.
text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text.
Details
This Wiki.
Workaround
Use full paths to included pages, e.g., on page ParentPage, you'd write [[Include(ParentPage/SubPage)]], not [[Include(/SubPage)]]
Discussion
That Include/TOC code is a mess. If somebody wants it fixed, debug it yourself. 
-- ThomasWaldmann 2005-08-17 10:09:36
Here is a possible fix:
- Create the toc while formatting the page, by hooking to formatter.heading and text. Each time heading is opened, an item will be added to the toc. Then all the text will go to both heading and toc item.
- The toc will be collected and saved for all pages, in request.toc
TableOfContents macro will put a place holder into the page: <<<TableOfContentsPlaceHolder>>>
- Page will be rendered into a buffer, always
- After the page is rendered, the table of contents placeholder will be replaced by the content of the toc, and the text sent to the client.
- Only the first toc macro will place a toc, next calls will add nothing. Multiple toc on one page does not make sense.
Just for the record, this problem (or a very similar one) has returned with a vengeance in 1.9.0. Even fully specified included pages fail to be linked correctly from the TOC. You can see it for example in the second TOC on MoinMoinQuestions. See Fix for Moin 1.9 below.
text_python output
The fix will create the text_python output bellow. headings can be static items, rendered on compile time, while tocItem will be dynamic call, made when the cache is executed, adding items to a toc object.
Alt 1: adding toc hook in the formatter
Add call to tocItem to formatter.base.heading
I'm not sure it will work, as text or _text is static.
formatter.tocItem(1, 1, "id")
request.write('<h1 id="id">)
formatter.text('heading')
request.write('heading')
formatter.tocItem(0)
request.write('</h1>')
Alt 2: adding toc hook in the parser
Call formatter.tocItem from the parser, just before or after the heading
This seems to be the easiest solution.
formatter.tocItem(1, "id", "heading")
request.write('<h1 id="id">)
request.write('heading')
request.write('</h1>')
Partial Fix
This code fixes the problem with the mis-pointed anchor. Isolating the problem, I noticed that if the included page was a complete Pagename - in this example
[[Include(/IncludedPageWhichIncludesAnotherPage)]]
- produces the error and doing
[[Include(MoinMoinBugs/TableOfContentsBrokenForIncludedPages/IncludedPageWhichIncludesAnotherPage)]]
worked.
It appears that the anchor in the TableOfContents is built based on the included page name and that mismatches when you have leading slash subpages or regular expressions in the Include. Change TableOfContents.py seems to work in my 1.5.3 Wiki.
1 @@ -114,9 +114,35 @@
2 else:
3 inc_page_lines = []
4
5 - inc_page_lines = inc_page_lines + self.IncludeMacro(self.macro, match.group(1), called_by_toc=1)
6 -
7 self.process_lines(inc_page_lines, inc_pagename)
8 +
9 + # get list of pages to include
10 + inc_name = wikiutil.AbsPageName(self.macro.request, self.macro.formatter.page.page_name, match.group(1) )
11 + pagelist = [inc_name]
12 + if inc_name.startswith("^"):
13 + try:
14 + inc_match = re.compile(inc_name)
15 + except re.error:
16 + pass # treat as plain page name
17 + else:
18 + # Get user filtered readable page list
19 + pagelist = self.macro.request.rootpage.getPageList(filter=inc_match.match)
20 +
21 + # sort and limit page list
22 + pagelist.sort()
23 + sort_dir = tmp.group('sort')
24 + if sort_dir == 'descending':
25 + pagelist.reverse()
26 + max_items = tmp.group('items')
27 + if max_items:
28 + pagelist = pagelist[:int(max_items)]
29 +
30 + for inc_name in pagelist:
31 + if not self.macro.request.user.may.read(inc_name):
32 + continue
33 + self.process_lines( self.IncludeMacro(self.macro, inc_name, called_by_toc=1),
34 + inc_name)
35 +
36 else:
37 self.parse_line(line, pagename)
Another partial fix (works for regexes)
-- MaximYanchenko 2008-05-27 09:57:11
The following fix changes the Include macro behavior when it's called called_by_toc parameter. In the current version, it returns raw text (so it can be only wiki text, you can't get headings from other markups). I changed the protocol to return just the information about the headers in this case, i.e. a list of tuples like (level, id, text).
So the TableOfContents macro get the heading information directly from the Include macro.
I failed to get correct heading ids inside the Include macro, so now I just get it directly from the rendered page. I believe MoinMoin gurus can easily fix this.
Good
- No dependency on the Include macro args
- No dependency on markup of the included page
- Works for any type of inclusion (by name, by regex)
Cleaner TableOfContents.py code
Bad
All bad things come from my fault to get correct heading ids inside the Include macro without rendering the page (I'm not a MoinMoin guru, I believe there are ways to do this without rendering the page). Namely:
- The included pages are rendered twice (i.e. all macros will be called twice, this may cause problems if macros have side-effects like running programs etc)
Include macro is called several times (as in the old version), so you need to tweak request._page_headings, otherwise all headings will get -2, -3 etc in their ids
TableOfContents should be in the beginning of the page (before included pages)
- Doesn't work for multiple inclusion of the same page (will jump to the first inclusion only).
1 --- moin-1.6.3/MoinMoin/macro/Include.py 2008-02-20 06:46:33.000000000 +0900
2 +++ pkg/x86_64/lib/python2.4/site-packages/MoinMoin/macro/Include.py 2008-05-27 12:15:42.507439000 +0900
3 @@ -32,12 +32,15 @@
4
5 _title_re = r"^(?P<heading>\s*(?P<hmarker>=+)\s.*\s(?P=hmarker))$"
6
7 +_heading_re = r'<h(?P<level>\d) id="(?P<id>[^"]+?)">(?P<title>.+?)</h(?P=level)>'
8 +
9 def extract_titles(body, title_re):
10 titles = []
11 for title, _ in title_re.findall(body):
12 @@ -169,10 +180,6 @@
13 ##result.append("*** f=%s t=%s ***" % (from_re, to_re))
14 ##result.append("*** f=%d t=%d ***" % (from_pos, to_pos))
15
16 - if called_by_toc:
17 - result.append(inc_page.get_raw_body())
18 - continue
19 -
20 if not hasattr(request, "_Include_backto"):
21 request._Include_backto = this_page.page_name
22
23 @@ -219,7 +226,7 @@
24 result.append(strfile.getvalue())
25 finally:
26 request.redirect()
27 -
28 +
29 # decrement or remove include marker
30 if this_page._macroInclude_pagelist[inc_name] > 1:
31 this_page._macroInclude_pagelist[inc_name] = \
32 @@ -237,6 +244,12 @@
33 ])
34 # XXX page.link_to is wrong now, it escapes the edit_icon html as it escapes normal text
35
36 + if called_by_toc:
37 + # find all headings in the HTML result and return them as a list of tuples (level, id, text)
38 + heading_re=re.compile(_heading_re)
39 + match = heading_re.findall( ''.join(result) )
40 + return match
41 +
42 # return include text
43 return ''.join(result)
44
45 --- moin-1.6.3/MoinMoin/macro/TableOfContents.py 2008-03-29 06:15:00.000000000 +0900
46 +++ pkg/x86_64/lib/python2.4/site-packages/MoinMoin/macro/TableOfContents.py 2008-05-27 12:10:51.572445000 +0900
47 @@ -14,21 +14,7 @@
48 #Dependencies = ["page"]
49 Dependencies = ["time"] # works around MoinMoinBugs/TableOfContentsLacksLinks
50
51 -# from macro Include (keep in sync!)
52 -_arg_heading = r'(?P<heading>,)\s*(|(?P<hquote>[\'"])(?P<htext>.+?)(?P=hquote))'
53 -_arg_level = r',\s*(?P<level>\d*)'
54 -_arg_from = r'(,\s*from=(?P<fquote>[\'"])(?P<from>.+?)(?P=fquote))?'
55 -_arg_to = r'(,\s*to=(?P<tquote>[\'"])(?P<to>.+?)(?P=tquote))?'
56 -_arg_sort = r'(,\s*sort=(?P<sort>(ascending|descending)))?'
57 -_arg_items = r'(,\s*items=(?P<items>\d+))?'
58 -_arg_skipitems = r'(,\s*skipitems=(?P<skipitems>\d+))?'
59 -_arg_titlesonly = r'(,\s*(?P<titlesonly>titlesonly))?'
60 -_arg_editlink = r'(,\s*(?P<editlink>editlink))?'
61 -_args_re_pattern = r'^(?P<name>[^,]+)(%s(%s)?%s%s%s%s%s%s%s)?$' % (
62 - _arg_heading, _arg_level, _arg_from, _arg_to, _arg_sort, _arg_items,
63 - _arg_skipitems, _arg_titlesonly, _arg_editlink)
64 -
65 -# from Include, too, but with extra htext group around header text
66 +# from Include macro, but with extra htext group around header text
67 _title_re = r"^(?P<heading>(?P<hmarker>=+)\s(?P<htext>.*)\s(?P=hmarker))$"
68
69 class TableOfContents:
70 @@ -41,7 +27,6 @@
71 self._ = self.macro.request.getText
72
73 self.inc_re = re.compile(r"^\<\<Include\((.*)\)\>\>")
74 - self.arg_re = re.compile(_args_re_pattern)
75 self.head_re = re.compile(_title_re) # single lines only
76 self.pre_re = re.compile(r'\{\{\{.+?\}\}\}', re.S)
77
78 @@ -67,7 +52,7 @@
79 if self.include_macro is None:
80 self.include_macro = wikiutil.importPlugin(self.macro.request.cfg,
81 'macro', "Include")
82 - return self.pre_re.sub('', self.include_macro(*args, **kwargs)).split('\n')
83 + return self.include_macro(*args, **kwargs)
84
85 def run(self):
86 _ = self._
87 @@ -92,31 +77,16 @@
88 match = self.inc_re.match(line)
89 if match:
90 # this is an <<Include()>> line.
91 - # now parse the included page and do the work on it.
92 -
93 - ## get heading and level from Include() line.
94 - tmp = self.arg_re.match(match.group(1))
95 - if tmp and tmp.group("name"):
96 - inc_pagename = tmp.group("name")
97 - else:
98 - # no pagename? ignore it
99 - continue
100 - if tmp.group("heading") and tmp.group("hquote"):
101 - if tmp.group("htext"):
102 - heading = tmp.group("htext")
103 - else:
104 - heading = inc_pagename
105 - if tmp.group("level"):
106 - level = int(tmp.group("level"))
107 - else:
108 - level = 1
109 - inc_page_lines = ["%s %s %s" % ("=" * level, heading, "=" * level)]
110 - else:
111 - inc_page_lines = []
112 -
113 - inc_page_lines = inc_page_lines + self.IncludeMacro(self.macro, match.group(1), called_by_toc=1)
114 -
115 - self.process_lines(inc_page_lines, inc_pagename)
116 + #
117 + # save and restore request._page_headings, otherwise all heading ids
118 + # will receive counters due to multiple IncludeMacro call
119 + saved_page_headings = self.macro.request._page_headings.copy()
120 + # get headings from the included page
121 + inc_result = self.IncludeMacro(self.macro, match.group(1), called_by_toc=1)
122 + self.macro.request._page_headings = saved_page_headings
123 + # add the headings to the TOC
124 + for h in inc_result:
125 + self.add_toc_line( int(h[0]), h[1], h[2] )
126 else:
127 self.parse_line(line, pagename)
128
129 @@ -130,8 +100,14 @@
130 self.titles.setdefault(pntt, 0)
131 self.titles[pntt] += 1
132
133 - # Get new indent level
134 - newindent = len(match.group('hmarker'))
135 + unique_id = ''
136 + if self.titles[pntt] > 1:
137 + unique_id = '-%d' % (self.titles[pntt],)
138 +
139 + self.add_toc_line( len(match.group('hmarker')), "head-" + sha.new(pntt.encode(config.charset)).hexdigest() + unique_id, title_text )
140 +
141 +
142 + def add_toc_line(self, newindent, anchor, title_text):
143 if newindent > self.maxdepth or newindent < self.mindepth:
144 return
145 if not self.indent:
146 @@ -148,20 +124,15 @@
147 self.result.append(self.macro.formatter.number_list(1))
148 self.result.append(self.macro.formatter.listitem(1))
149
150 - # Add the heading
151 - unique_id = ''
152 - if self.titles[pntt] > 1:
153 - unique_id = '-%d' % (self.titles[pntt],)
154 -
155 # close last listitem if same level
156 if self.indent == newindent:
157 self.result.append(self.macro.formatter.listitem(0))
158
159 + # Add the heading
160 if self.indent >= newindent:
161 self.result.append(self.macro.formatter.listitem(1))
162 - self.result.append(self.macro.formatter.anchorlink(1,
163 - "head-" + sha.new(pntt.encode(config.charset)).hexdigest() + unique_id) +
164 + self.result.append(self.macro.formatter.anchorlink( 1, anchor ) +
165 self.macro.formatter.text(title_text) +
166 self.macro.formatter.anchorlink(0))
167
168 # Set new indent level
Fix for Moin 1.9
RenatoSilva - This bug is not reproducible in Moin 1.8.5 but came back in 1.9.0, as reported by GreyCat. I propose the following patch for 1.9 (looks like someone forgot to update this part in uid refactoring):
1 --- MoinMoin/macro/TableOfContents.py 2009-12-19 19:49:35 +0000
2 +++ MoinMoin/macro/TableOfContents.py 2009-12-19 19:49:31 +0000
3 @@ -197,7 +197,7 @@
4 continue
5
6 # will be reset by pop_unique_ids below
7 - macro.request.include_id = incl_id
8 + macro.request.uid_generator.include_id = incl_id
9
10 need_li = lastlvl >= lvl
11 while lastlvl > lvl:
Worked for me. Thanks. -- GregWooledge a.k.a. greycat
Plan
- Priority:
- Assigned to:
Status: 1.9: fixed by http://hg.moinmo.in/moin/1.9/rev/3d098b21edb5