BibTeX Parser (version 1.5.1)

MoinMoin processor for BibTeX entries.

Copyright (C) 2004-2007 Jun Hu <j.hu@tue.nl>

Copyright (C) 2004 Alexandre Duret-Lutz <adl@gnu.org>

This module is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation;

Purpose

This module will run BibTeX blocks such as the following through bibtex2html.

{ { {#!bibtex
@Book{aho.74,
  author= {Alfred V. Aho and John E. Hopcroft and Jeffrey D. Ullman},
  title = {The Design and Analysis of Computer Algorithms},
  publisher= {Addison-Wesley},
  year  = {1974},
}
} } }

Several BibTeX entries can be listed into the same block (and they can \cite each other). The 'comment' element can be used to display a comment below each entry.

HlinsDatabase

If your wiki contains a page named HlinsDatabase, then the output of bibtex2html will be processed to add hyperlinks, using the contents of HlinsDatabase as input. It will search for:

• first level bullet lists that are formatted as wiki external links:

   * [url name]

• first level bullet lists that contains a '=':

   * name = url

• and first level description lists

   name:: url

to build a mapping list.

For example, if there is a line

 * [http://www.google.com Google]

or

 * Google = http://www.google.com

or

 Google:: http://www.google.com

All the appearance of 'Google' in the bibtex2html output will be replaced with '<a href="http://www.google.com">Google</a>'. The white spaces around mapping pairs will be stripped.

HTML tags are not allowed to appear in the keywords to prevent abuse.

URL links and moin attachments

Bibtex2html uses some additional fields such as "ftp", "http", "url", "ps" ,"dvi", "rtf", "pdf", "documenturl", "urlps" and "urldvi" in bibtex database. Web links are automatically inserted. The link name depends on the nature of the link:

• it is the file suffix, whenever this suffix is .dvi, .ps, .pdf,
• rtf, .txt or .html, possibly followed by a compression sufix,
• gz, .Z or .zip;
• otherwise the name of the link is either http or ftp depending on the protocol.

This parser also supports moin attachment links in these fields. for example:

{ { {#!bibtex  abstract=off usekeys=off keywords=off style=alpha
@INPROCEEDINGS{Hu+JanseETAL-UserEvalDistInte:05,
  author    = {Hu, J. and Janse, M.D. and Kong, H.},
  title     = {User Evaluation on a Distributed Interactive Movie},
  booktitle = {HCI International 2005 (accepted)},
  year      = {2005},
  pdf       = {attachment:HCII2005.pdf},
  ps        = {attachment:HCII2005.ps.gz},
}
} } }

Parameters

style = [plain|unsrt|alpha|abbrv]

default style is plain.

hlinsdb = <WikiPage>

<WikiPage> will be used as input, instead of the default HlinsDatabase

keywords = [on|off]

whether to show keywords (if any),

default is on

abstract = [on|off]

whether to show abstract (if any),

default is on

usekeys = [on|off]

whether to use the cite keys from the BibTeX input file (and not the ones generated by the BibTeX

style file). default is on.

Download and installation

• bibtex2html: http://www.lri.fr/~filliatr/bibtex2html/

• ParserMarket/OldParsers/bibtex/bibtex.py. Please refer to MoinMoin documentation about how

  • to install a paser.
• The locations of bibtex2html and bibtex need to be configured.

  • Please add the following lines in wiki configuration:

       bibtex_bibtex = '/path/to/bibtex'
       bibtex_bibtex2html = '/path/to/bibtex2html'

    Sometime it is also necessary to specify where the BibTeX style files (.bst) are located. Then it is necessary to add a line in

    wiki configuration:

       bibtex_bstfiles = '/path/to/bstfiles'

    Also see HelpOnConfiguration.

Examples

• Formatted references:

  • http://spot.lip6.fr/wiki/SpotReferences

  • http://wiki.id.tue.nl/JunHu/Publications

• hyperlink database:

  • http://wiki.id.tue.nl/HlinsDatabase

Revision history

• 1.5.1 By JunHu

  • Thanks to Joachim Selke (http://www.joachim-selke.de) again for the suggestion of moving the configuration lines in the code to wiki configuration. According to Joachim, "This should simplify updates of bibtex.py a lot."

  1.5.0 By JunHu

  • Version number jumps to 1.5.0 serials. Nothing really special about this jump, just to indicate that this parser works well

    in MoinMoin 1.5.

    Thanks to Joachim Selke (http://www.joachim-selke.de) for the ideas to improve the parser (quote from the email from Joachim):

  • The documentation says that attachments can be included e.g. by using

    • the line pdf = {attachment:foo.pdf}. Unfortunately, on my system this does not work. I had to use pdf = {./attachment:foo.pdf}. Thus, I suggest the following change to bibtex.py: Replace line 208 by if not url.startswith('attachment:'): Replace line 211 by url = url[11:]

  • I use MoinMoin on a server where no TeX system is installed and where

    • I am not allowed to install it. Therefore, I had to put the binaries for bibtex and bibtex2html somewhere in my server directory and add their locations to the lines 177 and 178 in bibtex.py. But this was not enough to get bibtex to work. I also had to specify where the BibTeX style files (.bst) are located. This can be set using the environment variable BSTINPUTS. Thus, I suggest the following additions to bibtex.py: Insert at line 179: bstfiles = '/path/to/bstfiles' Insert at line 246:

      os.environBSTINPUTS = bstfiles

    Thank you, Joachim.

  1.3.4 By JunHu

  • Added possibilities to refer to moin attachments in bibtex2html additional field whose name is "ftp", "http", "url", "ps" , "dvi", "rtf", "pdf", "documenturl", "urlps" or "urldvi". Limitation: Only the attachments attached to the same page is allowed.

  1.3.3-3 By JunHu

  • ReimarBauer had got some new ideas - formatting the link database items as wiki external links in first level bullet lists. Thank you very much for the nice idea and the example code.

  1.3.3-2 By JunHu

  • ReimarBauer suggested to use the MoinMoin convention for hyperlink database formatting. "In my eyes it is not good to have special list rules to remember. ". I agree. So the old formatting rules from hlins and the ones introduced in 1.3.3 are dropped in this version. However, in my eyes, a description list does not look bad either, so we will use first level bullet lists, and first level description lists as well.

  1.3.3-1 By JunHu

  • This version does the hyperlink substitution by itself, and hlins is not used anymore.

  1.3.0 By JunHu, Rewritten for MoinMoin 1.3 as a parser. This version

  • has introduced some parameters. Known bugs:
  • For some strange reasons, hlins has problems with blank lines in

    HlinsDatabase. Please notice that if anything went wrong during processing, you would possibly get an empty reslut.

    hlins: http://www.lsv.ens-cachan.fr/~treinen/hlins/

  1.2.0 By Alexandre Duret-Lutz

Known Bugs

• This parser does not render CJK characters correctly - it might be just a limitation of bibtex.

Discussion

Hi Jun, very useful parser, I would have 2 suggestions:

• is it possible to have next to the bibtex some buttons "bibtex, "abstract", "pdf" that would allow to show or not the corresponding entry (I wished to be able to show the raw bibtex entry without the people having to click on "see raw text)
• I use also the latex parser... how difficult is it to use both?

thanks again

Dear Jun

in the actual version there are no formatting used in HlinsDatabase

Howard Barringer = http://www.cs.man.ac.uk/fmethods/people/howard.html
Roderick Bloem = http://vlsi.colorado.edu/~rbloem/
Edmund M. Clarke = http://www-2.cs.cmu.edu/~emc/
Costas Courcoubetis = http://www.aueb.gr/users/courcou/
Jean-Michel Couvreur = http://www.labri.fr/~couvreur
Edsger Wybe Dijkstra = http://www.cs.utexas.edu/users/EWD/

I would prefer if this could be changed to list items

• Howard Barringer = http://www.cs.man.ac.uk/fmethods/people/howard.html

• Roderick Bloem = http://vlsi.colorado.edu/~rbloem/

• Edmund M. Clarke = http://www-2.cs.cmu.edu/~emc/

• Costas Courcoubetis = http://www.aueb.gr/users/courcou/

• Jean-Michel Couvreur = http://www.labri.fr/~couvreur

• Edsger Wybe Dijkstra = http://www.cs.utexas.edu/users/EWD/

  • The updated version does what you want But instead of a bullet list, a description list can be used:

 Howard Barringer:: http://www.cs.man.ac.uk/fmethods/people/howard.html
 Roderick Bloem:: http://vlsi.colorado.edu/~rbloem/
 Edmund M. Clarke:: http://www-2.cs.cmu.edu/~emc/
 Costas Courcoubetis:: http://www.aueb.gr/users/courcou/
 Jean-Michel Couvreur:: http://www.labri.fr/~couvreur
 Edsger Wybe Dijkstra:: http://www.cs.utexas.edu/users/EWD/

• (also notice that a white space ' ' can be used to match &nbsp;.) Then it will have some formatting:

Howard Barringer

http://www.cs.man.ac.uk/fmethods/people/howard.html

Roderick Bloem

http://vlsi.colorado.edu/~rbloem/

Edmund M. Clarke

http://www-2.cs.cmu.edu/~emc/

Costas Courcoubetis

http://www.aueb.gr/users/courcou/

Jean-Michel Couvreur

http://www.labri.fr/~couvreur

Edsger&nbsp;Wybe Dijkstra

http://www.cs.utexas.edu/users/EWD/

• -- JunHu 2005-03-09 22:08:27

  In the latest version, both first level bullet lists and first level description lists are supported. -- JunHu 2005-03-10 10:29:39

I don't understand why this code works and yours not. I have removed all my changes and now your code works too. Probably it was the flush on the pipe I don't know. Because I am not sure already missing something I better put the whole code on this page. -- ReimarBauer 2005-03-07 20:16:04

• Hi Reimar, thank you very much for your effort and help. But unfortunately it does not work on my machine. The LaTeX parser does not work either - it says something like "fail to remove /tmp/tmpAFAad...". both of them worked perfectly some time before (on moinmoin 1.3.1, and possbily on 1.3.3).

  In bibtex.py, yours and mine, everything works as expected until tmp = '\n'.join(pout.readlines()). the pipe does not work. tried the command and the input in a terminal console as a normal user, it worked. Wrote a small python program to simulate the behavior and run it from the command line, it also worked.

  • That's was the same behaviour in my standalone wiki which I use regular to test new things. There was one more thing which I haven't described. Sometimes during my trials there were bibtex processes hanging and they blocked to break down the local moin session. So I have killed all moin and bibtex processes. This could be a pretty good explanaition why afterwards the pipe works. -- ReimarBauer 2005-03-08 08:20:08

  Then I tried a simple command instead of bibtex2html, for example pin, pout = popen2('cat'), it should copy the bibtex source to the output - it does when I try directly from the command line, but the output is emply when from the cgi. I guess it might be a particular problem of my linux and appache.

  • Problem solved! it is indeed a problem of my bibtex and latex installation. few days ago I replaced tetex package with texlive, the default path to bibtex and latex has changed to somewhere else instead of the default /usr/bin. bibtex2html in our code just could not find where bibtex is. By build symbolic links in /usr/bin to bibtex and latex executables, problem disappeared. -- JunHu 2005-03-09 01:11:54

  W.r.t formatting in HlinsDatabase, I have been thinking of it - hlins takes a text file in a particular format as a -db parameter, if we want to use list items in HlinsDatabase, a tmp file has to be created to reformat the input and feed that parameter - I am no python programmer, it seems to me too much of hassle. -- JunHu 2005-03-07 23:55:04

  • Check out the updated version. -- JunHu 2005-03-09 21:49:27

    • Fine I'd looked into the macro RandomQuote.py. That's the part to separate the list items:

      # this selects lines looking like a list item
      # !!! TODO: make multi-line quotes possible (optionally split by "----" or something)
      quotes = raw.splitlines()
      quotes = [quote.strip() for quote in quotes]
      quotes = [quote[2:] for quote in quotes if quote.startswith('* ')]

      This seems not to be difficult to add for me. I feel better to use common rules for things which are type of list. In my eyes it is not good to have special list rules to remember. -- ReimarBauer DateTime(2005-03-09T22:35:25Z)

      • Agreed. Updated. -- JunHu 2005-03-10 10:29:39

        • While you have already implemented this I got another idea to make HlinsDatabase strict more wikilike e.g. {{{ * Howard Barringer

• [http://vlsi.colorado.edu/~rbloem/ Roderick Bloem]

• [http://www-2.cs.cmu.edu/~emc/ Edmund M. Clarke]

• [http://www.aueb.gr/users/courcou/ Costas Courcoubetis]

• [http://www.labri.fr/~couvreur Jean-Michel Couvreur]

}}} I am sorry sometimes I got ideas step by step. So here is the code of the formatter. -- ReimarBauer DateTime(2005-03-12T11:34:10Z)

•     def format(self, formatter):
  
          lines = self.raw.split('\n')
          output =''
  
          settings = '-s ' + self.style
  
          if not self.abstract:
             settings = settings + ' -noabstract'
          if not self.keywords:
             settings = settings + ' -nokeywords'
          if self.usekeys:
             settings = settings + ' -use-keys'
  
          try:
             all = '\n'.join(lines).strip()
  
             pin, pout, perr = os.popen3('%s -c %s %s %s ' % (bibtex2html, bibtex, param, settings))
             pin.write(all.encode('utf-8'))
             pin.flush()
             pin.close()
             output = '\n'.join(pout.readlines())
             pout.close()
             perr.close()
          except IOError:
             output = '\n'.join(perr.readlines())
             pin.close()
             pout.close()
             perr.close()
             self.out.write(formatter.preformatted(1)+ output+ formatter.preformatted(0))
             return
  
          pagename = self.hlinsdb
          page = Page(self.request, pagename)
          raw = page.get_raw_body()
          if not self.request.user.may.read(pagename):
             raw = ""
  
          quotes = raw.splitlines()
          quotes = [quote.strip() for quote in quotes]
          quotes = [quote[2:] for quote in quotes if quote.startswith('* ')]
  
  
          #process the hyperlinks db
  
          if output!='':
  
             for line in quotes:
                 link,name=line[1:-1].split(' ',1)
                 name=name.strip()
                 link=link.strip()
                 url=formatter.url(1, link)+' ' +name   + formatter.url(0)
                 output = re.sub(str(name),str(url),output)
  
  
          # To please Springer-Verlag.
          output = output.replace('Springer-Verlag', '©Springer-Verlag')
  
  
          self.request.write(formatter.rawHTML(wikiutil.decodeUnknownInput(output)))

  • Thank you very much for the nice idea and the example code. it is now implemented in the latest version. -- JunHu 2005-03-17 11:00:55

I really find it very useful if I can put the bibtex data in a page, and the bibtex macro sorts the entries (according to the style parameter, etc.) upon save. How do I get this functionality? -- MuseTalk 2006-05-09 06:04:45