Contents
Matplotlib parser
Having matplotlib integrated in Moin would integrate two great python packages. For scientific/engineering orientated wikis, a plotting feature is often a missing component that could add significant more value to the wiki. Similar to the integration of simple math formulars.
Parser
Below is a first matplotlib parser attempt with some special markup to use attached data files and include python source code from other pages. The implementation uses ipython TaskClient to execute the matplotlib python code (I did this in order to run the python code execution as different user that can not harm the moin instance -- can easily be changed).
Dependencies
- matplotlib
- numpy (matplotlib needs that anyway)
highlight parser ../highlight
You need to adjust the settings at the top of the source code. If you are not going to use ipython/TaskClient to run the python code, then replace the runExternally method in MplClient.
Usage
MoinMoin - Matplotlib integration
example: {{{#!mpl src=off,run=on,prefix=mpl,display=inline,klass=mpl,tablestyle="",rowstyle="",style="",persistence=True,debug=False
...
...
}}}
keyword parameters:
@param display: off|link|inline (default:inline)
@param columns: arange images in table with No columns (default:1)
if more than 1 image and display is inline
@param src: on|off (display mpl source, or not, default: off)
@param run: on|off (if off, execution is disabled, default:on)
@prefix: prefix for files generated (default: mpl). If you have more than one
image on a page which is not persistent, you should use a unique prefix
for each chart.
@params klass: class attribute for div
@params tablestyle: style attribute for table
@params rowstyle: style attribute for table row
@params style: style attribute for table cell
@params persistence: if True create delete.me.to.regenerate.images marker (default: True)
@params debug: if True keep files after processing (default: False)
Directives:
-----------
#! attach(pagename/*.xls)
#! page(pagename)
Mpl scripts can be made very modular using the '#! include(pagename)' directive.
You can see this similar to the import statement in python. In the Mpl script the
whole page content (#format python) is included. Use this to store your standard
settings and reuse them for consistent plotting.
example: #! include(MplSettings)
Data sources:
============
1) Page attachments
*******************
# --------------------------------
#! attach(pagename/*.xls)
# --------------------------------
The IN object contains all attached input files in the IN['files'] dictionary with the filenames as keys.
If you want to make a page attachment available for the plotting process,
use the '#! attach(filename)' directive and reference the file in the open
statement as shown in the example.
example:
#! attach('filename')
files = IN['files']
f = open(files['filename'])
2) Page raw data
****************
# --------------------------------
#! page(pagename)
# --------------------------------
The IN object contains the page raw data (text) in the IN['pages'] dictionary with the pagenames as keys
and the page content (as obtained from page.get_raw_body()). Absolute page names are used as keys.
example:
#! page(PageWithData)
pgData = IN['pages']['PageWithData']
3) Form data
************
The IN object also contains the form dictionary from the current request.
example: value = IN['form'].get('parname',['0'])[0]
The IN object contains also the current user name from the request in IN['user']
@copyright: 2008 F. Zieher
@license: GNU GPL, see COPYING for details.Some PRE-CODE and POST-CODE is injected automatically to set up the matplotlib environment. Python objects that must not be assigned any value because these are used within the plotting process are:
mpl: the matplotlib object
plt: the matplotlib pyplot object / use this for all your plotting commands
np: numpy
mm: utility object, mm.imgName(id) returns image name
IN: dictionary object with filename as key and full path as value
MoinMoin specific features
You can send files into the inbox with
and referene the file with
IN['files']['filename']
in file reading operations. filename can be a pattern and may reference files on a different page, e.g. "../*.xls" to select all xls files on the parent page.
You can also insert python code from another page by placing
within the python code. This will include the page content of Pagename at the selected location. Pagename must therefore only contain python code in the form (See MplSettings as example).
#format python mpl.rcParams['figure.figsize'] = (6,6/1.61803399) # use golden ratio mpl.rcParams['legend.shadow'] = True mpl.rcParams['legend.fancybox'] = True mpl.rcParams['savefig.dpi'] = 72
-- FranzZieher 2009-01-09 23:00:42
Example
Using attached data file (src=on)
The following input:
{{{#!mpl src=on
# -----------------------------------------
# -----------------------------------------
hits = np.loadtxt(IN['hits.dat'],usecols=(1,))[::-1]
plt.title('Moin server hits')
plt.plot(range(len(hits)),hits,'ro',label='Hits/day')
plt.legend()
plt.grid()
plt.savefig(mm.imgName())
}}}with the attachment hits.dat
#Date Views/day Edits/day 2008-12-03 1312.5 53.0 2008-10-30 2595.4 22.2 2008-09-26 2169.3 20.1 2008-08-23 1958.0 14.0 2008-07-20 1970.1 14.6 2008-06-16 1597.6 7.5 2008-05-14 1447.9 6.9 2008-04-10 1466.1 23.6 2008-03-07 1383.3 58.7 2008-02-02 1025.1 111.3 2007-12-30 591.8 24.7 2007-11-26 430.6 13.4 2007-10-24 487.1 33.3 2007-09-20 536.0 33.6 2007-08-17 675.0 42.6 2007-07-14 351.0 67.6 2007-06-10 376.1 26.3 2007-05-07 436.7 43.8 2007-04-04 166.3 18.5 2007-03-01 88.8 12.9 2007-01-26 108.2 13.5 2006-12-23 62.9 23.3 2006-11-19 69.5 11.3 2006-10-16 32.1 1.2 2006-09-13 52.3 3.4 2006-08-10 51.9 5.5 2006-07-07 39.1 3.8 2006-06-03 26.7 3.2 2006-04-30 62.2 12.
generates the following output:
1 # ------------------------------------------------------------------------------------
2 # Predefined objects:
3 # mpl ... matplotlib object
4 # plt ... matplotlib.pyplot object
5 # np ... numpy object
6 # IN ... dictionary containing 'files' dictionary, 'pages' dictionary,
7 # 'form' dictionary (copy of request.form) and 'user' name.
8 # - Access attached file with IN['files']['filename']
9 # - Accrss page data with IN['pages']['PageName']
10 # - Access form parameter with IN['form'].get(param,['default-value'])[0]
11 # mm ... use mm.imgName(imgno=0) and mm.nextImg() in plt.savefig
12 # examples: plt.savefig(mm.imgName()), and for all consecutive
13 # plt.savefig(mm.nextImg())
14 # ------------------------------------------------------------------------------------
15 # some plot settings from page MplSettings
16 # -----------------------------------------
17 #! include(MplSettings)
18 #! attach(hits.dat)
19 # -----------------------------------------
20
21 files = IN['files']
22
23 hits = np.loadtxt(files['hits.dat'],usecols=(1,))
24
25 plt.title('Moin server hits')
26 plt.plot(range(len(hits)),hits[::-1],'ro',label='Hits/day')
27 plt.legend()
28 plt.grid()
29 plt.savefig(mm.nextImg())
|
Data from Moin Pages
The above example can be realized by using the data from a page, i.e. HitsData
# format text #Date Views/day Edits/day 2008-12-03 1312.5 53.0 2008-10-30 2595.4 22.2 2008-09-26 2169.3 20.1 2008-08-23 1958.0 14.0 ...
and the following small mpl script
# ------------------------------
# ------------------------------
#! page(HitsData)
def getData(pageName):
from StringIO import StringIO
data = StringIO(IN['pages'][pageName])
return np.loadtxt(data,usecols=(1,))
hits = getData('HitsData')
plt.title('Moin server hits')
plt.plot(range(len(hits)),hits[::-1],'ro',label='Hits/day')
plt.legend()
plt.grid()
plt.savefig(mm.nextImg())
Released mpl-1.0.py
This release contains a number of changes.
- Uses direct connection to the ipcontroller (a seperate python process is no longer needed)
- persistence attribute, setting it False will always recreate the chart
- prefix attribute, chart prefix, set uniquely for each chart when you have more than one chart on the page
- Can use data stored on the page (see example above), chart is recreated when page data changes
I run this parser on a production site creating thousands of charts without any problems meanwhile (debugging is still a bit of a pain due to little information that is returned, so test your mpl code first in ipython/matplotlib environment 
-- FranzZieher 2009-02-19 23:10:09
Download
Copyright
@copyright: 2008-2009 by -- FranzZieher
License
This parser is released under the terms of the GNU GPL.
Discussion
Capturing stdout and rendering it nicely would be a great feature. Maybe somebody has a look at that. I'm thinking of having something like Gael Varoquaux's pyreport.
-- FranzZieher 2009-01-06 19:25:06
I had doing this integration on my todo list - what's the likelihood of being able to specify a wiki page as the source for data. I'd like to have a page /SomePlot and have it use columns from a table in /SomePlotData. -- Joel Johnson 2009-02-16 04:31:34
Dear Joel, I'll upload an update of the parser soon, with the feature of reading the data from a wiki page (I suppose you mean raw data on the page similar to the data in a text file). BTW, I have changed the parser somewhat, so that it can directly communicate with the ipcontroller (even with twisted as web server). -- FranzZieher 2009-02-16 10:10:41
Hi! The Copyright section says that the parser is released into the public domain. The License section says that the parser is released under the terms of the GNU GPL. Which is correct? -- DennisBenzinger 2009-02-16 10:30:14
Here's a version running on MoinMoin 1.9.3 DesktopMode for Unix. I replaced the whole IPython stuff which required Zope by a simple "exec". This is sufficient for the desktop use case.