Moin On PyPy Testing
This page is related to the GCI 2011 task "test moin2 with pypy 1.7" (Google Melange, EasyToDo)
Note: All benchmarks ran on a system with a 2.5 GHz dual-core CPU (AMD Athlon(tm) 7550 Dual-Core Processor) and 4 GB of RAM.
Unit tests
Interpreter |
time output |
py.test output |
full output |
CPython 2.7 |
real 1m0.103s / user 0m42.757s / sys 0m1.497s |
2 failed, 859 passed, 61 skipped in 59.58 seconds |
|
PyPy 1.6 |
real 2m7.532s / user 1m49.173s / sys 0m2.093s |
9 failed, 850 passed, 61 skipped, 1 error in 126.43 seconds |
|
PyPy 1.7 |
real 2m23.673s / user 2m2.345s / sys 0m1.957s |
4 failed, 857 passed, 61 skipped in 142.51 seconds |
HTTP performance
Requests per second
-n |
-c |
CPython 2.7 |
PyPy 1.6 |
PyPy 1.7 |
100 |
1 |
17.62 req/s |
4.67 req/s |
5.31 req/s |
500 |
5 |
17.49 req/s |
6.60 req/s |
8.13 req/s |
1000 |
10 |
17.49 req/s |
7.72 req/s |
9.76 req/s |
5000 |
50 |
17.40 req/s |
9.97 req/s |
13.63 req/s |
10000 |
100 |
17.41 req/s |
10.48 req/s |
14.90 req/s |
Moin was (re)started before each run with the following command:
moin runserver -r -p 8080
I had to disable the Werkzeug reloader as it somehow caused segfaults on PyPy 1.7 after a random number of requests.
Speed was measured using this command:
ab -n $n -c $c http://127.0.0.1:8080/Home
/Home contained 5 paragraphs of lorem ipsum. It was saved as Moin wiki syntax.
As runserver is single-threaded the -c (number of concurrent requests) parameter does hardly change anything but for the sake of completeness I've added it to the table.
Time per request
For PyPy (both 1.6 and 1.7) you can see how the time decreases over the first 50 requests, probably because of the JIT.
The values were measured with the following Go program:
package main
import (
"http"
"time"
"os"
"bytes"
"fmt"
)
const url = "http://localhost:8080/Home"
func main() {
var time_before, time_after int64
for i := 0; i < 400; i++ {
time_before = time.Nanoseconds()
resp, err := http.Get(url)
if err != nil {
fmt.Println(err.String())
os.Exit(1)
}
var buf bytes.Buffer
buf.ReadFrom(resp.Body)
resp.Body.Close()
time_after = time.Nanoseconds()
fmt.Println(float64(time_after - time_before) / 1e7)
}
}
Manual testing
Didn't find something broken on PyPy. Tested amongst others:
- register/login
- different item types (including archives)
- history/index/... views
Original Chart URLs
The charts were generated using Google's static chart API. These are the source URLs:
unittest_time.png
http://chart.apis.google.com/chart ?chxr=0,0,160 &chxt=x &chbh=a &chs=480x120 &cht=bhg &chco=3072F3,3D7930,FF9900 &chd=s:X,w,2 &chdl=CPython+2.7|PyPy+1.6|PyPy+1.7 &chma=|0,5 &chtt=time+to+run+unit+tests
req_per_sec.png
http://chart.apis.google.com/chart ?chxl=0:|5|10|15|20|25|1:|100|500|1000|5000|10000 &chxp=0,5,10,15,20,25 &chxr=0,0,25 &chxt=y,x &chbh=a,2,10 &chs=400x200 &cht=bvg &chco=3072F3,3D7930,FF9900 &chd=s:rrrqq,LQTYa,NUYhk &chdl=CPython+2.7|PyPy+1.6|PyPy+1.7 &chp=0,0,0.005 &chg=0,10 &chtt=request+per+second
time_per_request.png
http://chart.apis.google.com/chart ?chxl=1:|0|200|400|600|800|1000 &chxp=1,0,,,,,1000 &chxr=0,1,400|1,0,1000 &chxt=x,y &chs=800x200 &cht=lc &chco=3072F3,3D7930,FF9900 &chd=s:aDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDEDDDDEDDDDDDDDDDDEDDDDDDDDDDDEFDDDDDDDDDDDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEFDDDDDDDDDDDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDD,_TOPMSMNKKTMKKKSLLWKKNKSJIILRJJJIKIRJIJIKISIHIOQHIHIJKRIJKKITHHHHHHXHINIHQKHHHHIKQHHHKIHGHRHIKGKGRGHGHHPRHHHHHGGHHZvIGGGRHGGGGGGHQGGGJtZGHGGGGMUGGHGGGKRLGGGGGGHXJGGGGGHGQGGGGIGGGHUIGGGGGGGGGGSGGGJGGGGFGYGGHHFGGJHSGGGGKHGGHRGGGFGGGIKRGGGJFGFFGGYFGFGHFGFGVGGFIGFHJGSGKFGGRTGHJGFFGGQIFFFFGFGIRLFFFMFFRFFFFHFHFIFRFHFFFLIGFRFGHFLGFFFRGFFFPMTGFFFFHFJGFcLFFIFSFFFFIFFFFFFTHFFFHHFFGGSFFIJFFFFFFFFSFFFFFFFFGFF,_SPMXKNRJJMROJJTJLYJJSKJNOHIIJXIHWHHIHPHIGMPGIMGGPGHHJSGGQGOHGGGGGXMGIQJHFHFGGOGGHGGGGRFFHFFJYGGFHFHGPFGFIFHFFPFGFG_FGFFIQFFHFFFFPFFFFanFFFFFEISKFFGFOHEEJEEUFFFFMNEEHEGEHEEOHFGFEELQFEEEEEEEEaEFIFFQEEFEHKFNIEFEHTEFEFFEEIEOGFEEEEEEFXSEEOEHEEEEHPOEEFFEPFEEHGFFOEKFEGGOGGEGIbEHHFFREEEIGEEOGEEEEEEEGEPEGEFEEEGHEQEFHHEEEGSGEEFFLFRHEEFEEFEEQLFEEEEEHPFEETYEEEEEEEEGOEHFGEEGHEPFGEEFEFEEISJEEEEEEEEEEXHEEEEEFEE &chdl=CPython+2.7|PyPy+1.6|PyPy+1.7 &chls=1|1|1 &chtt=time+per+request+(ms)