= Moin On PyPy Testing =

This page is related to the GCI 2011 task "test moin2 with pypy 1.7" ([[http://www.google-melange.com/gci/task/view/google/gci2011/7185227|Google Melange]], [[EasyToDo/test moin2 with pypy 1.7|EasyToDo]])

'''Note''': All benchmarks ran on a system with a 2.5 GHz dual-core CPU (AMD Athlon(tm) 7550 Dual-Core Processor) and 4 GB of RAM.

== Unit tests ==

{{attachment:unittest_time.png}}

|| '''Interpreter''' || '''time output'''                             || '''py.test output '''                                       || '''full output'''                     ||
|| CPython 2.7       || real 1m0.103s / user 0m42.757s / sys 0m1.497s || 2 failed, 859 passed, 61 skipped in 59.58 seconds           || [[attachment:py.test-cpython2.7.txt]] ||
|| !PyPy 1.6          || real 2m7.532s / user 1m49.173s / sys 0m2.093s || 9 failed, 850 passed, 61 skipped, 1 error in 126.43 seconds || [[attachment:py.test-pypy1.6.txt]]    ||
|| !PyPy 1.7          || real 2m23.673s / user 2m2.345s / sys 0m1.957s || 4 failed, 857 passed, 61 skipped in 142.51 seconds          || [[attachment:py.test-pypy1.7.txt]]    ||

== HTTP performance ==

=== Requests per second ===

{{attachment:requests_per_second.png}}

|| '''-n''' || '''-c''' || '''CPython 2.7''' || '''!PyPy 1.6''' || '''!PyPy 1.7''' ||
|| 100      || 1        || 17.62 req/s       || 4.67 req/s      || 5.31 req/s      ||
|| 500      || 5        || 17.49 req/s       || 6.60 req/s      || 8.13 req/s      ||
|| 1000     || 10       || 17.49 req/s       || 7.72 req/s      || 9.76 req/s      ||
|| 5000     || 50       || 17.40 req/s       || 9.97 req/s      || 13.63 req/s     ||
|| 10000    || 100      || 17.41 req/s       || 10.48 req/s     || 14.90 req/s     ||

Moin was (re)started before each run with the following command:

{{{
moin runserver -r -p 8080
}}}

I had to disable the Werkzeug reloader as it somehow caused segfaults on !PyPy 1.7 after a random number of requests.

Speed was measured using this command:

{{{
ab -n $n -c $c http://127.0.0.1:8080/Home
}}}

{{{/Home}}} contained 5 paragraphs of lorem ipsum. It was saved as Moin wiki syntax.

As runserver is single-threaded the {{{-c}}} (number of concurrent requests) parameter does hardly change anything but for the sake of completeness I've added it to the table.

=== Time per request ===

{{attachment:time_per_request.png}}

For !PyPy (both 1.6 and 1.7) you can see how the time decreases over the first 50 requests, probably because of the JIT.

The values were measured with the following [[http://golang.org/|Go]] program:

{{{
package main

import (
	"http"
	"time"
	"os"
	"bytes"
	"fmt"
)

const url = "http://localhost:8080/Home"

func main() {
	var time_before, time_after int64
	for i := 0; i < 400; i++ {
		time_before = time.Nanoseconds()
		resp, err := http.Get(url)
		if err != nil {
			fmt.Println(err.String())
			os.Exit(1)
		}
		var buf bytes.Buffer
		buf.ReadFrom(resp.Body)
		resp.Body.Close()
		time_after = time.Nanoseconds()
		fmt.Println(float64(time_after - time_before) / 1e7)
	}
}
}}}

== Manual testing ==

Didn't find something broken on !PyPy. Tested amongst others:

 * register/login
 * different item types (including archives)
 * history/index/... views

== Original Chart URLs ==

The charts were generated using Google's static chart API. These are the source URLs:

=== unittest_time.png ===
{{{
http://chart.apis.google.com/chart
   ?chxr=0,0,160
   &chxt=x
   &chbh=a
   &chs=480x120
   &cht=bhg
   &chco=3072F3,3D7930,FF9900
   &chd=s:X,w,2
   &chdl=CPython+2.7|PyPy+1.6|PyPy+1.7
   &chma=|0,5
   &chtt=time+to+run+unit+tests
}}}

=== req_per_sec.png ===
{{{
http://chart.apis.google.com/chart
   ?chxl=0:|5|10|15|20|25|1:|100|500|1000|5000|10000
   &chxp=0,5,10,15,20,25
   &chxr=0,0,25
   &chxt=y,x
   &chbh=a,2,10
   &chs=400x200
   &cht=bvg
   &chco=3072F3,3D7930,FF9900
   &chd=s:rrrqq,LQTYa,NUYhk
   &chdl=CPython+2.7|PyPy+1.6|PyPy+1.7
   &chp=0,0,0.005
   &chg=0,10
   &chtt=request+per+second
}}}

=== time_per_request.png ===
{{{
http://chart.apis.google.com/chart
   ?chxl=1:|0|200|400|600|800|1000
   &chxp=1,0,,,,,1000
   &chxr=0,1,400|1,0,1000
   &chxt=x,y
   &chs=800x200
   &cht=lc
   &chco=3072F3,3D7930,FF9900
   &chd=s:aDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDEDDDDEDDDDDDDDDDDEDDDDDDDDDDDEFDDDDDDDDDDDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEFDDDDDDDDDDDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDDEDDDDDDDDDDD,_TOPMSMNKKTMKKKSLLWKKNKSJIILRJJJIKIRJIJIKISIHIOQHIHIJKRIJKKITHHHHHHXHINIHQKHHHHIKQHHHKIHGHRHIKGKGRGHGHHPRHHHHHGGHHZvIGGGRHGGGGGGHQGGGJtZGHGGGGMUGGHGGGKRLGGGGGGHXJGGGGGHGQGGGGIGGGHUIGGGGGGGGGGSGGGJGGGGFGYGGHHFGGJHSGGGGKHGGHRGGGFGGGIKRGGGJFGFFGGYFGFGHFGFGVGGFIGFHJGSGKFGGRTGHJGFFGGQIFFFFGFGIRLFFFMFFRFFFFHFHFIFRFHFFFLIGFRFGHFLGFFFRGFFFPMTGFFFFHFJGFcLFFIFSFFFFIFFFFFFTHFFFHHFFGGSFFIJFFFFFFFFSFFFFFFFFGFF,_SPMXKNRJJMROJJTJLYJJSKJNOHIIJXIHWHHIHPHIGMPGIMGGPGHHJSGGQGOHGGGGGXMGIQJHFHFGGOGGHGGGGRFFHFFJYGGFHFHGPFGFIFHFFPFGFG_FGFFIQFFHFFFFPFFFFanFFFFFEISKFFGFOHEEJEEUFFFFMNEEHEGEHEEOHFGFEELQFEEEEEEEEaEFIFFQEEFEHKFNIEFEHTEFEFFEEIEOGFEEEEEEFXSEEOEHEEEEHPOEEFFEPFEEHGFFOEKFEGGOGGEGIbEHHFFREEEIGEEOGEEEEEEEGEPEGEFEEEGHEQEFHHEEEGSGEEFFLFRHEEFEEFEEQLFEEEEEHPFEETYEEEEEEEEGOEHFGEEGHEPFGEEFEFEEISJEEEEEEEEEEXHEEEEEFEE
   &chdl=CPython+2.7|PyPy+1.6|PyPy+1.7
   &chls=1|1|1
   &chtt=time+per+request+(ms)
}}}