Title index is a page with huge amount of links, therefor is a good test for page linking generation and listing pages.
Note: the first request is not profiled, as it is expensive and not important in long running process. We should add cgi profile also.
ab -n 11 -c 1 'http://localhost:8000/TitleIndex?allpages=1' run 11 requests, only requests 2-11 are profiled.
patch-467
ab -n 11 -c 1 'http://localhost:8000/TitleIndex?allpages=1'
Requests per second: 0.65 [#/sec] (mean)
637568 function calls (592418 primitive calls) in 28.060 CPU seconds
Ordered by: internal time, call count
List reduced from 233 to 20 due to restriction <20>
ncalls tottime percall cumtime percall filename:lineno(function)
57520/32390 12.769 0.000 17.972 0.001 Page.py:78(get_rev)
70690 3.010 0.000 3.010 0.000 posixpath.py:171(exists)
40070/20050 1.887 0.000 16.598 0.001 Page.py:120(getPagePath)
202020 1.522 0.000 1.522 0.000 posixpath.py:56(join)
6560 1.318 0.000 16.372 0.002 Page.py:562(link_to)
20 1.207 0.060 10.590 0.530 Page.py:1316(listPages)
19790 1.169 0.000 1.169 0.000 Page.py:28(__init__)
40070 1.122 0.000 1.122 0.000 wikiutil.py:154(quoteWikinameFS)
6870 1.117 0.000 1.117 0.000 wikiutil.py:113(escape)
8560 0.405 0.000 0.405 0.000 wikiutil.py:75(quoteWikinameURL)
6540 0.357 0.000 0.427 0.000 wikiutil.py:187(unquoteWikiname)
10 0.291 0.029 27.573 2.757 wikimacro.py:254(_macro_TitleIndex)
6590 0.291 0.000 0.455 0.000 wikiacl.py:185(may)
52000 0.238 0.000 0.238 0.000 wikimacro.py:264(<lambda>)
6650 0.167 0.000 0.191 0.000 wikiutil.py:772(link_tag)
6590 0.124 0.000 7.533 0.001 Page.py:1266(getACL)
6690 0.119 0.000 0.119 0.000 posixpath.py:144(getmtime)
6590 0.099 0.000 8.144 0.001 security.py:52(<lambda>)
6470 0.083 0.000 8.691 0.001 AttachFile.py:50(getAttachDir)
6460 0.082 0.000 9.127 0.001 AttachFile.py:94(getIndicator)
Ordered by: internal time, call count
List reduced from 233 to 20 due to restriction <20>
Function was called by...
Page.py:78(get_rev) Page.py:120(getPagePath)(37340) 16.598
Page.py:199(_text_filename)(6650) 7.221
Page.py:332(exists)(6800) 4.397
Page.py:1316(listPages)(6730) 10.590
posixpath.py:171(exists) AttachFile.py:94(getIndicator)(6460) 9.127
Page.py:78(get_rev)(57520) 17.972
Page.py:120(getPagePath)(50) 16.598
Page.py:332(exists)(50) 4.397
Page.py:1266(getACL)(6590) 7.533
caching.py:37(exists)(20) 0.001
Page.py:120(getPagePath) AttachFile.py:50(getAttachDir)(6470) 8.691
Page.py:78(get_rev)(13450) 17.972
Page.py:120(getPagePath)(20020) 16.598
Page.py:228(_last_edited)(20) 0.010
Page.py:332(exists)(50) 4.397
Page.py:427(getPageList)(10) 10.593
Page.py:994(send_page_content)(10) 27.666
caching.py:13(__init__)(20) 0.009
editlog.py:74(__init__)(10) 0.002
eventlog.py:13(__init__)(10) 0.001
posixpath.py:56(join) Page.py:78(get_rev)(115040) 17.972
Page.py:120(getPagePath)(80140) 16.598
Page.py:427(getPageList)(10) 10.593
Page.py:1316(listPages)(6730) 10.590
caching.py:13(__init__)(10) 0.009
caching.py:34(_filename)(70) 0.001
wikiaction.py:836(getPlugins)(20) 0.003
Page.py:562(link_to) __init__.py:172(username)(10) 0.016
__init__.py:205(splitNavilink)(40) 0.054
__init__.py:282(navibar)(10) 0.062
text_html.py:212(pagelink)(40) 0.028
wikimacro.py:254(_macro_TitleIndex)(6460) 27.573
Page.py:1316(listPages) Page.py:427(getPageList)(20) 10.593
Page.py:28(__init__) AttachFile.py:50(getAttachDir)(6470) 8.691
__init__.py:287(getText)(60) 0.035
request.py:670(run)(10) 28.060
security.py:52(<lambda>)(6590) 8.144
text_html.py:212(pagelink)(40) 0.028
wikimacro.py:254(_macro_TitleIndex)(6460) 27.573
wikiutil.py:432(getSysPage)(150) 0.083
wikiutil.py:909(send_title)(10) 0.270
wikiutil.py:154(quoteWikinameFS) Page.py:120(getPagePath)(40070) 16.598
wikiutil.py:113(escape) Page.py:562(link_to)(6560) 16.372
__init__.py:150(title)(10) 0.001
__init__.py:536(searchform)(10) 0.001
base.py:90(image)(240) 0.005
text_html.py:251(url)(30) 0.001
wikiutil.py:909(send_title)(20) 0.270
wikiutil.py:75(quoteWikinameURL) AttachFile.py:94(getIndicator)(60) 9.127
Page.py:562(link_to)(6560) 16.372
Page.py:662(send_page)(10) 28.045
__init__.py:131(logo)(10) 0.006
__init__.py:910(editbar)(10) 0.108
wikimacro.py:43(<lambda>)(870) 0.023
wikimacro.py:254(_macro_TitleIndex)(900) 27.573
wikiutil.py:909(send_title)(140) 0.270
wikiutil.py:187(unquoteWikiname) Page.py:1316(listPages)(6540) 10.590
wikimacro.py:254(_macro_TitleIndex) wikimacro.py:98(execute)(10) 27.592
wikiacl.py:185(may) security.py:52(<lambda>)(6590) 8.144
wikimacro.py:264(<lambda>) wikimacro.py:254(_macro_TitleIndex)(52000) 27.573
wikiutil.py:772(link_tag) AttachFile.py:94(getIndicator)(60) 9.127
Page.py:562(link_to)(6560) 16.372
__init__.py:131(logo)(10) 0.006
__init__.py:910(editbar)(20) 0.108
Page.py:1266(getACL) security.py:52(<lambda>)(6590) 8.144
posixpath.py:144(getmtime) Page.py:272(lastEditInfo)(20) 0.030
Page.py:1266(getACL)(6590) 7.533
TitleIndex:2(?)(10) 27.635
caching.py:40(mtime)(10) 0.001
caching.py:46(needsUpdate)(50) 0.002
logfile.py:160(date)(10) 0.000
security.py:52(<lambda>) Page.py:662(send_page)(10) 28.045
Page.py:1316(listPages)(6530) 10.590
__init__.py:488(shouldShowPageinfo)(10) 0.013
__init__.py:750(shouldShowEditbar)(20) 0.073
__init__.py:910(editbar)(10) 0.108
request.py:366(getAvailableActions)(10) 0.017
AttachFile.py:50(getAttachDir) AttachFile.py:94(getIndicator)(6460) 9.127
AttachFile.py:248(_get_files)(10) 0.005
AttachFile.py:94(getIndicator) Page.py:562(link_to)(6460) 16.372This test uses only 10 requests, since it takes too much time to create the results from the profile even on 2G G5. Note the number of get_rev per request: 5752 calls per request - for a wiki with only about 577 pages! In this case - 17.9s from 28.0s are used only by get_rev and functions called by it.
patch 460 + experinamtal refactoring
ab -n 11 -c 1 'http://localhost:8000/TitleIndex?allpages=1'
Requests per second: 1.53 [#/sec] (mean)
365528 function calls in 11.900 CPU seconds
Ordered by: internal time, call count
List reduced from 269 to 20 due to restriction <20>
ncalls tottime percall cumtime percall filename:lineno(function)
6610 1.311 0.000 1.311 0.000 wikiutil.py:154(quoteWikinameFS)
51950 1.245 0.000 1.245 0.000 wikimacro.py:264(<lambda>)
6560 1.233 0.000 3.429 0.001 Page.py:590(link_to)
6770 1.141 0.000 1.141 0.000 posixpath.py:144(getmtime)
26580 1.107 0.000 1.165 0.000 Page.py:24(Page)
6860 1.071 0.000 1.071 0.000 wikiutil.py:113(escape)
25480 1.038 0.000 1.038 0.000 posixpath.py:171(exists)
12380 0.770 0.000 3.027 0.000 Page.py:185(getInfoForDomain)
6730 0.357 0.000 0.421 0.000 wikiutil.py:187(unquoteWikiname)
8570 0.352 0.000 0.352 0.000 wikiutil.py:75(quoteWikinameURL)
6660 0.276 0.000 0.434 0.000 wikiacl.py:185(may)
32310 0.270 0.000 0.270 0.000 posixpath.py:56(join)
10 0.222 0.022 11.546 1.155 wikimacro.py:254(_macro_TitleIndex)
20 0.193 0.010 6.577 0.329 Page.py:1337(listPages)
6670 0.115 0.000 0.132 0.000 wikiutil.py:772(link_tag)
26830 0.113 0.000 3.140 0.000 Page.py:169(getInfo)
6660 0.095 0.000 1.243 0.000 Page.py:1287(getACL)
13540 0.084 0.000 3.199 0.000 Page.py:345(exists)
6460 0.074 0.000 0.622 0.000 AttachFile.py:94(getIndicator)
6740 0.063 0.000 0.063 0.000 wikiutil.py:56(decodeUserInput)
Ordered by: internal time, call count
List reduced from 269 to 20 due to restriction <20>
Function was called by...
wikiutil.py:154(quoteWikinameFS) Page.py:105(getStorageName)(6610) 1.354
wikimacro.py:264(<lambda>) wikimacro.py:254(_macro_TitleIndex)(51950) 11.546
Page.py:590(link_to) __init__.py:172(username)(10) 0.002
__init__.py:205(splitNavilink)(40) 0.014
__init__.py:282(navibar)(10) 0.016
text_html.py:212(pagelink)(40) 0.014
wikimacro.py:254(_macro_TitleIndex)(6460) 11.546
posixpath.py:144(getmtime) Page.py:289(lastEditInfo)(20) 0.005
Page.py:1287(getACL)(6660) 1.243
TitleIndex:2(?)(10) 11.591
caching.py:40(mtime)(20) 0.001
caching.py:46(needsUpdate)(50) 0.003
logfile.py:160(date)(10) 0.000
Page.py:24(Page) AttachFile.py:50(getAttachDir)(6470) 0.174
Page.py:1337(listPages)(6710) 6.577
__init__.py:287(getText)(60) 0.008
request.py:64(__init__)(10) 0.022
request.py:669(run)(10) 11.875
security.py:52(<lambda>)(6660) 1.755
text_html.py:212(pagelink)(40) 0.014
wikimacro.py:254(_macro_TitleIndex)(6460) 11.546
wikiutil.py:432(getSysPage)(150) 0.042
wikiutil.py:909(send_title)(10) 0.164
wikiutil.py:113(escape) Page.py:590(link_to)(6560) 3.429
__init__.py:150(title)(10) 0.001
__init__.py:538(searchform)(10) 0.001
base.py:90(image)(240) 0.005
text_html.py:251(url)(20) 0.001
wikiutil.py:909(send_title)(20) 0.164
posixpath.py:171(exists) AttachFile.py:94(getIndicator)(6460) 0.622
Page.py:185(getInfoForDomain)(18980) 3.027
Page.py:242(_last_edited)(20) 0.003
caching.py:37(exists)(20) 0.001
Page.py:185(getInfoForDomain) Page.py:169(getInfo)(12380) 3.140
wikiutil.py:187(unquoteWikiname) Page.py:1337(listPages)(6730) 6.577
wikiutil.py:75(quoteWikinameURL) AttachFile.py:94(getIndicator)(60) 0.622
Page.py:590(link_to)(6560) 3.429
Page.py:690(send_page)(10) 11.829
Page.py:1029(send_page_content)(10) 11.613
__init__.py:131(logo)(10) 0.001
__init__.py:912(editbar)(10) 0.088
wikimacro.py:43(<lambda>)(870) 0.023
wikimacro.py:254(_macro_TitleIndex)(900) 11.546
wikiutil.py:909(send_title)(140) 0.164
wikiacl.py:185(may) security.py:52(<lambda>)(6660) 1.755
posixpath.py:56(join) Page.py:185(getInfoForDomain)(25610) 3.027
Page.py:435(getPageList)(10) 6.579
Page.py:524(getEventLogPath)(10) 0.000
Page.py:530(getEditLogPath)(30) 0.001
Page.py:536(getAttachmentPath)(6480) 0.092
Page.py:542(getPagesPath)(10) 0.000
Page.py:548(getCachePath)(20) 0.000
caching.py:13(__init__)(10) 0.001
caching.py:34(_filename)(80) 0.001
multiconfig.py:267(__init__)(30) 0.004
wikiaction.py:836(getPlugins)(20) 0.003
wikimacro.py:254(_macro_TitleIndex) wikimacro.py:98(execute)(10) 11.563
Page.py:1337(listPages) Page.py:435(getPageList)(20) 6.579
wikiutil.py:772(link_tag) AttachFile.py:94(getIndicator)(60) 0.622
Page.py:590(link_to)(6560) 3.429
Page.py:1029(send_page_content)(10) 11.613
__init__.py:131(logo)(10) 0.001
__init__.py:912(editbar)(30) 0.088
Page.py:169(getInfo) Page.py:217(_text_filename)(40) 0.000
Page.py:316(isWritable)(10) 0.001
Page.py:329(isUnderlayPage)(10) 0.000
Page.py:337(isStandardPage)(10) 0.000
Page.py:345(exists)(13540) 3.199
Page.py:496(get_raw_body)(10) 0.006
Page.py:524(getEventLogPath)(10) 0.000
Page.py:530(getEditLogPath)(30) 0.001
Page.py:536(getAttachmentPath)(6480) 0.092
Page.py:542(getPagesPath)(10) 0.000
Page.py:548(getCachePath)(20) 0.000
Page.py:1287(getACL)(6660) 1.243
Page.py:1287(getACL) security.py:52(<lambda>)(6660) 1.755
Page.py:345(exists) Page.py:289(lastEditInfo)(20) 0.005
Page.py:590(link_to)(6560) 3.429
Page.py:690(send_page)(20) 11.829
Page.py:1337(listPages)(6710) 6.577
__init__.py:287(getText)(60) 0.008
__init__.py:490(shouldShowPageinfo)(10) 0.002
__init__.py:752(shouldShowEditbar)(20) 0.060
request.py:365(getAvailableActions)(10) 0.007
wikiutil.py:432(getSysPage)(130) 0.042
AttachFile.py:94(getIndicator) Page.py:590(link_to)(6460) 3.429
wikiutil.py:56(decodeUserInput) request.py:581(_setup_args_from_cgi_form)(10) 0.001
wikiutil.py:187(unquoteWikiname)(6730) 0.421
patch 468
ab -n 11 -c 1 'http://localhost:8000/TitleIndex?allpages=1'
Requests per second: 0.69 [#/sec] (mean)
720710 function calls (695890 primitive calls) in 27.812 CPU seconds
Ordered by: internal time, call count
List reduced from 238 to 20 due to restriction <20>
ncalls tottime percall cumtime percall filename:lineno(function)
56830 5.775 0.000 5.775 0.000 posixpath.py:171(exists)
43660 5.528 0.000 5.758 0.000 Page.py:89(get_current_from_pagedir)
6590 2.295 0.000 2.455 0.000 wikiacl.py:185(may)
19790 1.975 0.000 1.975 0.000 wikiutil.py:154(quoteWikinameFS)
50380/25560 1.660 0.000 11.273 0.000 Page.py:122(get_rev)
6540 1.375 0.000 1.449 0.000 wikiutil.py:187(unquoteWikiname)
19790 1.363 0.000 5.110 0.000 Page.py:62(reset)
8560 1.363 0.000 1.363 0.000 wikiutil.py:75(quoteWikinameURL)
26400 1.175 0.000 5.719 0.000 Page.py:272(getPagePath)
153630 1.105 0.000 1.105 0.000 posixpath.py:56(join)
43660 0.726 0.000 6.055 0.000 Page.py:105(get_rev_dir)
39690 0.351 0.000 11.782 0.000 Page.py:241(getPageStatus)
10 0.316 0.032 27.434 2.743 wikimacro.py:254(_macro_TitleIndex)
6560 0.314 0.000 11.615 0.002 Page.py:664(link_to)
39690 0.306 0.000 11.207 0.000 Page.py:176(getPageBasePath)
19790 0.235 0.000 5.346 0.000 Page.py:26(__init__)
52000 0.230 0.000 0.230 0.000 wikimacro.py:264(<lambda>)
20 0.209 0.010 11.435 0.572 Page.py:1418(listPages)
6650 0.162 0.000 0.185 0.000 wikiutil.py:772(link_tag)
6590 0.124 0.000 4.541 0.001 Page.py:1368(getACL)
Ordered by: internal time, call count
List reduced from 238 to 20 due to restriction <20>
Function was called by...
posixpath.py:171(exists) AttachFile.py:94(getIndicator)(6460) 6.511
Page.py:105(get_rev_dir)(43660) 6.055
Page.py:241(getPageStatus)(50) 11.782
Page.py:429(exists)(50) 3.236
Page.py:1368(getACL)(6590) 4.541
caching.py:37(exists)(20) 0.001
Page.py:89(get_current_from_pagedir) Page.py:105(get_rev_dir)(6730) 6.055
Page.py:122(get_rev)(36930) 11.273
wikiacl.py:185(may) security.py:52(<lambda>)(6590) 7.870
wikiutil.py:154(quoteWikinameFS) Page.py:62(reset)(19790) 5.110
Page.py:122(get_rev) Page.py:176(getPageBasePath)(36930) 11.207
Page.py:296(_text_filename)(6650) 4.200
Page.py:429(exists)(6800) 3.236
wikiutil.py:187(unquoteWikiname) Page.py:1418(listPages)(6540) 11.435
Page.py:62(reset) Page.py:26(__init__)(19790) 5.346
wikiutil.py:75(quoteWikinameURL) AttachFile.py:94(getIndicator)(60) 6.511
Page.py:664(link_to)(6560) 11.615
Page.py:764(send_page)(10) 27.795
__init__.py:131(logo)(10) 0.005
__init__.py:910(editbar)(10) 0.099
wikimacro.py:43(<lambda>)(870) 0.023
wikimacro.py:254(_macro_TitleIndex)(900) 27.434
wikiutil.py:909(send_title)(140) 0.208
Page.py:272(getPagePath) AttachFile.py:50(getAttachDir)(6470) 5.061
Page.py:62(reset)(19790) 5.110
Page.py:325(_last_edited)(20) 0.004
Page.py:429(exists)(50) 3.236
Page.py:530(getPageList)(20) 11.438
Page.py:1096(send_page_content)(10) 27.515
caching.py:13(__init__)(20) 0.002
editlog.py:74(__init__)(10) 0.001
eventlog.py:13(__init__)(10) 0.001
posixpath.py:56(join) Page.py:62(reset)(19790) 5.110
Page.py:89(get_current_from_pagedir)(43660) 5.758
Page.py:105(get_rev_dir)(43660) 6.055
Page.py:241(getPageStatus)(39690) 11.782
Page.py:1418(listPages)(6730) 11.435
caching.py:13(__init__)(10) 0.002
caching.py:34(_filename)(70) 0.001
wikiaction.py:836(getPlugins)(20) 0.003
Page.py:105(get_rev_dir) Page.py:122(get_rev)(36930) 11.273
Page.py:1418(listPages)(6730) 11.435
Page.py:241(getPageStatus) Page.py:122(get_rev)(13290) 11.273
Page.py:272(getPagePath)(26400) 5.719
wikimacro.py:254(_macro_TitleIndex) wikimacro.py:98(execute)(10) 27.453
Page.py:664(link_to) __init__.py:172(username)(10) 0.009
__init__.py:205(splitNavilink)(40) 0.029
__init__.py:282(navibar)(10) 0.031
text_html.py:212(pagelink)(40) 0.026
wikimacro.py:254(_macro_TitleIndex)(6460) 27.434
Page.py:176(getPageBasePath) Page.py:241(getPageStatus)(39690) 11.782
Page.py:26(__init__) AttachFile.py:50(getAttachDir)(6470) 5.061
__init__.py:287(getText)(60) 0.033
request.py:670(run)(10) 27.812
security.py:52(<lambda>)(6590) 7.870
text_html.py:212(pagelink)(40) 0.026
wikimacro.py:254(_macro_TitleIndex)(6460) 27.434
wikiutil.py:432(getSysPage)(150) 0.074
wikiutil.py:909(send_title)(10) 0.208
wikimacro.py:264(<lambda>) wikimacro.py:254(_macro_TitleIndex)(52000) 27.434
Page.py:1418(listPages) Page.py:530(getPageList)(20) 11.438
wikiutil.py:772(link_tag) AttachFile.py:94(getIndicator)(60) 6.511
Page.py:664(link_to)(6560) 11.615
__init__.py:131(logo)(10) 0.005
__init__.py:910(editbar)(20) 0.099
Page.py:1368(getACL) security.py:52(<lambda>)(6590) 7.870