Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Update result docs #44

Open
jamadden opened this issue May 31, 2019 · 1 comment
Open

Update result docs #44

jamadden opened this issue May 31, 2019 · 1 comment

Comments

@jamadden
Copy link
Member

The page https://zodbshootout.readthedocs.io/en/latest/results.html is a bit outdated with respect to 0.7.0 and its new measurements.

@jamadden
Copy link
Member Author

jamadden commented Jun 1, 2019

Python 3.7, concurrency = 5 processes (zeo is not using a persistent cache, despite the name, because that doesn't work with mulitple processes):

MappingStorage: add 1000 objects: Mean +- std dev: 54.2 ms +- 3.2 ms
MappingStorage: update 1000 objects: Mean +- std dev: 51.8 ms +- 1.9 ms
MappingStorage: read 1000 cold objects: Mean +- std dev: 36.4 ms +- 1.7 ms
MappingStorage: read 1000 warm objects: Mean +- std dev: 87.9 ms +- 8.0 ms
MappingStorage: read 1000 hot objects: Mean +- std dev: 43.0 ms +- 2.3 ms
MappingStorage: read 1000 steamin objects: Mean +- std dev: 1.12 ms +- 0.19 ms
MappingStorage: empty commit: Mean +- std dev: 42.3 us +- 2.4 us
zeofs_pcache: add 1000 objects: Mean +- std dev: 528 ms +- 16 ms
zeofs_pcache: update 1000 objects: Mean +- std dev: 636 ms +- 7 ms
zeofs_pcache: read 1000 cold objects: Mean +- std dev: 764 ms +- 9 ms
zeofs_pcache: read 1000 warm objects: Mean +- std dev: 1.39 sec +- 0.04 sec
zeofs_pcache: read 1000 hot objects: Mean +- std dev: 50.7 ms +- 2.4 ms
zeofs_pcache: read 1000 steamin objects: Mean +- std dev: 1.15 ms +- 0.12 ms
zeofs_pcache: empty commit: Mean +- std dev: 38.4 us +- 1.5 us
psycopg2-pcache: add 1000 objects: Mean +- std dev: 155 ms +- 3 ms
psycopg2-pcache: update 1000 objects: Mean +- std dev: 145 ms +- 2 ms
psycopg2-pcache: read 1000 cold objects: Mean +- std dev: 73.8 ms +- 3.4 ms
psycopg2-pcache: read 1000 warm objects: Mean +- std dev: 256 ms +- 13 ms
psycopg2-pcache: read 1000 hot objects: Mean +- std dev: 65.9 ms +- 4.0 ms
psycopg2-pcache: read 1000 steamin objects: Mean +- std dev: 1.05 ms +- 0.06 ms
psycopg2-pcache: empty commit: Mean +- std dev: 364 us +- 10 us
mysqlclient-pcache: add 1000 objects: Mean +- std dev: 157 ms +- 2 ms
mysqlclient-pcache: update 1000 objects: Mean +- std dev: 215 ms +- 4 ms
mysqlclient-pcache: read 1000 cold objects: Mean +- std dev: 76.3 ms +- 3.0 ms
mysqlclient-pcache: read 1000 warm objects: Mean +- std dev: 338 ms +- 18 ms
mysqlclient-pcache: read 1000 hot objects: Mean +- std dev: 62.4 ms +- 4.0 ms
mysqlclient-pcache: read 1000 steamin objects: Mean +- std dev: 1.02 ms +- 0.06 ms
mysqlclient-pcache: empty commit: Mean +- std dev: 528 us +- 10 us
mysqlclient-hp: add 1000 objects: Mean +- std dev: 176 ms +- 4 ms
mysqlclient-hp: update 1000 objects: Mean +- std dev: 236 ms +- 3 ms
mysqlclient-hp: read 1000 cold objects: Mean +- std dev: 481 ms +- 173 ms
mysqlclient-hp: read 1000 warm objects: Mean +- std dev: 703 ms +- 28 ms
mysqlclient-hp: read 1000 hot objects: Mean +- std dev: 59.1 ms +- 2.5 ms
mysqlclient-hp: read 1000 steamin objects: Mean +- std dev: 1.03 ms +- 0.04 ms
mysqlclient-hp: empty commit: Mean +- std dev: 573 us +- 31 us
Benchmark zeofs_pcache_1000 mysqlclient-hp_1000 mysqlclient-pcache_1000 psycopg2-pcache_1000
add 528 ms 176 ms: 3.00x faster (-67%) 157 ms: 3.37x faster (-70%) 155 ms: 3.39x faster (-71%)
update 636 ms 236 ms: 2.69x faster (-63%) 215 ms: 2.95x faster (-66%) 145 ms: 4.38x faster (-77%)
cold 764 ms 481 ms: 1.59x faster (-37%) 76.3 ms: 10.02x faster (-90%) 73.8 ms: 10.36x faster (-90%)
warm 1.39 sec 703 ms: 1.97x faster (-49%) 338 ms: 4.10x faster (-76%) 256 ms: 5.42x faster (-82%)
hot 50.7 ms 59.1 ms: 1.17x slower (+17%) 62.4 ms: 1.23x slower (+23%) 65.9 ms: 1.30x slower (+30%)
steamin 1.15 ms 1.03 ms: 1.12x faster (-11%) 1.02 ms: 1.12x faster (-11%) not significant
commit 38.4 us 573 us: 14.89x slower (+1389%) 528 us: 13.75x slower (+1275%) 364 us: 9.48x slower (+848%)

EDIT: I should mention that those incorporate the persistent caching changes from zodb/relstorage#243

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant