-
Notifications
You must be signed in to change notification settings - Fork 76
Benchmark
PAN, Myautsai edited this page Feb 3, 2016
·
1 revision
To run benchmark for the Python version of libmc. You can get runbench.py first, then run:
virtualenv venv
source ./venv/bin/activate
pip install pylibmc python-memcached libmc
python ./misc/runbench.pyNOTE: pylibmc and libmc are 2 different python memcached client library.
INFO:libmc.bench:pylibmc: /usr/local/lib/python2.7/site-packages/pylibmc/__init__.pyc
INFO:libmc.bench:libmc: /Users/kelvin/GitHub/douban/libmc/venv/lib/python2.7/site-packages/libmc-0.5.6-py2.7-macosx-10.11-x86_64.egg/libmc/__init__.pyc
INFO:libmc.bench:4 participants in 16 benchmarks
INFO:libmc.bench:
INFO:libmc.bench:Multi set 10 keys with value size 100
INFO:libmc.bench: pylibmc (md5 / ketama): 0.0003s, σ=0.000432, n=3176, snr=1:1.44
INFO:libmc.bench: pylibmc (md5 / ketama / nodelay / nonblocking): 0.000285s, σ=0.000151, n=3331, snr=1.89:1
INFO:libmc.bench: python-memcached: 0.00048s, σ=0.000174, n=2036, snr=2.76:1
INFO:libmc.bench: libmc(md5 / ketama / nodelay / nonblocking, from douban): 0.000171s, σ=4.71e-05, n=5467, snr=3.63:1
INFO:libmc.bench:
INFO:libmc.bench:Multi get 10 keys with value size 100
INFO:libmc.bench: pylibmc (md5 / ketama): 0.000193s, σ=2.62e-05, n=4920, snr=7.38:1
INFO:libmc.bench: pylibmc (md5 / ketama / nodelay / nonblocking): 0.000147s, σ=4.31e-05, n=6291, snr=3.42:1
INFO:libmc.bench: python-memcached: 0.000464s, σ=0.000124, n=2103, snr=3.75:1
INFO:libmc.bench: libmc(md5 / ketama / nodelay / nonblocking, from douban): 0.000143s, σ=2.6e-05, n=6534, snr=5.5:1
INFO:libmc.bench:
INFO:libmc.bench:Multi set 100 keys with value size 100
INFO:libmc.bench: pylibmc (md5 / ketama): 0.00259s, σ=0.000675, n=384, snr=3.84:1
INFO:libmc.bench: pylibmc (md5 / ketama / nodelay / nonblocking): 0.00268s, σ=0.000872, n=371, snr=3.07:1
INFO:libmc.bench: python-memcached: 0.00226s, σ=0.000324, n=441, snr=6.96:1
INFO:libmc.bench: libmc(md5 / ketama / nodelay / nonblocking, from douban): 0.000649s, σ=0.000103, n=1506, snr=6.32:1
INFO:libmc.bench:
INFO:libmc.bench:Multi get 100 keys with value size 100
INFO:libmc.bench: pylibmc (md5 / ketama): 0.000705s, σ=0.000224, n=1384, snr=3.15:1
INFO:libmc.bench: pylibmc (md5 / ketama / nodelay / nonblocking): 0.000651s, σ=0.000175, n=1502, snr=3.71:1
INFO:libmc.bench: python-memcached: 0.00205s, σ=0.000528, n=483, snr=3.9:1
INFO:libmc.bench: libmc(md5 / ketama / nodelay / nonblocking, from douban): 0.000505s, σ=9.84e-05, n=1928, snr=5.13:1
INFO:libmc.bench:
INFO:libmc.bench:Multi set 10 keys with value size 1000
INFO:libmc.bench: pylibmc (md5 / ketama): 0.000296s, σ=8.94e-05, n=3207, snr=3.31:1
INFO:libmc.bench: pylibmc (md5 / ketama / nodelay / nonblocking): 0.000278s, σ=3.22e-05, n=3439, snr=8.65:1
INFO:libmc.bench: python-memcached: 0.000503s, σ=0.000138, n=1943, snr=3.66:1
INFO:libmc.bench: libmc(md5 / ketama / nodelay / nonblocking, from douban): 0.000171s, σ=2.47e-05, n=5492, snr=6.92:1
INFO:libmc.bench:
INFO:libmc.bench:Multi get 10 keys with value size 1000
INFO:libmc.bench: pylibmc (md5 / ketama): 0.000188s, σ=2.76e-05, n=5026, snr=6.83:1
INFO:libmc.bench: pylibmc (md5 / ketama / nodelay / nonblocking): 0.000147s, σ=4.67e-05, n=6306, snr=3.16:1
INFO:libmc.bench: python-memcached: 0.000517s, σ=0.000228, n=1882, snr=2.27:1
INFO:libmc.bench: libmc(md5 / ketama / nodelay / nonblocking, from douban): 0.00015s, σ=2.11e-05, n=6259, snr=7.1:1
INFO:libmc.bench:
INFO:libmc.bench:Small set
INFO:libmc.bench: pylibmc (md5 / ketama): 4.04e-05s, σ=2.14e-05, n=19369, snr=1.89:1
INFO:libmc.bench: pylibmc (md5 / ketama / nodelay / nonblocking): 3.91e-05s, σ=8.64e-06, n=20459, snr=4.53:1
INFO:libmc.bench: python-memcached: 7.31e-05s, σ=1.58e-05, n=12131, snr=4.62:1
INFO:libmc.bench: libmc(md5 / ketama / nodelay / nonblocking, from douban): 4.69e-05s, σ=1.67e-05, n=17610, snr=2.81:1
INFO:libmc.bench:
INFO:libmc.bench:Small get
INFO:libmc.bench: pylibmc (md5 / ketama): 3.72e-05s, σ=1.17e-05, n=21309, snr=3.19:1
INFO:libmc.bench: pylibmc (md5 / ketama / nodelay / nonblocking): 3.68e-05s, σ=5.72e-06, n=21960, snr=6.43:1
INFO:libmc.bench: python-memcached: 7.03e-05s, σ=1.27e-05, n=12653, snr=5.53:1
INFO:libmc.bench: libmc(md5 / ketama / nodelay / nonblocking, from douban): 3.72e-05s, σ=6.93e-06, n=21774, snr=5.37:1
INFO:libmc.bench:
INFO:libmc.bench:4k uncompressed set
INFO:libmc.bench: pylibmc (md5 / ketama): 3.94e-05s, σ=5.67e-06, n=20650, snr=6.95:1
INFO:libmc.bench: pylibmc (md5 / ketama / nodelay / nonblocking): 3.99e-05s, σ=6.34e-06, n=20331, snr=6.3:1
INFO:libmc.bench: python-memcached: 7.55e-05s, σ=2.16e-05, n=11793, snr=3.49:1
INFO:libmc.bench: libmc(md5 / ketama / nodelay / nonblocking, from douban): 4.71e-05s, σ=7.3e-06, n=17563, snr=6.46:1
INFO:libmc.bench:
INFO:libmc.bench:4k uncompressed get
INFO:libmc.bench: pylibmc (md5 / ketama): 4.51e-05s, σ=7.59e-06, n=18201, snr=5.95:1
INFO:libmc.bench: pylibmc (md5 / ketama / nodelay / nonblocking): 4.58e-05s, σ=9.09e-06, n=18015, snr=5.04:1
INFO:libmc.bench: python-memcached: 8.28e-05s, σ=2.36e-05, n=10806, snr=3.51:1
INFO:libmc.bench: libmc(md5 / ketama / nodelay / nonblocking, from douban): 4.72e-05s, σ=1.43e-05, n=17446, snr=3.31:1
INFO:libmc.bench:
INFO:libmc.bench:4k compressed set
INFO:libmc.bench: pylibmc (md5 / ketama): 8.13e-05s, σ=1.53e-05, n=10892, snr=5.33:1
INFO:libmc.bench: pylibmc (md5 / ketama / nodelay / nonblocking): 7.9e-05s, σ=1.22e-05, n=11208, snr=6.46:1
INFO:libmc.bench: python-memcached: 7.73e-05s, σ=1.65e-05, n=11507, snr=4.7:1
INFO:libmc.bench: libmc(md5 / ketama / nodelay / nonblocking, from douban): 9.47e-05s, σ=5.05e-05, n=9353, snr=1.87:1
INFO:libmc.bench:
INFO:libmc.bench:4k compressed get
INFO:libmc.bench: pylibmc (md5 / ketama): 4.89e-05s, σ=1.3e-05, n=17010, snr=3.75:1
INFO:libmc.bench: pylibmc (md5 / ketama / nodelay / nonblocking): 5.04e-05s, σ=1.32e-05, n=16463, snr=3.82:1
INFO:libmc.bench: python-memcached: 8.07e-05s, σ=2.81e-05, n=11050, snr=2.87:1
INFO:libmc.bench: libmc(md5 / ketama / nodelay / nonblocking, from douban): 5.11e-05s, σ=1.3e-05, n=16421, snr=3.93:1
INFO:libmc.bench:
INFO:libmc.bench:1M compressed set
INFO:libmc.bench: pylibmc (md5 / ketama): 0.00607s, σ=0.00112, n=165, snr=5.41:1
INFO:libmc.bench: pylibmc (md5 / ketama / nodelay / nonblocking): 0.00555s, σ=0.00042, n=180, snr=13.2:1
INFO:libmc.bench: python-memcached: 0.00571s, σ=0.000586, n=175, snr=9.75:1
INFO:libmc.bench: libmc(md5 / ketama / nodelay / nonblocking, from douban): 0.00612s, σ=0.00117, n=163, snr=5.24:1
INFO:libmc.bench:
INFO:libmc.bench:1M compressed get
INFO:libmc.bench: pylibmc (md5 / ketama): 0.00317s, σ=0.000564, n=313, snr=5.62:1
INFO:libmc.bench: pylibmc (md5 / ketama / nodelay / nonblocking): 0.00324s, σ=0.000866, n=306, snr=3.75:1
INFO:libmc.bench: python-memcached: 0.00276s, σ=0.000427, n=360, snr=6.47:1
INFO:libmc.bench: libmc(md5 / ketama / nodelay / nonblocking, from douban): 0.00291s, σ=0.000563, n=342, snr=5.16:1
INFO:libmc.bench:
INFO:libmc.bench:Complex data set
INFO:libmc.bench: pylibmc (md5 / ketama): 5.77e-05s, σ=1.02e-05, n=14732, snr=5.66:1
INFO:libmc.bench: pylibmc (md5 / ketama / nodelay / nonblocking): 9.03e-05s, σ=8.68e-05, n=9309, snr=1.04:1
INFO:libmc.bench: python-memcached: 0.000124s, σ=2.62e-05, n=7475, snr=4.73:1
INFO:libmc.bench: libmc(md5 / ketama / nodelay / nonblocking, from douban): 6.55e-05s, σ=1.11e-05, n=13157, snr=5.9:1
INFO:libmc.bench:
INFO:libmc.bench:Complex data get
INFO:libmc.bench: pylibmc (md5 / ketama): 7.03e-05s, σ=1.36e-05, n=12560, snr=5.16:1
INFO:libmc.bench: pylibmc (md5 / ketama / nodelay / nonblocking): 6.93e-05s, σ=1.39e-05, n=12734, snr=4.99:1
INFO:libmc.bench: python-memcached: 0.000136s, σ=3.24e-05, n=6953, snr=4.19:1
INFO:libmc.bench: libmc(md5 / ketama / nodelay / nonblocking, from douban): 6.77e-05s, σ=1.3e-05, n=13018, snr=5.21:1
labels = ['pylibmc (md5 / ketama)', 'pylibmc (md5 / ketama / nodelay / nonblocking)', 'python-memcached', 'libmc(md5 / ketama / nodelay / nonblocking, from douban)']
benchmarks = ['Multi set 10 keys with value size 100', 'Multi get 10 keys with value size 100', 'Multi set 100 keys with value size 100', 'Multi get 100 keys with value size 100', 'Multi set 10 keys with value size 1000', 'Multi get 10 keys with value size 1000', 'Small set', 'Small get', '4k uncompressed set', '4k uncompressed get', '4k compressed set', '4k compressed get', '1M compressed set', '1M compressed get', 'Complex data set', 'Complex data get']
means = [[0.0002996234256926949, 0.00019298109756098398, 0.0025896276041666813, 0.0007051979768785935, 0.00029609042719056736, 0.00018843971348984155, 4.0448861582946165e-05, 3.717992397578633e-05, 3.9406924939449686e-05, 4.510735673862905e-05, 8.127497245683664e-05, 4.8873956496182264e-05, 0.006066896969697192, 0.003167715654952214, 5.774701330439756e-05, 7.027969745222611e-05], [0.00028543020114079617, 0.00014713272929582508, 0.0026800188679246143, 0.0006508761651132068, 0.0002783972084908282, 0.00014748858230258767, 3.913299770271751e-05, 3.677367941710178e-05, 3.994781368356054e-05, 4.5792728281987246e-05, 7.90381870093115e-05, 5.0441353337769186e-05, 0.005547111111111115, 0.003244999999999906, 9.031410462987435e-05, 6.925640018850603e-05], [0.0004802195481335867, 0.00046383119353304565, 0.002255804988662166, 0.0020549544513457437, 0.0005030905815747712, 0.0005174208289053804, 7.311309867283995e-05, 7.02872046155136e-05, 7.549325871275615e-05, 8.283240792151622e-05, 7.734813591724249e-05, 8.065357466057745e-05, 0.005708474285713732, 0.0027625166666664725, 0.00012378167224080398, 0.0001355390478929987], [0.00017092354124748917, 0.00014322099785736344, 0.0006489349269588166, 0.0005049071576763192, 0.000170826656955569, 0.0001498185013580631, 4.686325951163478e-05, 3.721847157159245e-05, 4.7131298753032173e-05, 4.724137338069576e-05, 9.47352721052604e-05, 5.107185920468898e-05, 0.00611650920245417, 0.002905274853801163, 6.553439233868087e-05, 6.766392687050268e-05]]
stddevs = [[0.0004319545247666136, 2.6154432081276215e-05, 0.0006745602613682502, 0.00022374612366963343, 8.939654213090618e-05, 2.757525691105015e-05, 2.135932694659943e-05, 1.1652890718597232e-05, 5.671967557002017e-06, 7.58600025470656e-06, 1.5258223725256335e-05, 1.30424526451728e-05, 0.001120770858230152, 0.0005639924083887069, 1.0205584678663857e-05, 1.3623165519125422e-05], [0.0001506796416395157, 4.3075927780267695e-05, 0.0008717093516270504, 0.00017522978820084641, 3.2199542390402565e-05, 4.671697767918376e-05, 8.640868186900234e-06, 5.718340363834963e-06, 6.337993789880252e-06, 9.087241905008255e-06, 1.2236989099010757e-05, 1.3207342266009721e-05, 0.00042033353011973983, 0.0008659680593062705, 8.676809856242796e-05, 1.3887462646830188e-05], [0.0001739168681991394, 0.00012374973416727616, 0.000324019068155505, 0.0005275753927418769, 0.00013752825762628789, 0.000228164678376854, 1.5835974080099533e-05, 1.271790329761448e-05, 2.164979882199317e-05, 2.360653134829074e-05, 1.6450645557139243e-05, 2.8056832890422904e-05, 0.0005857139703417723, 0.000427188989857721, 2.6196411940258205e-05, 3.237094357141854e-05], [4.7096637824320416e-05, 2.602215532899476e-05, 0.00010260411297425745, 9.842592150169722e-05, 2.468947342197984e-05, 2.111311230850282e-05, 1.665063521238413e-05, 6.9322845352261316e-06, 7.295611168790217e-06, 1.4280859309071957e-05, 5.0534841290877844e-05, 1.2981455638549615e-05, 0.0011668003197254546, 0.0005625985038217712, 1.1099471777088412e-05, 1.2985321730018827e-05]]