Benchmark

[vincentsarago]

In addition to being written in Python and modern framework (FastAPI), we wanted TiMVT to be Fast. Here is a quick (and dirty) comparison with state of the art project

timvt

# 20 users / 1000 requests
$ siege --file benchmark/urls.txt -b -c 20 -r 1000
Transactions:                  20000 hits
Availability:                 100.00 %
Elapsed time:                  45.48 secs
Data transferred:              36.99 MB
Response time:                  0.04 secs
Transaction rate:             439.75 trans/sec
Throughput:                     0.81 MB/sec
Concurrency:                   18.81
Successful transactions:       20000
Failed transactions:               0
Longest transaction:            0.37
Shortest transaction:           0.00

# 200 users / 100 requests
$ siege --file benchmark/urls.txt -b -c 200 -r 100
Transactions:                  19977 hits
Availability:                  99.89 %
Elapsed time:                  55.99 secs
Data transferred:              33.82 MB
Response time:                  0.51 secs
Transaction rate:             356.80 trans/sec
Throughput:                     0.60 MB/sec
Concurrency:                  183.65
Successful transactions:       19977
Failed transactions:              23
Longest transaction:            4.25
Shortest transaction:           0.00

pg_tileserv

# 20 users / 1000 requests
$ siege --file benchmark/urls.txt -b -c 20 -r 1000
Transactions:                  20000 hits
Availability:                 100.00 %
Elapsed time:                  53.65 secs
Data transferred:              39.66 MB
Response time:                  0.05 secs
Transaction rate:             372.79 trans/sec
Throughput:                     0.74 MB/sec
Concurrency:                   19.27
Successful transactions:       20000
Failed transactions:               0
Longest transaction:            0.64
Shortest transaction:           0.00
 
# 200 user / 100 requests
$ siege --file benchmark/urls.txt -b -c 200 -r 100
Transactions:                  19948 hits
Availability:                  99.74 %
Elapsed time:                  48.55 secs
Data transferred:              36.18 MB
Response time:                  0.46 secs
Transaction rate:             410.88 trans/sec
Throughput:                     0.75 MB/sec
Concurrency:                  188.91
Successful transactions:       19948
Failed transactions:              52
Longest transaction:            3.90
Shortest transaction:           0.00

ref: #60