-
Notifications
You must be signed in to change notification settings - Fork 141
2022 04 23 bench results
Andrea edited this page Apr 23, 2022
·
3 revisions
Benchmarked on: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz, 32G RAM
Config:
14a8435 Fri, 22 Apr 2022 10:53:45 +0200 GitHub Update Java version (#232)
- GRPC_BENCHMARK_DURATION=120s
- GRPC_BENCHMARK_WARMUP=30s
- GRPC_SERVER_RAM=512m
- GRPC_CLIENT_CONNECTIONS=50
- GRPC_CLIENT_CONCURRENCY=1000
- GRPC_CLIENT_QPS=0
- GRPC_CLIENT_CPUS=9
- GRPC_REQUEST_SCENARIO=complex_proto
1 CPU Server
-----------------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
-----------------------------------------------------------------------------------------------------------------------------------------
| go_vtgrpc | 44074 | 21.50 ms | 32.63 ms | 36.62 ms | 43.99 ms | 99.96% | 25.28 MiB |
| rust_tonic_st | 41546 | 23.77 ms | 26.06 ms | 28.66 ms | 33.24 ms | 101.05% | 12.03 MiB |
| rust_thruster_st | 41334 | 23.88 ms | 29.69 ms | 30.94 ms | 33.35 ms | 101.35% | 10.99 MiB |
| rust_thruster_mt | 36793 | 26.93 ms | 12.27 ms | 46.86 ms | 646.81 ms | 103.57% | 12.17 MiB |
| rust_tonic_mt | 35443 | 27.98 ms | 12.08 ms | 13.37 ms | 668.38 ms | 101.02% | 14.05 MiB |
| go_grpc | 34289 | 27.97 ms | 41.94 ms | 47.70 ms | 60.31 ms | 98.23% | 26.06 MiB |
| dotnet_grpc | 33899 | 26.75 ms | 49.04 ms | 68.64 ms | 84.31 ms | 100.74% | 114.47 MiB |
| scala_akka | 32435 | 28.63 ms | 61.27 ms | 69.93 ms | 94.39 ms | 102.03% | 223.0 MiB |
| java_quarkus | 32408 | 29.78 ms | 52.93 ms | 56.27 ms | 75.82 ms | 100.05% | 110.34 MiB |
| rust_grpcio | 31035 | 32.04 ms | 35.33 ms | 36.46 ms | 38.97 ms | 102.08% | 17.37 MiB |
| cpp_grpc_st | 26700 | 37.33 ms | 41.89 ms | 43.15 ms | 46.01 ms | 103.33% | 9.32 MiB |
| cpp_grpc_mt | 26376 | 37.79 ms | 42.94 ms | 43.97 ms | 46.03 ms | 103.18% | 9.99 MiB |
| cpp_asio_grpc | 24926 | 40.00 ms | 45.27 ms | 46.25 ms | 49.44 ms | 103.18% | 9.75 MiB |
| java_openj9_grpc_gencon | 24139 | 40.07 ms | 72.49 ms | 103.75 ms | 289.95 ms | 95.85% | 118.91 MiB |
| java_hotspot_grpc_g1gc | 23657 | 41.37 ms | 75.27 ms | 83.25 ms | 110.66 ms | 100.82% | 84.28 MiB |
| java_hotspot_grpc_she | 23304 | 42.04 ms | 73.83 ms | 81.40 ms | 101.02 ms | 101.65% | 81.83 MiB |
| java_hotspot_grpc_sgc | 23290 | 42.04 ms | 76.21 ms | 83.70 ms | 99.94 ms | 101.25% | 83.88 MiB |
| java_hotspot_grpc_zgc | 22980 | 42.68 ms | 75.46 ms | 82.79 ms | 103.52 ms | 101.1% | 85.13 MiB |
| java_hotspot_grpc_pgc | 22305 | 44.03 ms | 79.36 ms | 88.79 ms | 121.52 ms | 101.13% | 95.37 MiB |
| crystal_grpc | 22047 | 45.23 ms | 60.54 ms | 63.95 ms | 69.05 ms | 102.9% | 51.74 MiB |
| java_quarkus_native | 19877 | 50.02 ms | 54.95 ms | 56.63 ms | 62.44 ms | 102.13% | 89.25 MiB |
| scala_fs2 | 18097 | 54.07 ms | 97.74 ms | 107.87 ms | 192.24 ms | 101.25% | 233.53 MiB |
| kotlin_grpc | 17570 | 55.98 ms | 93.90 ms | 104.54 ms | 180.47 ms | 100.97% | 216.03 MiB |
| java_armeria | 16664 | 59.55 ms | 92.90 ms | 103.08 ms | 179.40 ms | 100.82% | 247.55 MiB |
| swift_grpc | 11984 | 83.22 ms | 84.78 ms | 85.34 ms | 87.14 ms | 101.54% | 6.74 MiB |
| erlang_grpcbox | 4835 | 206.35 ms | 219.81 ms | 225.78 ms | 236.80 ms | 101.91% | 95.46 MiB |
| elixir_grpc | 4423 | 225.57 ms | 251.99 ms | 256.84 ms | 271.12 ms | 102.45% | 135.95 MiB |
| node_grpcjs_st | 4133 | 240.95 ms | 269.39 ms | 278.48 ms | 292.90 ms | 100.59% | 45.63 MiB |
| java_micronaut_workstealing | 4073 | 243.69 ms | 178.65 ms | 217.76 ms | 4.56 s | 53.11% | 303.58 MiB |
| scala_zio | 3285 | 303.48 ms | 379.98 ms | 501.44 ms | 785.32 ms | 102.48% | 237.78 MiB |
| java_micronaut | 3242 | 305.91 ms | 178.06 ms | 1.11 s | 5.16 s | 37.91% | 297.07 MiB |
| python_async_grpc | 2392 | 416.42 ms | 452.27 ms | 457.34 ms | 464.43 ms | 102.18% | 32.96 MiB |
| python_grpc | 1961 | 507.66 ms | 547.01 ms | 556.62 ms | 570.33 ms | 102.4% | 34.28 MiB |
| ruby_grpc | 1847 | 538.95 ms | 601.64 ms | 630.22 ms | 821.94 ms | 102.83% | 37.11 MiB |
| java_aot | 1675 | 592.97 ms | 996.33 ms | 1.11 s | 1.47 s | 100.12% | 172.5 MiB |
| d_grpc | 1379 | 712.65 ms | 499.63 ms | 591.70 ms | 11.93 s | 104.8% | 27.16 MiB |
| php_grpc | 869 | 1.14 s | 1.20 s | 1.20 s | 1.20 s | 102.34% | 66.97 MiB |
-----------------------------------------------------------------------------------------------------------------------------------------
2 CPU Server
-----------------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
-----------------------------------------------------------------------------------------------------------------------------------------
| cpp_grpc_mt | 69882 | 11.94 ms | 20.07 ms | 23.91 ms | 31.93 ms | 206.89% | 30.47 MiB |
| dotnet_grpc | 68266 | 11.76 ms | 18.63 ms | 22.90 ms | 32.94 ms | 192.11% | 176.25 MiB |
| cpp_asio_grpc | 66828 | 12.64 ms | 21.78 ms | 25.69 ms | 34.19 ms | 209.54% | 33.97 MiB |
| scala_akka | 66560 | 11.60 ms | 21.62 ms | 28.45 ms | 53.59 ms | 195.37% | 225.91 MiB |
| go_vtgrpc | 63353 | 12.99 ms | 21.59 ms | 25.44 ms | 34.55 ms | 187.78% | 25.76 MiB |
| rust_thruster_mt | 56740 | 16.02 ms | 45.50 ms | 69.57 ms | 90.42 ms | 199.16% | 14.47 MiB |
| rust_grpcio | 54498 | 16.89 ms | 26.14 ms | 29.20 ms | 35.30 ms | 214.73% | 35.8 MiB |
| rust_tonic_mt | 53832 | 17.29 ms | 46.74 ms | 67.39 ms | 102.35 ms | 201.71% | 17.24 MiB |
| go_grpc | 50939 | 17.41 ms | 26.68 ms | 29.79 ms | 37.80 ms | 196.25% | 25.6 MiB |
| java_openj9_grpc_gencon | 48086 | 17.37 ms | 48.42 ms | 56.48 ms | 67.50 ms | 204.61% | 127.52 MiB |
| java_hotspot_grpc_g1gc | 45493 | 19.66 ms | 37.76 ms | 42.97 ms | 60.05 ms | 191.97% | 83.02 MiB |
| java_hotspot_grpc_zgc | 44611 | 20.19 ms | 38.05 ms | 42.91 ms | 60.57 ms | 191.92% | 86.41 MiB |
| java_hotspot_grpc_she | 44422 | 20.21 ms | 39.38 ms | 45.61 ms | 71.83 ms | 192.21% | 91.81 MiB |
| java_hotspot_grpc_sgc | 43816 | 20.63 ms | 39.21 ms | 44.54 ms | 62.25 ms | 193.53% | 86.82 MiB |
| java_hotspot_grpc_pgc | 43605 | 20.69 ms | 40.97 ms | 47.11 ms | 75.71 ms | 191.5% | 85.6 MiB |
| java_quarkus | 43083 | 21.07 ms | 42.63 ms | 47.08 ms | 55.22 ms | 164.43% | 115.89 MiB |
| rust_tonic_st | 42072 | 23.46 ms | 29.26 ms | 30.42 ms | 33.04 ms | 102.14% | 12.78 MiB |
| rust_thruster_st | 41267 | 23.91 ms | 29.88 ms | 31.21 ms | 33.95 ms | 101.29% | 11.21 MiB |
| kotlin_grpc | 40416 | 21.45 ms | 56.37 ms | 69.01 ms | 85.08 ms | 205.81% | 212.22 MiB |
| scala_fs2 | 35342 | 25.26 ms | 57.35 ms | 66.60 ms | 83.52 ms | 205.03% | 232.06 MiB |
| java_quarkus_native | 33090 | 29.62 ms | 34.79 ms | 36.98 ms | 41.87 ms | 197.02% | 89.44 MiB |
| crystal_grpc | 26927 | 37.02 ms | 39.31 ms | 40.18 ms | 42.06 ms | 118.64% | 46.61 MiB |
| java_armeria | 26869 | 34.79 ms | 72.51 ms | 77.69 ms | 87.93 ms | 202.47% | 215.91 MiB |
| cpp_grpc_st | 26229 | 38.01 ms | 40.41 ms | 42.67 ms | 44.99 ms | 103.13% | 9.42 MiB |
| swift_grpc | 22090 | 45.05 ms | 46.97 ms | 47.77 ms | 49.73 ms | 203.28% | 6.77 MiB |
| java_micronaut | 8836 | 79.46 ms | 84.65 ms | 94.15 ms | 2.89 s | 95.84% | 340.04 MiB |
| erlang_grpcbox | 8444 | 118.21 ms | 132.61 ms | 137.34 ms | 147.49 ms | 208.41% | 94.2 MiB |
| elixir_grpc | 8149 | 122.50 ms | 140.60 ms | 146.17 ms | 157.06 ms | 205.02% | 125.42 MiB |
| scala_zio | 7288 | 136.82 ms | 173.21 ms | 188.18 ms | 206.52 ms | 203.33% | 256.41 MiB |
| java_aot | 6217 | 159.05 ms | 291.11 ms | 317.66 ms | 409.58 ms | 194.21% | 208.04 MiB |
| node_grpcjs_st | 6134 | 162.50 ms | 173.34 ms | 177.25 ms | 185.92 ms | 130.39% | 47.79 MiB |
| java_micronaut_workstealing | 6052 | 48.54 ms | 91.35 ms | 98.46 ms | 7.56 s | 66.58% | 348.2 MiB |
| python_async_grpc | 4381 | 227.73 ms | 237.70 ms | 240.66 ms | 249.10 ms | 151.22% | 31.7 MiB |
| python_grpc | 3457 | 288.46 ms | 303.48 ms | 308.30 ms | 319.50 ms | 150.99% | 34.11 MiB |
| d_grpc | 3232 | 306.26 ms | 207.30 ms | 218.66 ms | 7.58 s | 210.75% | 26.84 MiB |
| ruby_grpc | 2445 | 407.44 ms | 444.72 ms | 463.07 ms | 587.83 ms | 125.64% | 37.85 MiB |
| php_grpc | 2084 | 477.75 ms | 498.24 ms | 499.35 ms | 501.80 ms | 204.5% | 67.28 MiB |
-----------------------------------------------------------------------------------------------------------------------------------------
3 CPU Server
-----------------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
-----------------------------------------------------------------------------------------------------------------------------------------
| cpp_grpc_mt | 71806 | 10.86 ms | 18.16 ms | 22.47 ms | 32.25 ms | 232.84% | 35.57 MiB |
| cpp_asio_grpc | 71296 | 10.96 ms | 18.15 ms | 22.44 ms | 32.54 ms | 238.58% | 37.31 MiB |
| dotnet_grpc | 68821 | 11.36 ms | 18.07 ms | 21.74 ms | 31.44 ms | 210.3% | 171.76 MiB |
| rust_grpcio | 68377 | 12.11 ms | 18.79 ms | 22.81 ms | 32.61 ms | 259.81% | 25.3 MiB |
| scala_akka | 67796 | 10.84 ms | 20.08 ms | 26.33 ms | 40.51 ms | 217.8% | 237.17 MiB |
| go_vtgrpc | 67021 | 11.28 ms | 20.28 ms | 24.91 ms | 35.36 ms | 221.08% | 25.52 MiB |
| java_openj9_grpc_gencon | 66107 | 11.76 ms | 22.66 ms | 29.09 ms | 41.96 ms | 257.92% | 125.94 MiB |
| rust_thruster_mt | 65903 | 12.66 ms | 21.38 ms | 26.04 ms | 36.11 ms | 244.4% | 15.03 MiB |
| kotlin_grpc | 64373 | 12.29 ms | 23.02 ms | 31.74 ms | 49.44 ms | 279.07% | 228.26 MiB |
| rust_tonic_mt | 64091 | 13.53 ms | 23.42 ms | 28.17 ms | 37.61 ms | 277.41% | 18.16 MiB |
| go_grpc | 61002 | 13.26 ms | 22.56 ms | 27.46 ms | 40.01 ms | 261.0% | 26.56 MiB |
| java_quarkus | 60239 | 13.76 ms | 23.09 ms | 30.95 ms | 43.21 ms | 202.91% | 134.92 MiB |
| scala_fs2 | 59030 | 14.09 ms | 28.66 ms | 44.58 ms | 67.84 ms | 308.92% | 249.3 MiB |
| java_hotspot_grpc_sgc | 58729 | 13.93 ms | 26.38 ms | 32.01 ms | 40.96 ms | 234.93% | 89.32 MiB |
| java_hotspot_grpc_zgc | 58314 | 14.06 ms | 27.09 ms | 32.88 ms | 42.11 ms | 231.84% | 90.65 MiB |
| java_hotspot_grpc_she | 58232 | 14.09 ms | 27.41 ms | 33.56 ms | 43.60 ms | 230.85% | 86.37 MiB |
| java_hotspot_grpc_pgc | 57841 | 14.17 ms | 27.45 ms | 33.71 ms | 44.40 ms | 233.4% | 84.01 MiB |
| java_hotspot_grpc_g1gc | 57418 | 14.33 ms | 29.52 ms | 35.45 ms | 45.69 ms | 230.32% | 86.1 MiB |
| java_armeria | 44574 | 16.62 ms | 39.99 ms | 48.36 ms | 64.29 ms | 305.94% | 252.17 MiB |
| java_quarkus_native | 41835 | 22.89 ms | 28.84 ms | 31.22 ms | 37.00 ms | 283.7% | 90.57 MiB |
| rust_tonic_st | 41788 | 23.63 ms | 29.23 ms | 30.35 ms | 32.92 ms | 101.52% | 12.15 MiB |
| rust_thruster_st | 41561 | 23.73 ms | 29.27 ms | 30.52 ms | 32.82 ms | 101.74% | 11.16 MiB |
| swift_grpc | 30775 | 32.16 ms | 34.75 ms | 35.66 ms | 37.14 ms | 303.8% | 6.91 MiB |
| crystal_grpc | 27069 | 36.82 ms | 39.47 ms | 40.42 ms | 42.19 ms | 115.84% | 46.51 MiB |
| cpp_grpc_st | 26929 | 37.01 ms | 40.79 ms | 42.37 ms | 44.33 ms | 103.16% | 9.56 MiB |
| scala_zio | 12558 | 78.99 ms | 96.89 ms | 103.71 ms | 123.96 ms | 307.06% | 257.57 MiB |
| java_aot | 11815 | 82.75 ms | 165.91 ms | 191.42 ms | 243.36 ms | 280.4% | 208.04 MiB |
| elixir_grpc | 11270 | 88.58 ms | 104.72 ms | 110.84 ms | 125.03 ms | 307.79% | 133.17 MiB |
| erlang_grpcbox | 9981 | 99.96 ms | 111.06 ms | 114.91 ms | 126.13 ms | 287.02% | 86.4 MiB |
| java_micronaut_workstealing | 9980 | 53.18 ms | 62.48 ms | 73.09 ms | 3.60 s | 103.6% | 362.8 MiB |
| java_micronaut | 7629 | 128.33 ms | 60.03 ms | 71.95 ms | 3.62 s | 77.65% | 369.6 MiB |
| node_grpcjs_st | 6193 | 160.90 ms | 171.92 ms | 176.23 ms | 186.26 ms | 129.64% | 47.1 MiB |
| d_grpc | 6125 | 161.72 ms | 112.34 ms | 128.91 ms | 4.16 s | 319.64% | 26.5 MiB |
| python_async_grpc | 4422 | 225.62 ms | 235.74 ms | 239.11 ms | 250.92 ms | 153.93% | 34.79 MiB |
| php_grpc | 3625 | 275.10 ms | 293.55 ms | 295.30 ms | 298.24 ms | 309.75% | 67.13 MiB |
| python_grpc | 3450 | 289.02 ms | 303.33 ms | 308.49 ms | 318.72 ms | 151.56% | 34.01 MiB |
| ruby_grpc | 2449 | 406.82 ms | 438.07 ms | 446.93 ms | 497.47 ms | 130.64% | 36.99 MiB |
-----------------------------------------------------------------------------------------------------------------------------------------