Skip to content

Commit

Permalink
Added local BenchmarkMockNet runs for 8.0.0 work
Browse files Browse the repository at this point in the history
  • Loading branch information
@JasonBock
JasonBock committed Jan 22, 2024
1 parent 34f8fde commit eacb0e8
Show file tree
Hide file tree
Showing 4 changed files with 340 additions and 0 deletions.
85 changes: 85 additions & 0 deletions docs/notes/Rocks 7.3.0 - BenchmarkMockNet (local).md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
Rocks 7.3.0

Construction

| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|------------- |-------------:|--------------:|------------:|---------:|--------:|-------:|-------:|----------:|------------:|
| Stub | 22.35 ns | 222.69 ns | 12.21 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
| FakeItEasy | 3,721.20 ns | 42,854.45 ns | 2,349.00 ns | 160.75 | 15.97 | 0.6700 | 0.0100 | 2964 B | 123.50 |
| JustMockLite | 29,366.68 ns | 118,477.37 ns | 6,494.14 ns | 1,515.94 | 629.67 | 4.1500 | - | 17389 B | 724.54 |
| Moq | 4,534.82 ns | 36,646.38 ns | 2,008.71 ns | 210.13 | 44.06 | 0.4900 | - | 2088 B | 87.00 |
| NSubstitute | 4,226.07 ns | 49,172.01 ns | 2,695.28 ns | 183.57 | 17.47 | 1.2300 | - | 5176 B | 215.67 |
| PCLMock | 64.16 ns | 424.79 ns | 23.28 ns | 3.08 | 0.58 | 0.0300 | - | 144 B | 6.00 |
| Rocks | 70.66 ns | 576.14 ns | 31.58 ns | 3.28 | 0.33 | 0.0300 | - | 144 B | 6.00 |

Return

| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|------------- |--------------:|--------------:|-------------:|--------------:|----------:|---------:|-------:|-------:|----------:|------------:|
| Stub | 18.44 ns | 148.50 ns | 8.140 ns | 22.93 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
| FakeItEasy | 6,718.85 ns | 85,271.92 ns | 4,674.042 ns | 5,063.88 ns | 360.89 | 153.90 | 1.1900 | 0.0100 | 5057 B | 210.71 |
| JustMockLite | 166,670.01 ns | 139,674.39 ns | 7,656.025 ns | 162,260.68 ns | 10,850.83 | 6,155.68 | 8.4500 | 0.0200 | 35306 B | 1,471.08 |
| Moq | 6,852.76 ns | 86,662.92 ns | 4,750.287 ns | 5,777.26 ns | 358.40 | 147.23 | 0.9400 | - | 3953 B | 164.71 |
| NSubstitute | 6,082.31 ns | 94,397.03 ns | 5,174.220 ns | 3,162.29 ns | 332.07 | 195.17 | 1.8200 | - | 7649 B | 318.71 |
| PCLMock | 3,480.24 ns | 12,305.70 ns | 674.517 ns | 3,580.26 ns | 212.53 | 81.44 | 0.4400 | - | 1872 B | 78.00 |
| Rocks | 398.05 ns | 662.71 ns | 36.325 ns | 378.65 ns | 25.65 | 13.84 | 0.1600 | - | 680 B | 28.33 |

Empty Return

| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|------------- |-------------:|--------------:|-------------:|-------------:|---------:|--------:|-------:|-------:|----------:|------------:|
| Stub | 21.55 ns | 119.21 ns | 6.534 ns | 22.17 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
| FakeItEasy | 4,759.21 ns | 61,983.26 ns | 3,397.512 ns | 3,465.34 ns | 205.29 | 91.02 | 0.8700 | 0.0200 | 3836 B | 159.83 |
| JustMockLite | 34,959.05 ns | 139,014.15 ns | 7,619.835 ns | 30,656.16 ns | 1,675.72 | 353.15 | 5.0800 | - | 21270 B | 886.25 |
| Moq | 5,120.01 ns | 52,581.16 ns | 2,882.151 ns | 6,009.53 ns | 222.80 | 81.39 | 0.5800 | - | 2457 B | 102.38 |
| NSubstitute | 5,162.91 ns | 71,626.95 ns | 3,926.115 ns | 3,312.66 ns | 222.62 | 109.47 | 1.4000 | - | 5873 B | 244.71 |
| PCLMock | 3,465.24 ns | 13,632.91 ns | 747.266 ns | 3,673.85 ns | 163.95 | 15.93 | 0.4400 | - | 1872 B | 78.00 |
| Rocks | 419.60 ns | 952.68 ns | 52.219 ns | 398.45 ns | 20.37 | 4.78 | 0.1600 | - | 680 B | 28.33 |

Empty Method

| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|------------- |-------------:|--------------:|-------------:|-------------:|---------:|--------:|-------:|-------:|----------:|------------:|
| Stub | 18.76 ns | 134.45 ns | 7.370 ns | 20.28 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
| FakeItEasy | 4,869.66 ns | 45,523.86 ns | 2,495.316 ns | 3,459.90 ns | 264.58 | 81.53 | 0.8700 | 0.0200 | 3812 B | 158.83 |
| JustMockLite | 34,544.95 ns | 121,045.86 ns | 6,634.932 ns | 31,079.23 ns | 2,009.40 | 709.52 | 5.0600 | - | 21166 B | 881.92 |
| Moq | 4,642.43 ns | 51,362.37 ns | 2,815.345 ns | 4,682.56 ns | 231.19 | 63.20 | 0.5800 | - | 2432 B | 101.33 |
| NSubstitute | 5,076.18 ns | 72,624.17 ns | 3,980.776 ns | 3,140.69 ns | 254.53 | 116.34 | 1.4000 | - | 5873 B | 244.71 |
| PCLMock | 3,239.01 ns | 15,873.23 ns | 870.066 ns | 3,438.58 ns | 180.12 | 28.82 | 0.4300 | - | 1832 B | 76.33 |
| Rocks | 376.55 ns | 1,026.68 ns | 56.276 ns | 354.20 ns | 22.02 | 7.91 | 0.1600 | - | 672 B | 28.00 |

OneParameter

| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|------------- |--------------:|--------------:|------------:|--------------:|---------:|---------:|-------:|-------:|----------:|------------:|
| Stub | 24.59 ns | 227.14 ns | 12.45 ns | 19.92 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
| FakeItEasy | 4,884.48 ns | 52,719.64 ns | 2,889.74 ns | 4,320.82 ns | 192.33 | 34.41 | 0.9000 | 0.0200 | 3949 B | 164.54 |
| JustMockLite | 34,645.03 ns | 121,741.32 ns | 6,673.05 ns | 30,816.46 ns | 1,557.59 | 469.97 | 5.1700 | - | 21630 B | 901.25 |
| Moq | 5,112.96 ns | 41,863.36 ns | 2,294.67 ns | 5,631.68 ns | 212.69 | 60.91 | 0.5800 | - | 2464 B | 102.67 |
| NSubstitute | 5,375.58 ns | 79,368.09 ns | 4,350.43 ns | 3,298.42 ns | 198.52 | 60.27 | 1.4100 | - | 5905 B | 246.04 |
| PCLMock | 122,427.05 ns | 77,230.61 ns | 4,233.27 ns | 120,159.82 ns | 5,743.06 | 2,322.57 | 2.5100 | 2.5000 | 10523 B | 438.46 |
| Rocks | 533.57 ns | 1,139.98 ns | 62.49 ns | 504.15 ns | 24.46 | 8.42 | 0.1800 | - | 792 B | 33.00 |

Callback

| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|------------- |--------------:|--------------:|-------------:|--------------:|---------:|---------:|-------:|-------:|----------:|------------:|
| Stub | 17.44 ns | 107.40 ns | 5.887 ns | 18.62 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
| FakeItEasy | 6,644.55 ns | 83,592.48 ns | 4,581.986 ns | 4,889.83 ns | 358.33 | 143.23 | 1.2400 | 0.0100 | 5266 B | 219.42 |
| JustMockLite | 155,017.08 ns | 114,342.12 ns | 6,267.478 ns | 152,198.44 ns | 9,673.61 | 3,577.65 | 8.3600 | 0.0200 | 34958 B | 1,456.58 |
| Moq | 6,598.56 ns | 92,117.99 ns | 5,049.298 ns | 4,613.61 ns | 349.92 | 168.84 | 0.9800 | - | 4113 B | 171.38 |
| NSubstitute | 5,722.94 ns | 81,504.82 ns | 4,467.554 ns | 3,148.37 ns | 311.17 | 157.48 | 1.6400 | - | 6897 B | 287.38 |
| PCLMock | 3,331.23 ns | 23,065.06 ns | 1,264.274 ns | 3,273.52 ns | 189.87 | 14.19 | 0.4800 | - | 2040 B | 85.00 |
| Rocks | 390.66 ns | 486.62 ns | 26.673 ns | 384.63 ns | 24.36 | 9.06 | 0.1800 | - | 760 B | 31.67 |

Verify

| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|------------- |-------------:|--------------:|-------------:|-------------:|---------:|---------:|-------:|-------:|----------:|------------:|
| Stub | 18.81 ns | 128.20 ns | 7.027 ns | 19.81 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
| FakeItEasy | 6,574.53 ns | 89,425.76 ns | 4,901.728 ns | 3,876.04 ns | 332.75 | 144.56 | 1.3200 | 0.0100 | 5587 B | 232.79 |
| JustMockLite | 71,765.15 ns | 167,640.49 ns | 9,188.941 ns | 66,666.49 ns | 4,160.54 | 1,489.21 | 9.7900 | - | 40966 B | 1,706.92 |
| Moq | 6,766.84 ns | 96,079.48 ns | 5,266.441 ns | 4,337.44 ns | 334.55 | 151.89 | 0.9700 | - | 4073 B | 169.71 |
| NSubstitute | 6,318.10 ns | 81,011.32 ns | 4,440.504 ns | 3,813.94 ns | 323.68 | 130.06 | 1.7800 | - | 7465 B | 311.04 |
| PCLMock | 3,715.61 ns | 36,602.36 ns | 2,006.299 ns | 3,516.03 ns | 189.21 | 36.32 | 0.6300 | - | 2656 B | 110.67 |
| Rocks | 601.35 ns | 697.06 ns | 38.208 ns | 590.94 ns | 35.17 | 13.22 | 0.1800 | - | 760 B | 31.67 |
85 changes: 85 additions & 0 deletions docs/notes/Rocks 8.0.0-alpha.1 - BenchmarkMockNet (local).md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
Rocks 8.0.0-alpha.1

Construction

| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|------------- |-------------:|-------------:|------------:|---------:|--------:|-------:|-------:|----------:|------------:|
| Stub | 20.71 ns | 184.21 ns | 10.10 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
| FakeItEasy | 3,734.84 ns | 36,987.36 ns | 2,027.40 ns | 174.67 | 36.72 | 0.6700 | 0.0100 | 2964 B | 123.50 |
| JustMockLite | 25,827.57 ns | 61,063.58 ns | 3,347.10 ns | 1,450.81 | 684.44 | 4.1500 | - | 17389 B | 724.54 |
| Moq | 4,381.12 ns | 36,333.57 ns | 1,991.57 ns | 212.80 | 28.50 | 0.4900 | - | 2088 B | 87.00 |
| NSubstitute | 4,157.98 ns | 48,785.65 ns | 2,674.11 ns | 189.47 | 34.83 | 1.2300 | - | 5176 B | 215.67 |
| PCLMock | 58.76 ns | 348.16 ns | 19.08 ns | 3.09 | 0.99 | 0.0300 | - | 144 B | 6.00 |
| Rocks | 124.80 ns | 440.59 ns | 24.15 ns | 6.83 | 2.73 | 0.0600 | - | 288 B | 12.00 |

Return

| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|------------- |--------------:|--------------:|-------------:|--------------:|----------:|---------:|-------:|-------:|----------:|------------:|
| Stub | 15.67 ns | 117.97 ns | 6.467 ns | 18.45 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
| FakeItEasy | 5,930.73 ns | 81,170.98 ns | 4,449.255 ns | 3,749.64 ns | 369.89 | 170.91 | 1.1900 | 0.0100 | 5057 B | 210.71 |
| JustMockLite | 161,469.21 ns | 124,598.96 ns | 6,829.690 ns | 158,578.74 ns | 11,952.59 | 6,031.00 | 8.4500 | 0.0200 | 35301 B | 1,470.88 |
| Moq | 6,671.32 ns | 86,070.63 ns | 4,717.822 ns | 5,372.21 ns | 403.01 | 160.45 | 0.9400 | - | 3953 B | 164.71 |
| NSubstitute | 5,929.91 ns | 93,825.96 ns | 5,142.918 ns | 3,019.86 ns | 366.48 | 211.17 | 1.8200 | - | 7649 B | 318.71 |
| PCLMock | 3,052.42 ns | 18,041.38 ns | 988.909 ns | 3,064.41 ns | 204.56 | 41.51 | 0.4400 | - | 1872 B | 78.00 |
| Rocks | 217.72 ns | 352.65 ns | 19.330 ns | 210.18 ns | 15.92 | 7.48 | 0.1000 | - | 448 B | 18.67 |

Empty Return

| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|------------- |-------------:|--------------:|-------------:|-------------:|---------:|---------:|-------:|-------:|----------:|------------:|
| Stub | 15.62 ns | 118.11 ns | 6.474 ns | 18.87 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
| FakeItEasy | 4,409.87 ns | 47,890.46 ns | 2,625.038 ns | 3,928.52 ns | 275.17 | 81.03 | 0.8700 | 0.0100 | 3836 B | 159.83 |
| JustMockLite | 33,764.59 ns | 140,984.03 ns | 7,727.810 ns | 29,551.15 ns | 2,425.58 | 1,023.72 | 5.0800 | - | 21279 B | 886.62 |
| Moq | 5,148.24 ns | 46,838.51 ns | 2,567.377 ns | 5,706.40 ns | 320.86 | 45.53 | 0.5800 | - | 2457 B | 102.38 |
| NSubstitute | 5,091.48 ns | 72,864.49 ns | 3,993.949 ns | 3,248.96 ns | 315.99 | 159.71 | 1.4000 | - | 5873 B | 244.71 |
| PCLMock | 3,260.64 ns | 16,519.54 ns | 905.492 ns | 3,417.75 ns | 222.25 | 51.50 | 0.4400 | - | 1872 B | 78.00 |
| Rocks | 214.33 ns | 389.98 ns | 21.376 ns | 213.71 ns | 15.63 | 6.96 | 0.1000 | - | 448 B | 18.67 |

Empty Method

| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|------------- |-------------:|--------------:|-------------:|-------------:|---------:|--------:|-------:|-------:|----------:|------------:|
| Stub | 21.42 ns | 143.77 ns | 7.880 ns | 20.79 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
| FakeItEasy | 4,446.38 ns | 50,979.86 ns | 2,794.378 ns | 3,742.79 ns | 194.53 | 53.97 | 0.8700 | 0.0200 | 3812 B | 158.83 |
| JustMockLite | 32,608.67 ns | 124,306.27 ns | 6,813.646 ns | 28,867.69 ns | 1,603.16 | 389.95 | 5.0600 | - | 21166 B | 881.92 |
| Moq | 4,686.23 ns | 41,891.53 ns | 2,296.216 ns | 5,032.26 ns | 210.90 | 43.49 | 0.5800 | - | 2432 B | 101.33 |
| NSubstitute | 4,960.45 ns | 71,947.15 ns | 3,943.666 ns | 3,024.27 ns | 212.15 | 95.00 | 1.4000 | - | 5873 B | 244.71 |
| PCLMock | 3,389.81 ns | 12,969.23 ns | 710.887 ns | 3,724.95 ns | 165.16 | 29.92 | 0.4300 | - | 1832 B | 76.33 |
| Rocks | 216.25 ns | 410.67 ns | 22.510 ns | 215.65 ns | 10.81 | 2.98 | 0.1000 | - | 440 B | 18.33 |

OneParameter

| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|------------- |--------------:|--------------:|-------------:|--------------:|---------:|---------:|-------:|-------:|----------:|------------:|
| Stub | 17.49 ns | 136.58 ns | 7.486 ns | 18.40 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
| FakeItEasy | 4,246.27 ns | 58,232.17 ns | 3,191.902 ns | 2,770.63 ns | 229.46 | 87.21 | 0.9000 | 0.0200 | 3949 B | 164.54 |
| JustMockLite | 32,927.57 ns | 126,480.63 ns | 6,932.830 ns | 28,935.98 ns | 2,086.50 | 805.65 | 5.1700 | - | 21630 B | 901.25 |
| Moq | 4,977.14 ns | 41,904.89 ns | 2,296.949 ns | 5,561.89 ns | 280.01 | 23.80 | 0.5800 | - | 2464 B | 102.67 |
| NSubstitute | 5,323.26 ns | 71,902.95 ns | 3,941.243 ns | 3,824.44 ns | 284.37 | 101.95 | 1.4100 | - | 5905 B | 246.04 |
| PCLMock | 118,789.98 ns | 69,828.52 ns | 3,827.537 ns | 116,583.37 ns | 7,841.57 | 3,793.29 | 2.5100 | 2.5000 | 10523 B | 438.46 |
| Rocks | 283.42 ns | 1,007.11 ns | 55.203 ns | 266.61 ns | 17.82 | 6.12 | 0.1200 | - | 504 B | 21.00 |

Callback

| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|------------- |--------------:|--------------:|-------------:|--------------:|---------:|---------:|-------:|-------:|----------:|------------:|
| Stub | 19.13 ns | 101.14 ns | 5.544 ns | 21.52 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
| FakeItEasy | 6,019.04 ns | 78,925.07 ns | 4,326.149 ns | 3,809.42 ns | 302.45 | 155.74 | 1.2400 | 0.0100 | 5266 B | 219.42 |
| JustMockLite | 151,002.80 ns | 163,269.39 ns | 8,949.346 ns | 147,195.42 ns | 8,403.40 | 2,690.19 | 8.3600 | 0.0200 | 34954 B | 1,456.42 |
| Moq | 6,718.86 ns | 85,304.47 ns | 4,675.826 ns | 5,507.64 ns | 329.02 | 162.08 | 0.9800 | - | 4113 B | 171.38 |
| NSubstitute | 5,885.63 ns | 88,402.21 ns | 4,845.623 ns | 3,131.02 ns | 293.65 | 182.44 | 1.6400 | - | 6897 B | 287.38 |
| PCLMock | 3,915.24 ns | 16,165.69 ns | 886.096 ns | 4,259.72 ns | 207.88 | 16.88 | 0.4800 | - | 2040 B | 85.00 |
| Rocks | 267.53 ns | 466.11 ns | 25.549 ns | 254.05 ns | 14.78 | 4.26 | 0.1200 | - | 528 B | 22.00 |

Verify

| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
|------------- |-------------:|--------------:|-------------:|-------------:|---------:|--------:|-------:|-------:|----------:|------------:|
| Stub | 23.49 ns | 127.27 ns | 6.976 ns | 27.03 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
| FakeItEasy | 6,601.21 ns | 84,476.12 ns | 4,630.421 ns | 4,417.35 ns | 274.40 | 148.01 | 1.3200 | 0.0100 | 5587 B | 232.79 |
| JustMockLite | 69,376.99 ns | 161,471.42 ns | 8,850.794 ns | 64,612.77 ns | 3,137.10 | 963.13 | 9.7900 | - | 40966 B | 1,706.92 |
| Moq | 6,603.50 ns | 93,648.69 ns | 5,133.201 ns | 4,393.31 ns | 269.68 | 166.85 | 0.9700 | - | 4073 B | 169.71 |
| NSubstitute | 6,362.81 ns | 83,918.21 ns | 4,599.840 ns | 3,797.83 ns | 267.24 | 150.79 | 1.7800 | - | 7465 B | 311.04 |
| PCLMock | 4,235.99 ns | 29,775.16 ns | 1,632.077 ns | 4,622.20 ns | 177.38 | 27.30 | 0.6300 | - | 2656 B | 110.67 |
| Rocks | 1,213.64 ns | 1,180.71 ns | 64.719 ns | 1,231.39 ns | 54.92 | 16.44 | 0.4000 | - | 1712 B | 71.33 |
Loading

0 comments on commit eacb0e8

Please sign in to comment.