Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce cycle-based gas cost model. #4974

Merged

Conversation

mpapierski
Copy link
Collaborator

@mpapierski mpapierski commented Nov 22, 2024

This PR proposes a new approach to calculating gas costs for the Wasm interpreter that makes the system more fair, predictable, and adaptable to changing conditions. The idea is to base the gas costs on execution cycles rather than raw execution times, which allows us to better normalize and compare opcodes even in the presence of interpreter overhead.

The model works like this: each opcode's gas cost is calculated as the product of its "cycle count" and a "gas cost multiplier." The cycle count is determined through benchmarking, using the nop instruction as the baseline. By dividing the execution time of each opcode by the time it takes to execute nop, we get a relative measure of how expensive each opcode is compared to the simplest operation. Nop is selected as the baseline because in the Wasm interpreter it does not modify stack, and does not produce new stack elements. It simply advances the program counter. The gas cost multiplier, on the other hand, is an adjustable parameter that we can tweak to account for the overhead introduced by the interpreter itself or to align gas costs with network goals, like a target maximum gas per block. The overhead includes a gas counter injected into the Wasm, while the cycles were computed based on unmodified Wasm programs.

This approach makes it easier to adapt gas costs to real-world performance changes. For example, if new benchmarks show higher overhead due to updates in the VM or interpreter, we can adjust the gas cost multiplier without recalculating the cycle counts. Similarly, this flexibility allows us to scale gas costs to fit varying hardware or network configurations.

The main benefit of this model is that it ensures gas costs are proportional to actual computational effort, without relying on assumptions about the hardware-specific details of CPU cycles, which aren't directly observable in an interpreted environment. This way, we can make gas costs consistent and fair while still being able to respond to changes as needed.

To validate this, I've run benchmarks on representative workloads to measure execution times for various opcodes and determine the cycle counts relative to nop. I've also tested the system under gas-injected Wasm conditions to measure the interpreter's overhead and adjust the gas cost multiplier accordingly. These tests ensure that the costs are reasonable for typical programs and transactions.

This system is particularly useful because it decouples the underlying VM performance from the gas accounting logic, allowing for future optimizations without breaking the current cost structure. It's a step forward in making gas costs fairer and more predictable while maintaining the flexibility to adjust as the blockchain ecosystem evolves.

Reference hardware used for computing cycles and gas cost multipliers is Ryzen 3950X (https://www.cpubenchmark.net/cpu.php?id=3598&cpu=AMD+Ryzen+9+3950X). The gas cost multipliers in this PR are calculated for 200 CSPR block_gas_limit (changed from 3300 CSPR) and 16384ms minimum_block_time. These parameters allow us to lower the gas costs for computational Wasm (i.e. those not using storage).

Impact

As measured in faucet smart contract inside the repo here's the cost difference

Entrypoint Previous cost New cost Change
install 143_782_934_244 141_835_235_435 -2%
set_variables 143_866_090 70_639_675 -51%
call_by_installer 2_705_857_043 2_644_931_333 -2.25%
call_by_user 2_644_397_116 2_545_472_446 -3.74%

Benchmarking data and results used in these calculations are here https://gist.github.com/mpapierski/a3a3856e64197a4aea0aae24fdce11c8

Speed

Slow Wasm timeout (GasLimit error) as measured by the run_wasm tool

Computation Timeout
fibonacci_rec(40) 14.32s
infinite_loop() 9.39s
infinite_loop_br_if() 8.79s
countPrimes(0, 9223372036854775807) 6.98s

Ideally, each timeout will be reached within the same amount of time, although the issue is that the overhead of the gas injector is non-deterministic and difficult to measure within the benchmark. As seen in the benchmark the overhead of gas injector is less visible around arithmetic operations, and memory operations, but more visible around control operations. This is because the gas injector can decide to group costs of multiple opcodes together and call the gas counter around control op structures (i.e. call, block, loop)

Closes #4951

This PR introduces a revised gas cost calculation model for the
WebAssembly (Wasm) interpreter. The model ensures fair and adaptable gas
costs by normalizing execution times and accounting for interpreter overhead.
mpapierski and others added 2 commits November 22, 2024 16:07
Updates multipliers based on the output of a benchmark that also
includes modified wasm with gas injector and stack limiter. This
benchmark output should be as close as possible to measure real
overhead of running modified Wasm.
@mpapierski mpapierski marked this pull request as ready for review November 25, 2024 15:49
mpapierski and others added 2 commits November 25, 2024 15:59
 Conflicts:
	execution_engine_testing/tests/src/test/explorer/faucet.rs
Co-authored-by: Karan Dhareshwar <karan@casperlabs.io>
@darthsiroftardis
Copy link
Contributor

bors r+

casperlabs-bors-ng bot added a commit that referenced this pull request Nov 25, 2024
4974: Introduce cycle-based gas cost model. r=darthsiroftardis a=mpapierski

This PR proposes a new approach to calculating gas costs for the Wasm interpreter that makes the system more fair, predictable, and adaptable to changing conditions. The idea is to base the gas costs on execution cycles rather than raw execution times, which allows us to better normalize and compare opcodes even in the presence of interpreter overhead.

The model works like this: each opcode's gas cost is calculated as the product of its "cycle count" and a "gas cost multiplier." The cycle count is determined through benchmarking, using the `nop` instruction as the baseline. By dividing the execution time of each opcode by the time it takes to execute `nop`, we get a relative measure of how expensive each opcode is compared to the simplest operation. Nop is selected as the baseline because in the Wasm interpreter it does not modify stack, and does not produce new stack elements. It simply advances the program counter.  The gas cost multiplier, on the other hand, is an adjustable parameter that we can tweak to account for the overhead introduced by the interpreter itself or to align gas costs with network goals, like a target maximum gas per block. The overhead includes a gas counter injected into the Wasm, while the cycles were computed based on unmodified Wasm programs.

This approach makes it easier to adapt gas costs to real-world performance changes. For example, if new benchmarks show higher overhead due to updates in the VM or interpreter, we can adjust the gas cost multiplier without recalculating the cycle counts. Similarly, this flexibility allows us to scale gas costs to fit varying hardware or network configurations.

The main benefit of this model is that it ensures gas costs are proportional to actual computational effort, without relying on assumptions about the hardware-specific details of CPU cycles, which aren't directly observable in an interpreted environment. This way, we can make gas costs consistent and fair while still being able to respond to changes as needed.

To validate this, I've run benchmarks on representative workloads to measure execution times for various opcodes and determine the cycle counts relative to nop. I've also tested the system under gas-injected Wasm conditions to measure the interpreter's overhead and adjust the gas cost multiplier accordingly. These tests ensure that the costs are reasonable for typical programs and transactions.

This system is particularly useful because it decouples the underlying VM performance from the gas accounting logic, allowing for future optimizations without breaking the current cost structure. It's a step forward in making gas costs fairer and more predictable while maintaining the flexibility to adjust as the blockchain ecosystem evolves.

Reference hardware used for computing cycles and gas cost multipliers is Ryzen 3950X (https://www.cpubenchmark.net/cpu.php?id=3598&cpu=AMD+Ryzen+9+3950X). The gas cost multipliers in this PR are calculated for **200 CSPR** `block_gas_limit` (changed from 3300 CSPR) and **16384ms** `minimum_block_time`. These parameters allow us to lower the gas costs for computational Wasm (i.e. those not using storage).

# Impact

As measured in `faucet` smart contract inside the repo here's the cost difference

| Entrypoint  | Previous cost | New cost | Change |
| -------------  | ------------- |  ------------- |  ------------- | 
| install              | 143_782_934_244  |  141_835_235_435 | -2% |
| set_variables  | 143_866_090  | 70_639_675 | -51% | 
| call_by_installer | 2_705_857_043 | 2_644_931_333 | -2.25% |
| call_by_user | 2_644_397_116 | 2_545_472_446 | -3.74% |

Benchmarking data and results used in these calculations are here https://gist.github.com/mpapierski/a3a3856e64197a4aea0aae24fdce11c8

# Speed

Slow Wasm timeout (GasLimit error) as measured by the `run_wasm` tool

| Computation  | Timeout |
| -------------  | ------------- |
| `fibonacci_rec(40)` | 14.32s  |
| `infinite_loop()`  | 9.39s  |
| `infinite_loop_br_if()` | 8.79s |
| `countPrimes(0, 9223372036854775807)`  | 6.98s |

Ideally, each timeout will be reached within the same amount of time, although the issue is that the overhead of the gas injector is non-deterministic and difficult to measure within the benchmark. As seen in the benchmark the overhead of gas injector is less visible around arithmetic operations, and memory operations, but more visible around control operations. This is because the gas injector can decide to group costs of multiple opcodes together and call the gas counter around control op structures (i.e. call, block, loop)

Closes #4951

Co-authored-by: Michał Papierski <michal@casper.network>
Co-authored-by: Michał Papierski <michal@papierski.net>
Copy link
Contributor

Build failed:

@darthsiroftardis
Copy link
Contributor

bors r+

Copy link
Contributor

Build succeeded:

@casperlabs-bors-ng casperlabs-bors-ng bot merged commit 6191928 into casper-network:feat-2.0 Nov 25, 2024
3 checks passed
@devendran-m devendran-m added the rc-5 Release Candidate 5 label Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
rc-5 Release Candidate 5
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants