-
Notifications
You must be signed in to change notification settings - Fork 224
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce cycle-based gas cost model. #4974
Merged
casperlabs-bors-ng
merged 6 commits into
casper-network:feat-2.0
from
mpapierski:update-opcode-costs
Nov 25, 2024
Merged
Introduce cycle-based gas cost model. #4974
casperlabs-bors-ng
merged 6 commits into
casper-network:feat-2.0
from
mpapierski:update-opcode-costs
Nov 25, 2024
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR introduces a revised gas cost calculation model for the WebAssembly (Wasm) interpreter. The model ensures fair and adaptable gas costs by normalizing execution times and accounting for interpreter overhead.
mpapierski
requested review from
EdHastingsCasperAssociation,
igor-casper,
AlexanderLimonov and
darthsiroftardis
November 22, 2024 15:00
Updates multipliers based on the output of a benchmark that also includes modified wasm with gas injector and stack limiter. This benchmark output should be as close as possible to measure real overhead of running modified Wasm.
Conflicts: execution_engine_testing/tests/src/test/explorer/faucet.rs
darthsiroftardis
approved these changes
Nov 25, 2024
bors r+ |
casperlabs-bors-ng bot
added a commit
that referenced
this pull request
Nov 25, 2024
4974: Introduce cycle-based gas cost model. r=darthsiroftardis a=mpapierski This PR proposes a new approach to calculating gas costs for the Wasm interpreter that makes the system more fair, predictable, and adaptable to changing conditions. The idea is to base the gas costs on execution cycles rather than raw execution times, which allows us to better normalize and compare opcodes even in the presence of interpreter overhead. The model works like this: each opcode's gas cost is calculated as the product of its "cycle count" and a "gas cost multiplier." The cycle count is determined through benchmarking, using the `nop` instruction as the baseline. By dividing the execution time of each opcode by the time it takes to execute `nop`, we get a relative measure of how expensive each opcode is compared to the simplest operation. Nop is selected as the baseline because in the Wasm interpreter it does not modify stack, and does not produce new stack elements. It simply advances the program counter. The gas cost multiplier, on the other hand, is an adjustable parameter that we can tweak to account for the overhead introduced by the interpreter itself or to align gas costs with network goals, like a target maximum gas per block. The overhead includes a gas counter injected into the Wasm, while the cycles were computed based on unmodified Wasm programs. This approach makes it easier to adapt gas costs to real-world performance changes. For example, if new benchmarks show higher overhead due to updates in the VM or interpreter, we can adjust the gas cost multiplier without recalculating the cycle counts. Similarly, this flexibility allows us to scale gas costs to fit varying hardware or network configurations. The main benefit of this model is that it ensures gas costs are proportional to actual computational effort, without relying on assumptions about the hardware-specific details of CPU cycles, which aren't directly observable in an interpreted environment. This way, we can make gas costs consistent and fair while still being able to respond to changes as needed. To validate this, I've run benchmarks on representative workloads to measure execution times for various opcodes and determine the cycle counts relative to nop. I've also tested the system under gas-injected Wasm conditions to measure the interpreter's overhead and adjust the gas cost multiplier accordingly. These tests ensure that the costs are reasonable for typical programs and transactions. This system is particularly useful because it decouples the underlying VM performance from the gas accounting logic, allowing for future optimizations without breaking the current cost structure. It's a step forward in making gas costs fairer and more predictable while maintaining the flexibility to adjust as the blockchain ecosystem evolves. Reference hardware used for computing cycles and gas cost multipliers is Ryzen 3950X (https://www.cpubenchmark.net/cpu.php?id=3598&cpu=AMD+Ryzen+9+3950X). The gas cost multipliers in this PR are calculated for **200 CSPR** `block_gas_limit` (changed from 3300 CSPR) and **16384ms** `minimum_block_time`. These parameters allow us to lower the gas costs for computational Wasm (i.e. those not using storage). # Impact As measured in `faucet` smart contract inside the repo here's the cost difference | Entrypoint | Previous cost | New cost | Change | | ------------- | ------------- | ------------- | ------------- | | install | 143_782_934_244 | 141_835_235_435 | -2% | | set_variables | 143_866_090 | 70_639_675 | -51% | | call_by_installer | 2_705_857_043 | 2_644_931_333 | -2.25% | | call_by_user | 2_644_397_116 | 2_545_472_446 | -3.74% | Benchmarking data and results used in these calculations are here https://gist.github.com/mpapierski/a3a3856e64197a4aea0aae24fdce11c8 # Speed Slow Wasm timeout (GasLimit error) as measured by the `run_wasm` tool | Computation | Timeout | | ------------- | ------------- | | `fibonacci_rec(40)` | 14.32s | | `infinite_loop()` | 9.39s | | `infinite_loop_br_if()` | 8.79s | | `countPrimes(0, 9223372036854775807)` | 6.98s | Ideally, each timeout will be reached within the same amount of time, although the issue is that the overhead of the gas injector is non-deterministic and difficult to measure within the benchmark. As seen in the benchmark the overhead of gas injector is less visible around arithmetic operations, and memory operations, but more visible around control operations. This is because the gas injector can decide to group costs of multiple opcodes together and call the gas counter around control op structures (i.e. call, block, loop) Closes #4951 Co-authored-by: Michał Papierski <michal@casper.network> Co-authored-by: Michał Papierski <michal@papierski.net>
Build failed: |
bors r+ |
Build succeeded: |
casperlabs-bors-ng
bot
merged commit Nov 25, 2024
6191928
into
casper-network:feat-2.0
3 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR proposes a new approach to calculating gas costs for the Wasm interpreter that makes the system more fair, predictable, and adaptable to changing conditions. The idea is to base the gas costs on execution cycles rather than raw execution times, which allows us to better normalize and compare opcodes even in the presence of interpreter overhead.
The model works like this: each opcode's gas cost is calculated as the product of its "cycle count" and a "gas cost multiplier." The cycle count is determined through benchmarking, using the
nop
instruction as the baseline. By dividing the execution time of each opcode by the time it takes to executenop
, we get a relative measure of how expensive each opcode is compared to the simplest operation. Nop is selected as the baseline because in the Wasm interpreter it does not modify stack, and does not produce new stack elements. It simply advances the program counter. The gas cost multiplier, on the other hand, is an adjustable parameter that we can tweak to account for the overhead introduced by the interpreter itself or to align gas costs with network goals, like a target maximum gas per block. The overhead includes a gas counter injected into the Wasm, while the cycles were computed based on unmodified Wasm programs.This approach makes it easier to adapt gas costs to real-world performance changes. For example, if new benchmarks show higher overhead due to updates in the VM or interpreter, we can adjust the gas cost multiplier without recalculating the cycle counts. Similarly, this flexibility allows us to scale gas costs to fit varying hardware or network configurations.
The main benefit of this model is that it ensures gas costs are proportional to actual computational effort, without relying on assumptions about the hardware-specific details of CPU cycles, which aren't directly observable in an interpreted environment. This way, we can make gas costs consistent and fair while still being able to respond to changes as needed.
To validate this, I've run benchmarks on representative workloads to measure execution times for various opcodes and determine the cycle counts relative to nop. I've also tested the system under gas-injected Wasm conditions to measure the interpreter's overhead and adjust the gas cost multiplier accordingly. These tests ensure that the costs are reasonable for typical programs and transactions.
This system is particularly useful because it decouples the underlying VM performance from the gas accounting logic, allowing for future optimizations without breaking the current cost structure. It's a step forward in making gas costs fairer and more predictable while maintaining the flexibility to adjust as the blockchain ecosystem evolves.
Reference hardware used for computing cycles and gas cost multipliers is Ryzen 3950X (https://www.cpubenchmark.net/cpu.php?id=3598&cpu=AMD+Ryzen+9+3950X). The gas cost multipliers in this PR are calculated for 200 CSPR
block_gas_limit
(changed from 3300 CSPR) and 16384msminimum_block_time
. These parameters allow us to lower the gas costs for computational Wasm (i.e. those not using storage).Impact
As measured in
faucet
smart contract inside the repo here's the cost differenceBenchmarking data and results used in these calculations are here https://gist.github.com/mpapierski/a3a3856e64197a4aea0aae24fdce11c8
Speed
Slow Wasm timeout (GasLimit error) as measured by the
run_wasm
toolfibonacci_rec(40)
infinite_loop()
infinite_loop_br_if()
countPrimes(0, 9223372036854775807)
Ideally, each timeout will be reached within the same amount of time, although the issue is that the overhead of the gas injector is non-deterministic and difficult to measure within the benchmark. As seen in the benchmark the overhead of gas injector is less visible around arithmetic operations, and memory operations, but more visible around control operations. This is because the gas injector can decide to group costs of multiple opcodes together and call the gas counter around control op structures (i.e. call, block, loop)
Closes #4951