Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(blockifier): update test_insufficient_max_fee_reverts to include new resource bounds #1313

Conversation

dorimedini-starkware
Copy link
Collaborator

@dorimedini-starkware dorimedini-starkware commented Oct 10, 2024

This change is Reviewable

Copy link

codecov bot commented Oct 10, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.46%. Comparing base (e3165c4) to head (f0f7db2).
Report is 81 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff             @@
##             main    #1313       +/-   ##
===========================================
+ Coverage   40.10%   77.46%   +37.36%     
===========================================
  Files          26      100       +74     
  Lines        1895    13447    +11552     
  Branches     1895    13447    +11552     
===========================================
+ Hits          760    10417     +9657     
- Misses       1100     2574     +1474     
- Partials       35      456      +421     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_l1_bounds_-_all_bounds_or_parametrize_in_account_transaction_test branch from b04682d to eb8ef23 Compare October 10, 2024 15:11
@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_update_test_insufficient_max_fee_reverts_to_include_new_resource_bounds branch from a400602 to e569b59 Compare October 10, 2024 15:11
@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_l1_bounds_-_all_bounds_or_parametrize_in_account_transaction_test branch from eb8ef23 to 0173dfa Compare October 13, 2024 08:13
@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_update_test_insufficient_max_fee_reverts_to_include_new_resource_bounds branch from e569b59 to 26404f4 Compare October 13, 2024 08:13
@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_l1_bounds_-_all_bounds_or_parametrize_in_account_transaction_test branch from 0173dfa to 43ad094 Compare October 13, 2024 08:16
@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_update_test_insufficient_max_fee_reverts_to_include_new_resource_bounds branch 2 times, most recently from 1e42bb3 to b0bcf2b Compare October 13, 2024 08:36
@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_l1_bounds_-_all_bounds_or_parametrize_in_account_transaction_test branch from 43ad094 to 1e7005b Compare October 13, 2024 12:38
@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_update_test_insufficient_max_fee_reverts_to_include_new_resource_bounds branch from b0bcf2b to 241f6da Compare October 13, 2024 12:39
@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_l1_bounds_-_all_bounds_or_parametrize_in_account_transaction_test branch from 1e7005b to eacda12 Compare October 13, 2024 13:04
@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_update_test_insufficient_max_fee_reverts_to_include_new_resource_bounds branch from 241f6da to 1b6e002 Compare October 13, 2024 13:04
@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_l1_bounds_-_all_bounds_or_parametrize_in_account_transaction_test branch from eacda12 to fd85e54 Compare October 14, 2024 08:06
@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_update_test_insufficient_max_fee_reverts_to_include_new_resource_bounds branch from 1b6e002 to 7dc156d Compare October 14, 2024 08:07
Copy link

Benchmark movements:
tree_computation_flow performance improved 😺
tree_computation_flow time: [34.122 ms 34.180 ms 34.250 ms]
change: [-3.9562% -2.4333% -1.0950%] (p = 0.00 < 0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) high mild
3 (3.00%) high severe

@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_update_test_insufficient_max_fee_reverts_to_include_new_resource_bounds branch from e941695 to a09dcf2 Compare October 22, 2024 11:57
Copy link

Benchmark movements:
tree_computation_flow performance improved 😺
tree_computation_flow time: [33.871 ms 33.905 ms 33.945 ms]
change: [-4.2749% -2.6448% -1.1876%] (p = 0.00 < 0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
4 (4.00%) high mild
5 (5.00%) high severe

Copy link
Contributor

@yoavGrs yoavGrs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 files reviewed, 6 unresolved discussions (waiting on @dorimedini-starkware)


crates/blockifier/src/transaction/account_transactions_test.rs line 1167 at r2 (raw file):

) {
    let gas_mode = resource_bounds.get_gas_vector_computation_mode();
    let overdraft_resource = match gas_mode {

Move it next to its use.
It's the overdraft resource only for the second tx.

Code quote:

let overdraft_resource

crates/blockifier/src/transaction/account_transactions_test.rs line 1206 at r2 (raw file):

                }
                GasVectorComputationMode::All => {
                    // We do not expect the exact same fee in the All case.

Despite the expectation, in practice, we do get equality. Why?

Code quote:

 We do not expect the exact same fee in the All case.

crates/blockifier/src/transaction/account_transactions_test.rs line 1227 at r2 (raw file):

    // This call should fail due to insufficient max fee (steps bound based on max_fee is not so
    // tight as to stop execution between iterations 1 and 2).
    let resource_bounds_depth1 = match gas_mode {

It's not the bounds in tx_execution_info1

Suggestion:

resource_used_depth1

crates/blockifier/src/transaction/account_transactions_test.rs line 1248 at r2 (raw file):

    )
    .unwrap();
    // In the deprecated bounds case, due to resource limit being estimated by steps, the execution

Is that the case?

Suggestion:

In the l1 gas bounds case

crates/blockifier/src/transaction/account_transactions_test.rs line 1267 at r2 (raw file):

            resource_bounds: resource_bounds_depth1,
            nonce: nonce_manager.next(account_address),
            calldata: recursive_function_calldata(&contract_address, 824, false),

Is it tight?
Is it going to change frequently?

Code quote:

824

crates/blockifier/src/transaction/account_transactions_test.rs line 1273 at r2 (raw file):

    .unwrap();
    // In the deprecated bounds case, the entire step limit is derived from 100% of all the
    // bounds (only L1 gas), so expected revert fee is identical to the original fee.

This comment should appear above the NoL2Gas case in the macro.

Code quote:

    // In the deprecated bounds case, the entire step limit is derived from 100% of all the
    // bounds (only L1 gas), so expected revert fee is identical to the original fee.

Copy link
Collaborator Author

@dorimedini-starkware dorimedini-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 files reviewed, 6 unresolved discussions (waiting on @yoavGrs)


crates/blockifier/src/transaction/account_transactions_test.rs line 1167 at r2 (raw file):

Previously, yoavGrs wrote…

Move it next to its use.
It's the overdraft resource only for the second tx.

Done.


crates/blockifier/src/transaction/account_transactions_test.rs line 1206 at r2 (raw file):

Previously, yoavGrs wrote…

Despite the expectation, in practice, we do get equality. Why?

WDYM in practice we get the same? equality isn't asserted here


crates/blockifier/src/transaction/account_transactions_test.rs line 1227 at r2 (raw file):

Previously, yoavGrs wrote…

It's not the bounds in tx_execution_info1

Done.


crates/blockifier/src/transaction/account_transactions_test.rs line 1248 at r2 (raw file):

Previously, yoavGrs wrote…

Is that the case?

Done.


crates/blockifier/src/transaction/account_transactions_test.rs line 1267 at r2 (raw file):

Previously, yoavGrs wrote…

Is it tight?
Is it going to change frequently?

this test is old, and it didn't change now, so I guess it's not very sensitive


crates/blockifier/src/transaction/account_transactions_test.rs line 1273 at r2 (raw file):

Previously, yoavGrs wrote…

This comment should appear above the NoL2Gas case in the macro.

Done.

@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_l1_bounds_-_all_bounds_or_parametrize_in_account_transaction_test branch from f056a9d to fbbc727 Compare October 28, 2024 14:48
@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_update_test_insufficient_max_fee_reverts_to_include_new_resource_bounds branch from a09dcf2 to c9ce605 Compare October 28, 2024 14:48
Copy link

Artifacts upload triggered. View details here

Copy link
Contributor

@yoavGrs yoavGrs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 2 files at r2, all commit messages.
Reviewable status: 1 of 2 files reviewed, 1 unresolved discussion (waiting on @dorimedini-starkware)


crates/blockifier/src/transaction/account_transactions_test.rs line 1206 at r2 (raw file):

Previously, dorimedini-starkware wrote…

WDYM in practice we get the same? equality isn't asserted here

In the code, you don't assert inequality.
But the explanations point out that execution_info.receipt.gas.l2_gas < tx_execution_info1.receipt.gas.l2_gas, but it's not the case: when I ran the test I got equality.

@dorimedini-starkware dorimedini-starkware changed the base branch from 10-10-test_blockifier_l1_bounds_-_all_bounds_or_parametrize_in_account_transaction_test to graphite-base/1313 October 29, 2024 10:42
Copy link

Artifacts upload triggered. View details here

@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_update_test_insufficient_max_fee_reverts_to_include_new_resource_bounds branch from c9ce605 to 52ba961 Compare October 29, 2024 10:53
Copy link

Artifacts upload triggered. View details here

@dorimedini-starkware dorimedini-starkware changed the base branch from graphite-base/1313 to main October 29, 2024 10:54
@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_update_test_insufficient_max_fee_reverts_to_include_new_resource_bounds branch from 52ba961 to 0b8953a Compare October 29, 2024 10:54
Copy link

Artifacts upload triggered. View details here

Copy link

Artifacts upload triggered. View details here

Copy link

Artifacts upload triggered. View details here

Copy link

Benchmark movements:
tree_computation_flow performance regressed!
tree_computation_flow time: [34.757 ms 35.215 ms 35.762 ms]
change: [+1.3113% +2.6924% +4.2146%] (p = 0.00 < 0.05)
Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
2 (2.00%) high mild
8 (8.00%) high severe

@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_update_test_insufficient_max_fee_reverts_to_include_new_resource_bounds branch from 0b8953a to 6de4946 Compare October 29, 2024 15:30
Copy link
Collaborator Author

@dorimedini-starkware dorimedini-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 1 of 2 files reviewed, 1 unresolved discussion (waiting on @yoavGrs)


crates/blockifier/src/transaction/account_transactions_test.rs line 1206 at r2 (raw file):

Previously, yoavGrs wrote…

In the code, you don't assert inequality.
But the explanations point out that execution_info.receipt.gas.l2_gas < tx_execution_info1.receipt.gas.l2_gas, but it's not the case: when I ran the test I got equality.

Done. You're right, the final fee should be the same (snap to bounds). The final gas vector however need not be the same on revert, so I simplified this a bit (no need for the macro anymore)

Copy link

Artifacts upload triggered. View details here

Copy link

Benchmark movements:
tree_computation_flow performance regressed!
tree_computation_flow time: [35.271 ms 35.716 ms 36.255 ms]
change: [+2.4527% +3.6453% +5.2050%] (p = 0.00 < 0.05)
Performance has regressed.
Found 16 outliers among 100 measurements (16.00%)
7 (7.00%) low mild
1 (1.00%) high mild
8 (8.00%) high severe

Copy link
Contributor

@yoavGrs yoavGrs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r4, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @dorimedini-starkware)

@dorimedini-starkware dorimedini-starkware force-pushed the 10-10-test_blockifier_update_test_insufficient_max_fee_reverts_to_include_new_resource_bounds branch from 6de4946 to f0f7db2 Compare October 30, 2024 11:37
Copy link

Artifacts upload triggered. View details here

Copy link
Collaborator Author

@dorimedini-starkware dorimedini-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r5, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @dorimedini-starkware)

@dorimedini-starkware dorimedini-starkware merged commit 512e8ef into main Oct 30, 2024
15 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Nov 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants