Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(starknet_batcher): save BlockExecutionArtifacts in ProposalManager #2725

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

DvirYo-starkware
Copy link
Contributor

No description provided.

@reviewable-StarkWare
Copy link

This change is Reviewable

@DvirYo-starkware DvirYo-starkware force-pushed the dvir/save_exec_info_in_batcher branch from 5f9e18f to e0adeb1 Compare December 17, 2024 12:35
Copy link
Contributor Author

@DvirYo-starkware DvirYo-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 9 files reviewed, 1 unresolved discussion (waiting on @alonh5, @dafnamatsry, @noaov1, @Yael-Starkware, and @yair-starkware)


a discussion (no related file):
I assume that was some discussion about all the optional cloning. If important it can be prevented in two ways:

  • add another flag in addition to the testing flag and import blockier in the batcher with this flag
  • save in the ProposalManager only the BlockExecutionArtifacts and only when writing the block to the storage make a conversion. This will increase the latency of the transaction to accept on layer 2 in version14.0

@DvirYo-starkware DvirYo-starkware changed the title feat(blockifier): update gas and vm resources computations (#2153) feat(starknet_batcher): update gas and vm resources computations (#2153) Dec 17, 2024
@DvirYo-starkware DvirYo-starkware force-pushed the dvir/save_exec_info_in_batcher branch from e0adeb1 to a74f90c Compare December 17, 2024 12:41
@DvirYo-starkware DvirYo-starkware changed the title feat(starknet_batcher): update gas and vm resources computations (#2153) feat(starknet_batcher): save BlockExecutionArtifacts in ProposalManager Dec 17, 2024
Copy link
Contributor

@yair-starkware yair-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 9 of 9 files at r1, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @alonh5, @dafnamatsry, @noaov1, and @Yael-Starkware)

Copy link
Collaborator

@alonh5 alonh5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @dafnamatsry, @DvirYo-starkware, @noaov1, and @Yael-Starkware)


crates/starknet_batcher/src/batcher.rs line 372 at r3 (raw file):

    }

    // TODO(dvir): return `BlockExecutionArtifacts`

Why?

Code quote:

// TODO(dvir): return `BlockExecutionArtifacts`

Copy link
Contributor Author

@DvirYo-starkware DvirYo-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @alonh5, @dafnamatsry, @noaov1, and @Yael-Starkware)


crates/starknet_batcher/src/batcher.rs line 372 at r3 (raw file):

Previously, alonh5 (Alon Haramati) wrote…

Why?

This information is needed for writing to Aerospike

Copy link
Collaborator

@alonh5 alonh5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @dafnamatsry, @DvirYo-starkware, @noaov1, and @Yael-Starkware)


crates/starknet_batcher/src/utils.rs line 69 at r3 (raw file):

            tx_hashes,
            nonces,
            block_execution_artifacts: cloned_artifacts,

In this case, consider deleting the ProposalOutput type and storing only the BlockExecutionArtifacts. Fields used from ProposalOutput can be calculated when needed (they are used only once, IIRC).

Copy link
Contributor Author

@DvirYo-starkware DvirYo-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @alonh5, @dafnamatsry, @noaov1, and @Yael-Starkware)


crates/starknet_batcher/src/utils.rs line 69 at r3 (raw file):

Previously, alonh5 (Alon Haramati) wrote…

In this case, consider deleting the ProposalOutput type and storing only the BlockExecutionArtifacts. Fields used from ProposalOutput can be calculated when needed (they are used only once, IIRC).

See here:
https://reviewable.io/reviews/starkware-libs/sequencer/2725#:~:text=save%20in%20the%20ProposalManager%20only%20the%20BlockExecutionArtifacts%20and%20only%20when%20writing%20the%20block%20to%20the%20storage%20make%20a%20conversion.%20This%20will%20increase%20the%20latency%20of%20the%20transaction%20to%20accept%20on%20layer%202%20in%20version14.0

In addition, it requires more significant changes in the batcher for something that will be deleted after version 14.0, and I try to avoid it.

@DvirYo-starkware DvirYo-starkware force-pushed the dvir/save_exec_info_in_batcher branch from a74f90c to 8b9631b Compare December 18, 2024 13:47
Copy link
Collaborator

@alonh5 alonh5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 7 of 10 files reviewed, 2 unresolved discussions (waiting on @dafnamatsry, @DvirYo-starkware, @noaov1, @Yael-Starkware, and @yair-starkware)


crates/starknet_batcher/src/utils.rs line 69 at r3 (raw file):

Previously, DvirYo-starkware wrote…

See here:
https://reviewable.io/reviews/starkware-libs/sequencer/2725#:~:text=save%20in%20the%20ProposalManager%20only%20the%20BlockExecutionArtifacts%20and%20only%20when%20writing%20the%20block%20to%20the%20storage%20make%20a%20conversion.%20This%20will%20increase%20the%20latency%20of%20the%20transaction%20to%20accept%20on%20layer%202%20in%20version14.0

In addition, it requires more significant changes in the batcher for something that will be deleted after version 14.0, and I try to avoid it.

On the other hand, we're now doing these calculations for every proposal even though we only need them for the one we decided on.
We recently did a refactor to the batcher (deleting the proposal manager), and I think the ProposalOutput is redundant anyway, this won't be deleted after 0.14.0. Could you try to do this and see if it makes sense? I think it would be better

Copy link

Benchmark movements:
tree_computation_flow performance improved 😺
tree_computation_flow time: [34.400 ms 34.428 ms 34.460 ms]
change: [-3.8063% -2.3479% -1.0764%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
2 (2.00%) high mild
5 (5.00%) high severe

Copy link
Contributor

@Yael-Starkware Yael-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 7 of 10 files reviewed, 2 unresolved discussions (waiting on @dafnamatsry, @DvirYo-starkware, @noaov1, and @yair-starkware)


a discussion (no related file):

Previously, DvirYo-starkware wrote…

I assume that was some discussion about all the optional cloning. If important it can be prevented in two ways:

  • add another flag in addition to the testing flag and import blockier in the batcher with this flag
  • save in the ProposalManager only the BlockExecutionArtifacts and only when writing the block to the storage make a conversion. This will increase the latency of the transaction to accept on layer 2 in version14.0

I don't think the consensus can start a new height before decision reached is done, since it will clear all prev height results.
if this is the case, it doesn't really matter if the conversion is done in the propose_block step or in the decision_reached step.


crates/starknet_batcher/src/batcher.rs line 372 at r3 (raw file):

Previously, DvirYo-starkware wrote…

This information is needed for writing to Aerospike

you can also return it in Get_proposal_content and Send_proposal_content

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants