Skip to content

Commit

Permalink
Merge branch 'main' into nelson-gas-limit-fix
Browse files Browse the repository at this point in the history
  • Loading branch information
non-fungible-nelson authored Jan 18, 2024
2 parents bd4992f + 2d7941f commit 8266c95
Show file tree
Hide file tree
Showing 20 changed files with 492 additions and 242 deletions.
6 changes: 4 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,20 @@
- New `EXECUTION_HALTED` error returned if there is an error executing or simulating a transaction, with the reason for execution being halted. Replaces the generic `INTERNAL_ERROR` return code in certain cases which some applications may be checking for [#6343](https://github.com/hyperledger/besu/pull/6343)
- The Besu Docker images with `openjdk-latest` tags since 23.10.3 were incorrectly using UID 1001 instead of 1000 for the container's `besu` user. The user now uses 1000 again. Containers created from or migrated to images using UID 1001 will need to chown their persistent database files to UID 1000 [#6360](https://github.com/hyperledger/besu/pull/6360)
- The deprecated `--privacy-onchain-groups-enabled` option has now been removed. Use the `--privacy-flexible-groups-enabled` option instead. [#6411](https://github.com/hyperledger/besu/pull/6411)
- The time that can be spent selecting transactions during block creation is not capped at 5 seconds for PoS and PoW networks, and for PoA networks, at 75% of the block period specified in the genesis, this to prevent possible DoS in case a single transaction is taking too long to execute, and to have a stable block production rate, but it could be a breaking change if an existing network used to have transactions that takes more time to executed that the newly introduced limit, if it is mandatory for these network to keep processing these long processing transaction, then the default value of `block-txs-selection-max-time` or `poa-block-txs-selection-max-time` needs to be tuned accordingly.

### Deprecations

### Additions and Improvements
- Optimize RocksDB WAL files, allows for faster restart and a more linear disk space utilization [#6328](https://github.com/hyperledger/besu/pull/6328)
- Disable transaction handling when the node is not in sync, to avoid unnecessary transaction validation work [#6302](https://github.com/hyperledger/besu/pull/6302)
- Introduce TransactionEvaluationContext to pass data between transaction selectors and plugin, during block creation [#6381](https://github.com/hyperledger/besu/pull/6381)
- Introduce TransactionEvaluationContext to pass data between transaction selectors and plugin, during block creation [#6381](https://github.com/hyperledger/besu/pull/6381)
- Upgrade dependencies [#6377](https://github.com/hyperledger/besu/pull/6377)
- Upgrade `com.fasterxml.jackson` dependencies [#6378](https://github.com/hyperledger/besu/pull/6378)
- Upgrade `com.fasterxml.jackson` dependencies [#6378](https://github.com/hyperledger/besu/pull/6378)
- Upgrade Guava dependency [#6396](https://github.com/hyperledger/besu/pull/6396)
- Upgrade Mockito [#6397](https://github.com/hyperledger/besu/pull/6397)
- Upgrade `tech.pegasys.discovery:discovery` [#6414](https://github.com/hyperledger/besu/pull/6414)
- Options to tune the max allowed time that can be spent selecting transactions during block creation are now stable [#6423](https://github.com/hyperledger/besu/pull/6423)

### Bug fixes
- INTERNAL_ERROR from `eth_estimateGas` JSON/RPC calls [#6344](https://github.com/hyperledger/besu/issues/6344)
Expand Down
8 changes: 3 additions & 5 deletions besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java
Original file line number Diff line number Diff line change
Expand Up @@ -2907,17 +2907,15 @@ private MiningParameters getMiningParameters() {
ImmutableMiningParameters.builder().from(miningOptions.toDomainObject());
final var actualGenesisOptions = getActualGenesisConfigOptions();
if (actualGenesisOptions.isPoa()) {
miningParametersBuilder.unstable(
ImmutableMiningParameters.Unstable.builder()
.minBlockTime(getMinBlockTime(actualGenesisOptions))
.build());
miningParametersBuilder.genesisBlockPeriodSeconds(
getGenesisBlockPeriodSeconds(actualGenesisOptions));
}
miningParameters = miningParametersBuilder.build();
}
return miningParameters;
}

private int getMinBlockTime(final GenesisConfigOptions genesisConfigOptions) {
private int getGenesisBlockPeriodSeconds(final GenesisConfigOptions genesisConfigOptions) {
if (genesisConfigOptions.isClique()) {
return genesisConfigOptions.getCliqueConfigOptions().getBlockPeriodSeconds();
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
/*
* Copyright Hyperledger Besu Contributors.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.cli.converter;

import org.hyperledger.besu.cli.converter.exception.PercentageConversionException;
import org.hyperledger.besu.util.number.PositiveNumber;

import picocli.CommandLine;

/** The PositiveNumber Cli type converter. */
public class PositiveNumberConverter implements CommandLine.ITypeConverter<PositiveNumber> {

@Override
public PositiveNumber convert(final String value) throws PercentageConversionException {
try {
return PositiveNumber.fromString(value);
} catch (NullPointerException | IllegalArgumentException e) {
throw new PercentageConversionException(value);
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
/*
* Copyright Hyperledger Besu Contributors.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.cli.converter.exception;

import static java.lang.String.format;

/** The custom PositiveNumber conversion exception. */
public final class PositiveNumberConversionException extends Exception {

/**
* Instantiates a new PositiveNumber conversion exception.
*
* @param value the invalid value to add in exception message
*/
public PositiveNumberConversionException(final String value) {
super(format("Invalid value: %s, should be a positive number >0.", value));
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -16,28 +16,28 @@

import static java.util.Arrays.asList;
import static java.util.Collections.singletonList;
import static org.hyperledger.besu.ethereum.core.MiningParameters.DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME;
import static org.hyperledger.besu.ethereum.core.MiningParameters.DEFAULT_POA_BLOCK_TXS_SELECTION_MAX_TIME;
import static org.hyperledger.besu.ethereum.core.MiningParameters.MutableInitValues.DEFAULT_EXTRA_DATA;
import static org.hyperledger.besu.ethereum.core.MiningParameters.MutableInitValues.DEFAULT_MIN_BLOCK_OCCUPANCY_RATIO;
import static org.hyperledger.besu.ethereum.core.MiningParameters.MutableInitValues.DEFAULT_MIN_PRIORITY_FEE_PER_GAS;
import static org.hyperledger.besu.ethereum.core.MiningParameters.MutableInitValues.DEFAULT_MIN_TRANSACTION_GAS_PRICE;
import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_MAX_OMMERS_DEPTH;
import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME;
import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_POA_BLOCK_TXS_SELECTION_MAX_TIME;
import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_POS_BLOCK_CREATION_MAX_TIME;
import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_POS_BLOCK_CREATION_REPETITION_MIN_DURATION;
import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_POW_JOB_TTL;
import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_REMOTE_SEALERS_LIMIT;
import static org.hyperledger.besu.ethereum.core.MiningParameters.Unstable.DEFAULT_REMOTE_SEALERS_TTL;

import org.hyperledger.besu.cli.converter.PercentageConverter;
import org.hyperledger.besu.cli.converter.PositiveNumberConverter;
import org.hyperledger.besu.cli.util.CommandLineUtils;
import org.hyperledger.besu.config.GenesisConfigOptions;
import org.hyperledger.besu.datatypes.Address;
import org.hyperledger.besu.datatypes.Wei;
import org.hyperledger.besu.ethereum.core.ImmutableMiningParameters;
import org.hyperledger.besu.ethereum.core.ImmutableMiningParameters.MutableInitValues;
import org.hyperledger.besu.ethereum.core.MiningParameters;
import org.hyperledger.besu.util.number.Percentage;
import org.hyperledger.besu.util.number.PositiveNumber;

import java.util.List;

Expand Down Expand Up @@ -115,6 +115,24 @@ public class MiningOptions implements CLIOptions<MiningParameters> {
+ " If set, each block's gas limit will approach this setting over time.")
private Long targetGasLimit = null;

@Option(
names = {"--block-txs-selection-max-time"},
converter = PositiveNumberConverter.class,
description =
"Specifies the maximum time, in milliseconds, that could be spent selecting transactions to be included in the block."
+ " Not compatible with PoA networks, see poa-block-txs-selection-max-time. (default: ${DEFAULT-VALUE})")
private PositiveNumber nonPoaBlockTxsSelectionMaxTime =
DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME;

@Option(
names = {"--poa-block-txs-selection-max-time"},
converter = PositiveNumberConverter.class,
description =
"Specifies the maximum time that could be spent selecting transactions to be included in the block, as a percentage of the fixed block time of the PoA network."
+ " To be only used on PoA networks, for other networks see block-txs-selection-max-time."
+ " (default: ${DEFAULT-VALUE})")
private PositiveNumber poaBlockTxsSelectionMaxTime = DEFAULT_POA_BLOCK_TXS_SELECTION_MAX_TIME;

@CommandLine.ArgGroup(validate = false)
private final Unstable unstableOptions = new Unstable();

Expand Down Expand Up @@ -168,25 +186,6 @@ static class Unstable {
+ " then it waits before next repetition. Must be positive and ≤ 2000 (default: ${DEFAULT-VALUE} milliseconds)")
private Long posBlockCreationRepetitionMinDuration =
DEFAULT_POS_BLOCK_CREATION_REPETITION_MIN_DURATION;

@CommandLine.Option(
hidden = true,
names = {"--Xblock-txs-selection-max-time"},
description =
"Specifies the maximum time, in milliseconds, that could be spent selecting transactions to be included in the block."
+ " Not compatible with PoA networks, see Xpoa-block-txs-selection-max-time."
+ " Must be positive and ≤ (default: ${DEFAULT-VALUE})")
private Long nonPoaBlockTxsSelectionMaxTime = DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME;

@CommandLine.Option(
hidden = true,
names = {"--Xpoa-block-txs-selection-max-time"},
converter = PercentageConverter.class,
description =
"Specifies the maximum time that could be spent selecting transactions to be included in the block, as a percentage of the fixed block time of the PoA network."
+ " To be only used on PoA networks, for other networks see Xblock-txs-selection-max-time."
+ " (default: ${DEFAULT-VALUE})")
private Percentage poaBlockTxsSelectionMaxTime = DEFAULT_POA_BLOCK_TXS_SELECTION_MAX_TIME;
}

private MiningOptions() {}
Expand Down Expand Up @@ -270,26 +269,17 @@ public void validate(
if (genesisConfigOptions.isPoa()) {
CommandLineUtils.failIfOptionDoesntMeetRequirement(
commandLine,
"--Xblock-txs-selection-max-time can't be used with PoA networks,"
+ " see Xpoa-block-txs-selection-max-time instead",
"--block-txs-selection-max-time can't be used with PoA networks,"
+ " see poa-block-txs-selection-max-time instead",
false,
singletonList("--Xblock-txs-selection-max-time"));
singletonList("--block-txs-selection-max-time"));
} else {
CommandLineUtils.failIfOptionDoesntMeetRequirement(
commandLine,
"--Xpoa-block-txs-selection-max-time can be only used with PoA networks,"
+ " see --Xblock-txs-selection-max-time instead",
"--poa-block-txs-selection-max-time can be only used with PoA networks,"
+ " see --block-txs-selection-max-time instead",
false,
singletonList("--Xpoa-block-txs-selection-max-time"));

if (unstableOptions.nonPoaBlockTxsSelectionMaxTime <= 0
|| unstableOptions.nonPoaBlockTxsSelectionMaxTime
> DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME) {
throw new ParameterException(
commandLine,
"--Xblock-txs-selection-max-time must be positive and ≤ "
+ DEFAULT_NON_POA_BLOCK_TXS_SELECTION_MAX_TIME);
}
singletonList("--poa-block-txs-selection-max-time"));
}
}

Expand All @@ -303,6 +293,10 @@ static MiningOptions fromConfig(final MiningParameters miningParameters) {
miningOptions.minTransactionGasPrice = miningParameters.getMinTransactionGasPrice();
miningOptions.minPriorityFeePerGas = miningParameters.getMinPriorityFeePerGas();
miningOptions.minBlockOccupancyRatio = miningParameters.getMinBlockOccupancyRatio();
miningOptions.nonPoaBlockTxsSelectionMaxTime =
miningParameters.getNonPoaBlockTxsSelectionMaxTime();
miningOptions.poaBlockTxsSelectionMaxTime = miningParameters.getPoaBlockTxsSelectionMaxTime();

miningOptions.unstableOptions.remoteSealersLimit =
miningParameters.getUnstable().getRemoteSealersLimit();
miningOptions.unstableOptions.remoteSealersTimeToLive =
Expand All @@ -317,10 +311,6 @@ static MiningOptions fromConfig(final MiningParameters miningParameters) {
miningParameters.getUnstable().getPosBlockCreationMaxTime();
miningOptions.unstableOptions.posBlockCreationRepetitionMinDuration =
miningParameters.getUnstable().getPosBlockCreationRepetitionMinDuration();
miningOptions.unstableOptions.nonPoaBlockTxsSelectionMaxTime =
miningParameters.getUnstable().getBlockTxsSelectionMaxTime();
miningOptions.unstableOptions.poaBlockTxsSelectionMaxTime =
miningParameters.getUnstable().getPoaBlockTxsSelectionMaxTime();

miningParameters.getCoinbase().ifPresent(coinbase -> miningOptions.coinbase = coinbase);
miningParameters.getTargetGasLimit().ifPresent(tgl -> miningOptions.targetGasLimit = tgl);
Expand Down Expand Up @@ -350,6 +340,8 @@ public MiningParameters toDomainObject() {
.isStratumMiningEnabled(iStratumMiningEnabled)
.stratumNetworkInterface(stratumNetworkInterface)
.stratumPort(stratumPort)
.nonPoaBlockTxsSelectionMaxTime(nonPoaBlockTxsSelectionMaxTime)
.poaBlockTxsSelectionMaxTime(poaBlockTxsSelectionMaxTime)
.unstable(
ImmutableMiningParameters.Unstable.builder()
.remoteSealersLimit(unstableOptions.remoteSealersLimit)
Expand All @@ -360,8 +352,6 @@ public MiningParameters toDomainObject() {
.posBlockCreationMaxTime(unstableOptions.posBlockCreationMaxTime)
.posBlockCreationRepetitionMinDuration(
unstableOptions.posBlockCreationRepetitionMinDuration)
.nonPoaBlockTxsSelectionMaxTime(unstableOptions.nonPoaBlockTxsSelectionMaxTime)
.poaBlockTxsSelectionMaxTime(unstableOptions.poaBlockTxsSelectionMaxTime)
.build());

return miningParametersBuilder.build();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -591,7 +591,9 @@ public BesuController build() {
prepForBuild();

final ProtocolSchedule protocolSchedule = createProtocolSchedule();
final GenesisState genesisState = GenesisState.fromConfig(genesisConfig, protocolSchedule);
final GenesisState genesisState =
GenesisState.fromConfig(
dataStorageConfiguration.getDataStorageFormat(), genesisConfig, protocolSchedule);

final VariablesStorage variablesStorage = storageProvider.createVariablesStorage();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@
import org.hyperledger.besu.plugin.services.rpc.PluginRpcRequest;
import org.hyperledger.besu.util.number.Fraction;
import org.hyperledger.besu.util.number.Percentage;
import org.hyperledger.besu.util.number.PositiveNumber;
import org.hyperledger.besu.util.platform.PlatformDetector;

import java.io.File;
Expand Down Expand Up @@ -846,6 +847,8 @@ public void tomlThatConfiguresEverythingExceptPermissioningToml() throws IOExcep
tomlResult.getDouble(tomlKey);
} else if (Percentage.class.isAssignableFrom(optionSpec.type())) {
tomlResult.getLong(tomlKey);
} else if (PositiveNumber.class.isAssignableFrom(optionSpec.type())) {
tomlResult.getLong(tomlKey);
} else {
tomlResult.getString(tomlKey);
}
Expand Down
Loading

0 comments on commit 8266c95

Please sign in to comment.