Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Add validator-manager #3502

Closed
wants to merge 151 commits into from
Closed
Show file tree
Hide file tree
Changes from 133 commits
Commits
Show all changes
151 commits
Select commit Hold shift + click to select a range
9503e33
Add JSON deposit data to `create`
paulhauner Aug 15, 2022
5af83be
Add deposit data to recover function
paulhauner Aug 15, 2022
e3b2020
Revert "Add deposit data to recover function"
paulhauner Aug 15, 2022
ec33951
Add incomplete test script
paulhauner Aug 15, 2022
66e24e2
Add changes to test.py
paulhauner Aug 16, 2022
1984c6b
Add initial progress
paulhauner Aug 16, 2022
ff3a025
Add progress
paulhauner Aug 16, 2022
e25526e
Add progress
paulhauner Aug 16, 2022
e463518
Start refactoring into separate commands
paulhauner Aug 17, 2022
fec2969
Remove old command
paulhauner Aug 17, 2022
85d0843
Add function to check against beacon node
paulhauner Aug 17, 2022
ce7f29d
Appease clippy
paulhauner Aug 17, 2022
709598f
Link validator_manager into Lighthouse binary
paulhauner Aug 17, 2022
0cc72bc
Fix mistake in command name
paulhauner Aug 17, 2022
b025185
Avoid pulling env into blocking context
paulhauner Aug 17, 2022
3b324fc
Tidy logging
paulhauner Aug 17, 2022
7b62064
Fix deposit log creation
paulhauner Aug 17, 2022
c27d3dd
Refactor to use `Config` struct
paulhauner Aug 17, 2022
61e0347
Add testing for create command
paulhauner Aug 17, 2022
5edcfb7
Start working on test vectors
paulhauner Aug 17, 2022
79b5c6f
Add working test vector generator
paulhauner Aug 18, 2022
3bab10b
Pass test vectors
paulhauner Aug 18, 2022
ed0f354
Remove keystore json
paulhauner Aug 18, 2022
516e4ee
Add test vectors
paulhauner Aug 18, 2022
2873cb8
Tidy directory structure
paulhauner Aug 18, 2022
5e31754
Expose VC `ApiTester` via `test_utils`
paulhauner Aug 18, 2022
2677443
Fix visibility
paulhauner Aug 18, 2022
391ece9
Migrate VC tests to tokio::test
paulhauner Aug 21, 2022
fa6eba9
Start adding import validator testing
paulhauner Aug 21, 2022
7ed821b
Progress with testing
paulhauner Aug 22, 2022
52e50f5
Progress with testing
paulhauner Aug 22, 2022
3bdddb3
Fix warning
paulhauner Aug 22, 2022
1d4e7ba
Add DumpConfigs
paulhauner Aug 22, 2022
4886e27
Start adding CLI tests
paulhauner Aug 22, 2022
5b86fce
Add tests for validator create
paulhauner Aug 23, 2022
630cb91
Add import tests
paulhauner Aug 23, 2022
bcbafc0
Start adding validator move
paulhauner Aug 23, 2022
c0e9087
Add export endpoint to LH
paulhauner Aug 23, 2022
71409e4
Progress with move command
paulhauner Aug 23, 2022
e98a015
Progress with `move` implementation
paulhauner Aug 24, 2022
adc87f7
Add basic `move` testing
paulhauner Aug 24, 2022
92edb1b
Progress more with testing
paulhauner Aug 24, 2022
d20e65a
Progress with testing
paulhauner Aug 24, 2022
43b16b1
Fix clippy lints
paulhauner Aug 24, 2022
bd00a79
Remove commented-out code
paulhauner Aug 24, 2022
0f5092e
Fix clippy lint
paulhauner Aug 24, 2022
06fa52e
Improve tests
paulhauner Aug 24, 2022
65e0fe2
Fix failing tests
paulhauner Aug 24, 2022
bdffff1
Remove abandoned files
paulhauner Aug 24, 2022
1b40dd7
Revert changes to fork choice tests
paulhauner Aug 24, 2022
328a450
Revert abandoned changes
paulhauner Aug 24, 2022
40b1619
Don't run validator-manager tests in debug
paulhauner Aug 24, 2022
05bafee
Retry rather than saving file to disk
paulhauner Aug 24, 2022
0c32b5a
Remove intermediate file from `move`
paulhauner Aug 25, 2022
64decbc
Handle status from import
paulhauner Aug 25, 2022
0c786c0
Add tests for duplicates
paulhauner Aug 25, 2022
7f89f1e
Add `move` tests, "builder-proposals" takes bool
paulhauner Aug 25, 2022
e5476cd
Fix tests for windows
paulhauner Aug 25, 2022
89b2334
Use common version of regex across workspace
paulhauner Aug 25, 2022
e8da281
Revert "Use common version of regex across workspace"
paulhauner Aug 25, 2022
1f8f125
Add keystore export flag to VC
paulhauner Aug 25, 2022
be84637
Add count to `move`, fix tests
paulhauner Aug 25, 2022
2abde0b
Allow VC to create password files via HTTP API
paulhauner Aug 29, 2022
47eaa50
Shorten "validator-client" to "vc"
paulhauner Aug 29, 2022
d1aff84
Handle a missing command
paulhauner Aug 29, 2022
fdd0311
Strip newlines from VC token
paulhauner Aug 29, 2022
b2887f8
Ensure password dir exists in val dir builder
paulhauner Aug 29, 2022
4ccb7cb
Exit early on keystore upload error
paulhauner Aug 29, 2022
0719680
Remove naughty return Err
paulhauner Aug 29, 2022
da2a8ba
Start adding docs
paulhauner Aug 29, 2022
9539ca2
Add "done" message
paulhauner Aug 29, 2022
4d79002
Add dodgy progress to docs
paulhauner Aug 29, 2022
6af351a
Update docs
paulhauner Aug 30, 2022
9dfea42
Add more to docs
paulhauner Aug 30, 2022
2abb4a7
Add password migrate command
paulhauner Sep 5, 2022
c60b9c6
Revert "Add password migrate command"
paulhauner Sep 5, 2022
6569a5c
Remove comments from deposit `amount`
paulhauner Sep 5, 2022
dbe2fa4
Add comment about `amount` field
paulhauner Sep 5, 2022
958d1e4
Merge branch 'unstable' into validator-manager
paulhauner Oct 31, 2022
95c997b
Merge branch 'unstable' into validator-manager
paulhauner Oct 31, 2022
fb49408
Merge branch 'unstable' into validator-manager
paulhauner Nov 14, 2022
c63fece
Appease clippy
paulhauner Nov 14, 2022
94f97da
Merge branch 'unstable' into validator-manager
paulhauner Nov 30, 2022
53e492e
Fix Cargo.lock
paulhauner Nov 30, 2022
06f7efc
Fix clippy error
paulhauner Nov 30, 2022
0ece17e
Remove `validators` subcommand
paulhauner Nov 30, 2022
b5c463e
Fix cli tests
paulhauner Nov 30, 2022
af6affa
Fix compile errors
paulhauner Nov 30, 2022
0fe8958
Add test for moving validators back again
paulhauner Dec 1, 2022
86c3bcd
Add misc changes
paulhauner Dec 1, 2022
bbcf7b8
Revert "Add misc changes"
paulhauner Dec 5, 2022
6bef2be
Allow export without password
paulhauner Dec 14, 2022
0abb6a3
Read password from user in move command
paulhauner Dec 14, 2022
afed3fb
Add test for supplying password
paulhauner Dec 14, 2022
d096a07
Add more password tests
paulhauner Dec 14, 2022
aeb70ca
Merge branch 'unstable' into validator-manager
paulhauner Dec 14, 2022
8ca7f32
Add stdin-inputs flag to move command
paulhauner Dec 14, 2022
d278bab
Merge branch 'unstable' into validator-manager
paulhauner Feb 14, 2023
fb72975
Merge branch 'unstable' into validator-manager
paulhauner Feb 23, 2023
aafc587
Merge branch 'unstable' into validator-manager
paulhauner Mar 13, 2023
f27908a
Update docs as per removal of `validators`
paulhauner Mar 13, 2023
0de05e9
Merge branch 'unstable' into validator-manager
paulhauner Mar 24, 2023
9c76d45
Clarify help message
paulhauner Mar 27, 2023
1b225e9
Tidy message
paulhauner Mar 27, 2023
707d2a2
Further improve help message
paulhauner Mar 27, 2023
ee430c0
Merge branch 'unstable' into validator-manager
paulhauner May 15, 2023
fb51b09
Merge branch 'stable' into validator-manager
paulhauner Jul 10, 2023
ad65cf6
Fix clippy lints
paulhauner Jul 10, 2023
8d3aa44
Use async tests
paulhauner Jul 10, 2023
4af863a
Fix warnings
paulhauner Jul 10, 2023
fbb844d
Add `--count` flag as per @michaelsproul's comment
paulhauner Jul 11, 2023
66fd861
Update `KeyCache` after removing validator
paulhauner Jul 11, 2023
4c4f227
Check key cache consistency in tests
paulhauner Jul 11, 2023
c7ee1ea
Fix clippy lints
paulhauner Jul 11, 2023
98bcb94
Reduce loglevel for `UnknownPubkey` in the VC
paulhauner Jul 12, 2023
038c7a2
Update help text for move command
paulhauner Jul 12, 2023
856cd7e
Delete password files unless its being used elsewhere
paulhauner Jul 12, 2023
1ec051c
Update docs
paulhauner Jul 13, 2023
31dc520
Avoid using BLS credentials
paulhauner Jul 13, 2023
c3fe9d7
Add more docs
paulhauner Jul 13, 2023
b11ce55
Update error for missing fee recip
paulhauner Jul 13, 2023
e2a2e89
Add suggested fee recip
paulhauner Jul 13, 2023
2341f03
Update docs more
paulhauner Jul 13, 2023
b5027a0
Add docs to summary
paulhauner Jul 13, 2023
392624b
Tidy docs
paulhauner Jul 13, 2023
aa54777
Add new pages to summary
paulhauner Jul 13, 2023
6c9bea8
Fix failing test
paulhauner Jul 13, 2023
95de58e
De-duplicate some code
paulhauner Jul 16, 2023
745f9af
Fix comments
paulhauner Jul 16, 2023
2900b6c
Merge branch 'unstable' into validator-manager
paulhauner Jul 17, 2023
3d8d3de
Update Cargo.lock
paulhauner Jul 17, 2023
7b4b96d
Update validator_manager/src/move_validators.rs
paulhauner Jul 17, 2023
09bf5ab
Fix format string
paulhauner Jul 17, 2023
5c4b837
Update validator_manager/src/move_validators.rs
paulhauner Jul 18, 2023
d749888
Apply suggestions from code review
paulhauner Jul 18, 2023
5bc0f51
Add comment about omitting --beacon-node
paulhauner Jul 18, 2023
9ff1679
Remove references to dupe flag in move
paulhauner Jul 18, 2023
1766dbd
Apply suggestions from code review
paulhauner Jul 18, 2023
db88cd8
Fix clippy error
paulhauner Jul 19, 2023
fac1df7
Apply suggestions from code review
paulhauner Aug 3, 2023
c8317f2
Apply suggestions from code review
paulhauner Aug 3, 2023
a803529
Merge branch 'unstable' into validator-manager
paulhauner Aug 3, 2023
37c7003
Apply suggestions from code review
paulhauner Aug 3, 2023
1689ed6
Remove unencrypted flags
paulhauner Aug 7, 2023
3422225
Fix log message with swapped number
paulhauner Aug 7, 2023
ec9cf1c
Apply suggestions from code review
paulhauner Aug 7, 2023
e86c0e6
Apply suggestions from code review
paulhauner Aug 7, 2023
c07a554
Fix --validators help text
paulhauner Aug 7, 2023
88db492
Update "no validators present" log
paulhauner Aug 7, 2023
5654966
Add deprecation notice to "Key Management"
paulhauner Aug 7, 2023
a8865f9
Update book/src/key-management.md
paulhauner Aug 7, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 56 additions & 26 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,8 @@ members = [
"validator_client",
"validator_client/slashing_protection",

"validator_manager",

"watch",
]
resolver = "2"
Expand Down
2 changes: 2 additions & 0 deletions account_manager/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ safe_arith = {path = "../consensus/safe_arith"}
slot_clock = { path = "../common/slot_clock" }
filesystem = { path = "../common/filesystem" }
sensitive_url = { path = "../common/sensitive_url" }
serde = { version = "1.0.116", features = ["derive"] }
serde_json = "1.0.58"

[dev-dependencies]
tempfile = "3.1.0"
50 changes: 1 addition & 49 deletions account_manager/src/common.rs
Original file line number Diff line number Diff line change
@@ -1,55 +1,7 @@
use account_utils::PlainText;
use account_utils::{read_input_from_user, strip_off_newlines};
use eth2_wallet::bip39::{Language, Mnemonic};
use std::fs;
use std::path::PathBuf;
use std::str::from_utf8;
use std::thread::sleep;
use std::time::Duration;
use account_utils::read_input_from_user;

pub const MNEMONIC_PROMPT: &str = "Enter the mnemonic phrase:";
pub const WALLET_NAME_PROMPT: &str = "Enter wallet name:";

pub fn read_mnemonic_from_cli(
mnemonic_path: Option<PathBuf>,
stdin_inputs: bool,
) -> Result<Mnemonic, String> {
let mnemonic = match mnemonic_path {
Some(path) => fs::read(&path)
.map_err(|e| format!("Unable to read {:?}: {:?}", path, e))
.and_then(|bytes| {
let bytes_no_newlines: PlainText = strip_off_newlines(bytes).into();
let phrase = from_utf8(bytes_no_newlines.as_ref())
.map_err(|e| format!("Unable to derive mnemonic: {:?}", e))?;
Mnemonic::from_phrase(phrase, Language::English).map_err(|e| {
format!(
"Unable to derive mnemonic from string {:?}: {:?}",
phrase, e
)
})
})?,
None => loop {
eprintln!();
eprintln!("{}", MNEMONIC_PROMPT);

let mnemonic = read_input_from_user(stdin_inputs)?;

match Mnemonic::from_phrase(mnemonic.as_str(), Language::English) {
Ok(mnemonic_m) => {
eprintln!("Valid mnemonic provided.");
eprintln!();
sleep(Duration::from_secs(1));
break mnemonic_m;
}
Err(_) => {
eprintln!("Invalid mnemonic");
}
}
},
};
Ok(mnemonic)
}

/// Reads in a wallet name from the user. If the `--wallet-name` flag is provided, use it. Otherwise
/// read from an interactive prompt using tty unless the `--stdin-inputs` flag is provided.
pub fn read_wallet_name_from_cli(
Expand Down
8 changes: 5 additions & 3 deletions account_manager/src/validator/import.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ use account_utils::{
eth2_keystore::Keystore,
read_password_from_user,
validator_definitions::{
recursively_find_voting_keystores, ValidatorDefinition, ValidatorDefinitions,
CONFIG_FILENAME,
recursively_find_voting_keystores, PasswordStorage, ValidatorDefinition,
ValidatorDefinitions, CONFIG_FILENAME,
},
ZeroizeString,
};
Expand Down Expand Up @@ -277,7 +277,9 @@ pub fn cli_run(matches: &ArgMatches, validator_dir: PathBuf) -> Result<(), Strin
let suggested_fee_recipient = None;
let validator_def = ValidatorDefinition::new_keystore_with_password(
&dest_keystore,
password_opt,
password_opt
.map(PasswordStorage::ValidatorDefinitions)
.unwrap_or(PasswordStorage::None),
graffiti,
suggested_fee_recipient,
None,
Expand Down
3 changes: 1 addition & 2 deletions account_manager/src/validator/recover.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
use super::create::STORE_WITHDRAW_FLAG;
use crate::common::read_mnemonic_from_cli;
use crate::validator::create::COUNT_FLAG;
use crate::wallet::create::STDIN_INPUTS_FLAG;
use crate::SECRETS_DIR_FLAG;
use account_utils::eth2_keystore::{keypair_from_secret, Keystore, KeystoreBuilder};
use account_utils::random_password;
use account_utils::{random_password, read_mnemonic_from_cli};
use clap::{App, Arg, ArgMatches};
use directory::ensure_dir_exists;
use directory::{parse_path_or_default_with_flag, DEFAULT_SECRET_DIR};
Expand Down
2 changes: 1 addition & 1 deletion account_manager/src/wallet/recover.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use crate::common::read_mnemonic_from_cli;
use crate::wallet::create::{create_wallet_from_mnemonic, STDIN_INPUTS_FLAG};
use crate::wallet::create::{HD_TYPE, NAME_FLAG, PASSWORD_FLAG, TYPE_FLAG};
use account_utils::read_mnemonic_from_cli;
use clap::{App, Arg, ArgMatches};
use std::path::PathBuf;

Expand Down
3 changes: 3 additions & 0 deletions book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@
* [Run a Node](./run_a_node.md)
* [Become a Validator](./mainnet-validator.md)
* [Validator Management](./validator-management.md)
* [The `validator-manager` Command](./validator-manager.md)
* [Creating validators](./validator-manager-create.md)
* [Moving validators](./validator-manager-move.md)
* [Slashing Protection](./slashing-protection.md)
* [Voluntary Exits](./voluntary-exit.md)
* [Partial Withdrawals](./partial-withdrawal.md)
Expand Down
4 changes: 4 additions & 0 deletions book/src/validator-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ standard directories and do not start their `lighthouse vc` with the
this document. However, users with more complex needs may find this document
useful.

The [lighthouse validator-manager](./validator-manager.md) command can be used
to create and import validators to a Lighthouse VC. It can also be used to move
validators between two Lighthouse VCs.

## Introducing the `validator_definitions.yml` file

The `validator_definitions.yml` file is located in the `validator-dir`, which
Expand Down
Loading
Loading