wip: add uncleaned roots when we shrink away non-zero lamport accounts #2865
Conversation
|
If this PR represents a change to the public RPC API:
Thank you for keeping the RPC clients in sync with the server API @jeffwashington. |
|
The Firedancer team maintains a line-for-line reimplementation of the |
jeffwashington
force-pushed
the
test_pack_normal_log_fixes
branch
from
afa3c96
to
a483d91
Compare
|
I think this shows the metrics from running this on mnb: |
jeffwashington
force-pushed
the
test_pack_normal_log_fixes
branch
from
a483d91
to
a7fc984
Compare
|
|
||
| /// offsets to accounts that are zero lamport single ref stored in this storage. | ||
| /// These are still alive. But, shrink will be able to remove them. | ||
| zero_lamport_single_ref_offsets: RwLock<IntSet<usize>>, |
There was a problem hiding this comment.
Do we need to keep all the offsets here?
Is the count of the number of single-ref zeros in the storage enough?
| @@ -4353,6 +4403,37 @@ impl AccountsDb { | |||
| ); | |||
| } | |||
|
|
|||
| pub(crate) fn zero_lamport_single_ref_found(&self, slot: Slot, offset: usize) { | |||
| if let Some(store) = self.storage.get_slot_storage_entry(slot) { | |||
| let mut zero_lamport_single_ref_offsets = | |||
There was a problem hiding this comment.
Can we move this logic one level up, i.e. after 'index.scan(...)`?
In index.scan(...), we keep a counter of how many single_ref_zeros.
After the index.scan(), we do the check below.
In this way, we don't need zero_lamport_single_ref_offsets: RwLock<IntSet> on account_storage at all?
Can we just keep the count, zero_lamport_single_ref_counts on account_storage instead, for the later usage in do_shrink_store()?
| .adding_dead_slots_to_clean | ||
| .fetch_add(1, Ordering::Relaxed); | ||
| } else { | ||
| if Self::is_shrinking_productive(slot, &store) |
There was a problem hiding this comment.
nit: we can check if slot is already in self.shrink_candidate_slot here.
if it is already in, we can skip this if shrinking_productive check () ....
| // should we also check for ref counts here? | ||
| if let Some((slot_alive, acct_info)) = slot_list.first() { | ||
| if acct_info.is_zero_lamport() && !acct_info.is_cached() { | ||
| self.zero_lamport_single_ref_found( |
There was a problem hiding this comment.
this hsould probably ALSO run when we are doing clean and we find a zero lamport account that has a single ref. This is a reason why the hashset may be necessary so we don't double-count it.
Problem
monitor # zero lamport single ref accounts for better cleaning
Summary of Changes
Fixes #