Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rkuris/streaming iterator from start #346

Merged
merged 34 commits into from
Nov 15, 2023
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
969684f
Streaming iterator spike
rkuris Nov 3, 2023
c806b7a
Format
rkuris Nov 4, 2023
9bb6cd3
WIP
rkuris Nov 6, 2023
46aecef
Complete implementation
rkuris Nov 8, 2023
116d83b
Formatting
rkuris Nov 8, 2023
0c31eb9
itertools is only needed at runtime
rkuris Nov 8, 2023
f5e69c6
No need to borrow K for get_node
rkuris Nov 8, 2023
e6e36d0
Use let-else-return instead of match arms
rkuris Nov 8, 2023
e25d42b
Remove unnecessary continue
rkuris Nov 8, 2023
8273434
Rename current_node to previously_returned_node
rkuris Nov 8, 2023
36ce8cc
Redo the loop to have an IteratorState
rkuris Nov 9, 2023
4d5f65c
Removed unnecessary reference in remove()
rkuris Nov 9, 2023
692018a
Fix typo
rkuris Nov 9, 2023
17f4713
Add some blank lines
rkuris Nov 9, 2023
7ab9003
Remove pin! macro
rkuris Nov 9, 2023
4867f82
In-person review comments
rkuris Nov 9, 2023
afe6357
Format
rkuris Nov 10, 2023
8b3648f
Streaming iterator from start
rkuris Nov 11, 2023
9803ae3
Use enumerate/find instead of position
rkuris Nov 14, 2023
2af1991
Break with the leaf rather than recomputing
rkuris Nov 14, 2023
936d0b5
Add doc comments to MerkleKeyValueStream
rkuris Nov 14, 2023
94a4244
Use let/else instead of if/let/else
rkuris Nov 14, 2023
dcfb338
Found_offset is now a separate statement
rkuris Nov 14, 2023
1e8193b
Fix doc comment problem
rkuris Nov 14, 2023
98f1b28
Allow position to be None while walking up
rkuris Nov 14, 2023
a862043
Use test_case to cover additional test cases
rkuris Nov 14, 2023
b952bc6
Remove unnecessary commented out code
rkuris Nov 14, 2023
b18d91f
Oops, have to increment :)
rkuris Nov 14, 2023
895507d
You can't just increment
rkuris Nov 14, 2023
33553e6
Add additional test cases
rkuris Nov 14, 2023
8dfd789
Lint
rkuris Nov 14, 2023
b64e98e
Remove some unwraps
rkuris Nov 14, 2023
55bc47a
Apply suggestions from code review
rkuris Nov 14, 2023
133f949
Make merkle::get_iter crate-only public
rkuris Nov 15, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 20 additions & 6 deletions firewood/src/db.rs
Original file line number Diff line number Diff line change
@@ -1,12 +1,6 @@
// Copyright (C) 2023, Ava Labs, Inc. All rights reserved.
// See the file LICENSE.md for licensing terms.

use crate::shale::{
self,
compact::{CompactSpace, CompactSpaceHeader},
disk_address::DiskAddress,
CachedStore, Obj, ShaleError, ShaleStore, SpaceId, Storable, StoredView,
};
pub use crate::{
config::{DbConfig, DbRevConfig},
storage::{buffer::DiskBufferConfig, WalConfig},
Expand All @@ -23,8 +17,18 @@ use crate::{
},
v2::api::{self, HashKey, KeyType, Proof, ValueType},
};
use crate::{
merkle,
shale::{
self,
compact::{CompactSpace, CompactSpaceHeader},
disk_address::DiskAddress,
CachedStore, Obj, ShaleError, ShaleStore, SpaceId, Storable, StoredView,
},
};
use async_trait::async_trait;
use bytemuck::{cast_slice, AnyBitPattern};

use metered::metered;
use parking_lot::{Mutex, RwLock};
use std::{
Expand Down Expand Up @@ -312,6 +316,16 @@ impl<S: ShaleStore<Node> + Send + Sync> api::DbView for DbRev<S> {
}

impl<S: ShaleStore<Node> + Send + Sync> DbRev<S> {
pub fn stream<K: KeyType>(
&self,
start_key: Option<K>,
) -> Result<merkle::MerkleKeyValueStream<'_, S>, api::Error> {
// TODO: get first key when start_key is None
self.merkle
.get_iter(start_key, self.header.kv_root)
.map_err(|e| api::Error::InternalError(e.into()))
}

fn flush_dirty(&mut self) -> Option<()> {
self.header.flush_dirty();
self.merkle.flush_dirty()?;
Expand Down
Loading