Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rkuris/streaming iterator from start #346

Merged
merged 34 commits into from
Nov 15, 2023
Merged

Commits on Nov 3, 2023

  1. Streaming iterator spike

    rkuris committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    969684f View commit details
    Browse the repository at this point in the history

Commits on Nov 4, 2023

  1. Format

    rkuris committed Nov 4, 2023
    Configuration menu
    Copy the full SHA
    c806b7a View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2023

  1. WIP

    rkuris committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    9bb6cd3 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2023

  1. Complete implementation

    There are several TODOs here, but the code works for the two test cases
    I created. Both of these avoid making extension nodes, which are going
    away, and this code may need some tweaks to support values inside branch
    nodes.
    rkuris committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    46aecef View commit details
    Browse the repository at this point in the history
  2. Formatting

    rkuris committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    116d83b View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0c31eb9 View commit details
    Browse the repository at this point in the history
  4. No need to borrow K for get_node

    Per review comments
    rkuris committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    f5e69c6 View commit details
    Browse the repository at this point in the history
  5. Use let-else-return instead of match arms

    Per review comments
    rkuris committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    e6e36d0 View commit details
    Browse the repository at this point in the history
  6. Remove unnecessary continue

    Per review comments
    rkuris committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    e25d42b View commit details
    Browse the repository at this point in the history
  7. Rename current_node to previously_returned_node

    Per review comments
    rkuris committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    8273434 View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2023

  1. Redo the loop to have an IteratorState

    IteratorState is any of:
     - Start at the beginning of the trie
     - Start at a specific key value of the trie
     - Continue after a saved node and parents
    
    See the inline comments for more details.
    
    TODO:
     - Implement StartAtBegininng (and tests)
     - If a key is provided that doesn't exist, start iterating at the next
       one (if we really need this)
     - If an error is returned, we should next return None, then start from
       the beginning. This makes the resumption after an error a little
       cleaner. We'd need another IteratorState for this.
     - Remove extension nodes (depends on other diffs)
    rkuris committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    36ce8cc View commit details
    Browse the repository at this point in the history
  2. Removed unnecessary reference in remove()

    This was an artifact from an earlier change. Reverted back to original.
    rkuris committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    4d5f65c View commit details
    Browse the repository at this point in the history
  3. Fix typo

    Not sure how this got in there. Reverting it.
    rkuris committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    692018a View commit details
    Browse the repository at this point in the history
  4. Add some blank lines

    rkuris committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    17f4713 View commit details
    Browse the repository at this point in the history
  5. Remove pin! macro

    Seems like we can just reference self and it works!
    rkuris committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    7ab9003 View commit details
    Browse the repository at this point in the history
  6. In-person review comments

    Removed unnecessary clone
    Added additional comments
    Added TODO for a few things:
     - possible to return a reference to the value instead of a copy
     - should handle case of branch nodes with values while traversing
    
    Adding a randomized key-value test after extension nodes are removed is
    also recommended.
    rkuris committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    4867f82 View commit details
    Browse the repository at this point in the history

Commits on Nov 10, 2023

  1. Format

    rkuris committed Nov 10, 2023
    Configuration menu
    Copy the full SHA
    afe6357 View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2023

  1. Streaming iterator from start

    This walks down the leftmost branch every time, recording the path down,
    then figuring out what the key and value is at the bottom, then resumes.
    
    Also refactors get_key_from_parents, as this is now done 3 times in the
    code, sometimes when we add a leaf key value, and other times when not.
    rkuris committed Nov 11, 2023
    Configuration menu
    Copy the full SHA
    8b3648f View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2023

  1. Use enumerate/find instead of position

    Per review comments
    rkuris committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    9803ae3 View commit details
    Browse the repository at this point in the history
  2. Break with the leaf rather than recomputing

    From review comments
    rkuris committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    2af1991 View commit details
    Browse the repository at this point in the history
  3. Add doc comments to MerkleKeyValueStream

    Moved the comments about the behavior at the end of the stream to doc
    comments on the iterator.
    
    Per review comments.
    rkuris committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    936d0b5 View commit details
    Browse the repository at this point in the history
  4. Use let/else instead of if/let/else

    Per review comments
    rkuris committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    94a4244 View commit details
    Browse the repository at this point in the history
  5. Found_offset is now a separate statement

    Not 100% sure this is what the reviewer asked for, but this is okay too.
    rkuris committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    dcfb338 View commit details
    Browse the repository at this point in the history
  6. Fix doc comment problem

    rkuris committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    1e8193b View commit details
    Browse the repository at this point in the history
  7. Allow position to be None while walking up

    Per review comments
    rkuris committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    98f1b28 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    a862043 View commit details
    Browse the repository at this point in the history
  9. Remove unnecessary commented out code

    Per review comments
    rkuris committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    b952bc6 View commit details
    Browse the repository at this point in the history
  10. Oops, have to increment :)

    rkuris committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    b18d91f View commit details
    Browse the repository at this point in the history
  11. You can't just increment

    You have to add one only if it wasn't None
    rkuris committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    895507d View commit details
    Browse the repository at this point in the history
  12. Add additional test cases

    Added cases of starting in the middle and at the last key
    rkuris committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    33553e6 View commit details
    Browse the repository at this point in the history
  13. Lint

    rkuris committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    8dfd789 View commit details
    Browse the repository at this point in the history
  14. Remove some unwraps

    Per review comments
    rkuris committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    b64e98e View commit details
    Browse the repository at this point in the history
  15. Apply suggestions from code review

    The loop to calculate the next found position and address can be a
    little simpler using enumerate, skip, and filter_map.
    rkuris committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    55bc47a View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2023

  1. Make merkle::get_iter crate-only public

    Also removed incorrect comment and comment about DiskAddress::is_null
    rkuris committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    133f949 View commit details
    Browse the repository at this point in the history