Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DAOS-15682 dfuse: Perform reads in larger chunks. #14212

Merged
merged 27 commits into from
Sep 26, 2024

Commits on Apr 22, 2024

  1. DAOS-15682 dfuse: Perform reads in larger chunks.

    When dfuse sees I/O as well-aligned 128k reads then read MB at
    a time and cache the result allowing for faster read bandwidth
    for well behaved applicaions and large files.
    
    Create a new in-memory descriptor for file contents, pull in a
    whole descriptor on first read and perform all other reads from
    the same result.
    
    This should give much higher bandwidth for well behaved applications
    and should be easy to extend to proper readahead in future.
    
    Test-tag: test_dfuse_caching_check
    
    Signed-off-by: Ashley Pittman <ashley.m.pittman@intel.com>
    ashleypittman committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    5a2ca11 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. Merge branch 'master' into amd/dfuse-chunk-read

    Test-tag: dfuse
    ashleypittman committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    2fa4c31 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2024

  1. Merge branch 'master' into amd/dfuse-chunk-read

    Test-tag: dfuse
    
    Signed-off-by: Ashley Pittman <ashley.m.pittman@intel.com>
    ashleypittman committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    79fda3b View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. Merge branch 'master' into amd/dfuse-chunk-read

    Test-tag: dfuse
    ashleypittman committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    aae10fd View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2024

  1. Configuration menu
    Copy the full SHA
    11eb551 View commit details
    Browse the repository at this point in the history
  2. Check the size after read.

    Signed-off-by: Ashley Pittman <ashley.m.pittman@intel.com>
    ashleypittman committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    5ca6b95 View commit details
    Browse the repository at this point in the history
  3. Set the len to 0.

    Signed-off-by: Ashley Pittman <ashley.m.pittman@intel.com>
    ashleypittman committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    3e98bf3 View commit details
    Browse the repository at this point in the history
  4. Fix an issue with position.

    Test-tag: test_dfuse_caching_check
    
    Signed-off-by: Ashley Pittman <ashley.m.pittman@intel.com>
    ashleypittman committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    be55b28 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    0986e7c View commit details
    Browse the repository at this point in the history
  6. Log when there is a short read.

    Skip-unit-tests: true
    Skip-fault-injection-test: true
    Test-tag: test_dfuse_caching_check
    
    Signed-off-by: Ashley Pittman <ashley.m.pittman@intel.com>
    ashleypittman committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    2a4173a View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    a963751 View commit details
    Browse the repository at this point in the history
  8. Improve error logging.

    Skip-unit-tests: true
    Skip-fault-injection-test: true
    Test-tag: test_dfuse_caching_check
    
    Signed-off-by: Ashley Pittman <ashley.m.pittman@intel.com>
    ashleypittman committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    1dd78ea View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2024

  1. Make bucket larger, 64 bit type.

    Skip-unit-tests: true
    Skip-fault-injection-test: true
    Test-tag: test_dfuse_caching_check
    Signed-off-by: Ashley Pittman <ashley.m.pittman@intel.com>
    ashleypittman committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    8691c67 View commit details
    Browse the repository at this point in the history
  2. Turn off debugging again.

    Skip-unit-tests: true
    Skip-fault-injection-test: true
    Test-tag: test_dfuse_caching_check
    Signed-off-by: Ashley Pittman <ashley.m.pittman@intel.com>
    ashleypittman committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    fa5689f View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. Move test to vm.

    Skip-unit-tests: true
    Skip-fault-injection-test: true
    Test-tag: test_dfuse_caching_check
    Signed-off-by: Ashley Pittman <ashley.m.pittman@intel.com>
    ashleypittman committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    6ada20d View commit details
    Browse the repository at this point in the history
  2. Reference count the descriptors and free when done.

    Skip-unit-tests: true
    Skip-fault-injection-test: true
    Test-tag: dfuse
    Signed-off-by: Ashley Pittman <ashley.m.pittman@intel.com>
    ashleypittman committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    cf31ce9 View commit details
    Browse the repository at this point in the history
  3. Merge branch 'master' into amd/dfuse-chunk-read

    Skip-unit-tests: true
    Skip-fault-injection-test: true
    Test-tag: test_dfuse_caching_check
    ashleypittman committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    d2b9fa9 View commit details
    Browse the repository at this point in the history

Commits on Apr 30, 2024

  1. Merge branch 'master' into amd/dfuse-chunk-read

    Features: dfuse,-test_dfuse_find
    ashleypittman committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    f79cb98 View commit details
    Browse the repository at this point in the history
  2. Fix formatting.

    Features: dfuse,-test_dfuse_find,-test_dfuse_daos_build_wt_pil4dfs
    Signed-off-by: Ashley Pittman <ashley.m.pittman@intel.com>
    ashleypittman committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    a43df49 View commit details
    Browse the repository at this point in the history

Commits on May 2, 2024

  1. Configuration menu
    Copy the full SHA
    fb632ff View commit details
    Browse the repository at this point in the history
  2. Fix merge.

    Features: dfuse
    
    Signed-off-by: Ashley Pittman <ashley.m.pittman@intel.com>
    ashleypittman committed May 2, 2024
    Configuration menu
    Copy the full SHA
    2bd4256 View commit details
    Browse the repository at this point in the history

Commits on May 20, 2024

  1. Configuration menu
    Copy the full SHA
    1810c0a View commit details
    Browse the repository at this point in the history

Commits on May 30, 2024

  1. Configuration menu
    Copy the full SHA
    eab66e3 View commit details
    Browse the repository at this point in the history
  2. Features: dfuse

    Required-githooks: true
    
    Signed-off-by: Jeff Olivier <jeffolivier@google.com>
    jolivier23 committed May 30, 2024
    Configuration menu
    Copy the full SHA
    b035b8a View commit details
    Browse the repository at this point in the history

Commits on May 31, 2024

  1. Configuration menu
    Copy the full SHA
    b12800f View commit details
    Browse the repository at this point in the history
  2. Features: dfuse

    Allow-unstable-test: true
    
    Required-githooks: true
    
    Signed-off-by: Jeff Olivier <jeffolivier@google.com>
    jolivier23 committed May 31, 2024
    Configuration menu
    Copy the full SHA
    89822d9 View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2024

  1. Merge branch 'master' into amd/dfuse-chunk-read

    Features: dfuse
    ashleypittman committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    149b8fc View commit details
    Browse the repository at this point in the history