Skip to content

Commit

Permalink
DAOS-14411 dfuse: Add daos fs query
Browse files Browse the repository at this point in the history
Multiple cherry-picks to add daos fs query feature for fuse
statistics.  Helpful for understanding and tuning DAOS
performance when dfuse is used.

DAOS-13625 dfuse: Merge the info and projection_info structs. (#11881)
DAOS-13658 dfuse: Add filesystem query command. (#12367)
DAOS-12751 control: Add a daos filesystem evict command. (#12331)
DAOS-12751 dfuse: Improve evict command. (#12633)
DAOS-13625 dfuse: Remove dfuse_projection_info entirely. (#12796)
DAOS-13625 dfuse: Replace fs_handle with dfuse_info. (#12894)
DAOS-13625 dfuse: Add core inode_lookup() and inode_decref() functions. (#12573)
DAOS-14411 dfuse: Add per-container statistics. (#12819)
DAOS-14411 control: Expose dfuse statistics as yaml. (#13876)

Features: dfuse

Required-githooks: true

Change-Id: I8ae3cc743697c2434ae0d54b382ee6c585a3b033
Signed-off-by: Ashley Pittman <ashley.m.pittman@intel.com>
Signed-off-by: Jeff Olivier <jeffolivier@google.com>
  • Loading branch information
ashleypittman authored and jolivier23 committed Apr 26, 2024
1 parent 2d68c65 commit b4f7294
Show file tree
Hide file tree
Showing 28 changed files with 1,528 additions and 950 deletions.
24 changes: 24 additions & 0 deletions docs/user/filesystem.md
Original file line number Diff line number Diff line change
Expand Up @@ -696,6 +696,30 @@ These are two command line options to control the DFuse process itself.

These will affect all containers accessed via DFuse, regardless of any container attributes.

### Managing memory usage and disconnecting from containers

DFuse can be instructed to evict paths from local memory which drops any open handles on containers
or pools as well as reducing the working set size and memory consumption. This is an asynchronous
operation and there is no automatic way to tell if it's completed. In addition, any lookup of the
path specified in the eviction call will cause a new lookup and prevent the eviction from
completing.

Paths can be requested for eviction from dfuse using the `daos filesystem evict` command. This does
not change any data that is stored in DAOS in any way but rather releases local resources. This
command will return the inode number of the path as well as key dfuse metrics.

DFuse metrics can be queried with the `daos filesystem query` command which takes an optional
`--inode` parameter. This will return information on the number of inodes held in memory, the
number of open files as well as the number of pools and containers that DFuse is connected to. If
the `--inode` option is given then this command will also report if the inode is in memory or not.

Together these two commands can be used to request eviction of a path and to poll for its release,
although lookups from other processes might block the eviction process.

If `daos filesystem evict` is passed the root of the DFuse mount then the path itself cannot be
evicted - in this case all top-level entries in the directory are evicted instead and no inode
number is returned.

### Permissions

DFuse can serve data from any user's container, but needs appropriate permissions in order to do
Expand Down
Loading

0 comments on commit b4f7294

Please sign in to comment.