Updated nroff-generated man pages

Signed-off-by: OFIWG Bot <ofiwg@lists.openfabrics.org>

man/man3/fi_av.3

@@ -14,7 +14,7 @@
 . ftr VB CB
 . ftr VBI CBI
 .\}
-.TH "fi_av" "3" "2024\-10\-11" "Libfabric Programmer\[cq]s Manual" "#VERSION#"
+.TH "fi_av" "3" "2024\-10\-15" "Libfabric Programmer\[cq]s Manual" "#VERSION#"
 .hy
 .SH NAME
 .PP
@@ -430,9 +430,10 @@ authorization keys.
 These fi_addr_t\[cq]s can be used as the target for local data transfer
 operations.
 .PP
-If the endpoint supports \f[V]FI_DIRECTED_RECV\f[R], these
-fi_addr_t\[cq]s can be used to restrict receive buffers to a specific
-endpoint address and authorization key.
+If the endpoint supports \f[V]FI_DIRECTED_RECV\f[R] or
+\f[V]FI_TAGGED_DIRECTED_RECV\f[R], these fi_addr_t\[cq]s can be used to
+restrict receive buffers to a specific endpoint address and
+authorization key.
 .PP
 For address vectors configured with FI_AV_USER_ID, all subsequent target
 events corresponding to the address being inserted will return

man/man3/fi_endpoint.3

@@ -14,7 +14,7 @@
 . ftr VB CB
 . ftr VBI CBI
 .\}
-.TH "fi_endpoint" "3" "2024\-10\-11" "Libfabric Programmer\[cq]s Manual" "#VERSION#"
+.TH "fi_endpoint" "3" "2024\-10\-15" "Libfabric Programmer\[cq]s Manual" "#VERSION#"
 .hy
 .SH NAME
 .PP
@@ -1502,8 +1502,9 @@ capability bits from the fi_info structure will be used.
 .PP
 The following capabilities apply to the receive attributes: FI_MSG,
 FI_RMA, FI_TAGGED, FI_ATOMIC, FI_REMOTE_READ, FI_REMOTE_WRITE, FI_RECV,
-FI_HMEM, FI_TRIGGER, FI_RMA_PMEM, FI_DIRECTED_RECV, FI_MULTI_RECV,
-FI_SOURCE, FI_RMA_EVENT, FI_SOURCE_ERR, FI_COLLECTIVE, and FI_XPU.
+FI_HMEM, FI_TRIGGER, FI_RMA_PMEM, FI_DIRECTED_RECV,
+FI_TAGGED_DIRECTED_RECV, FI_TAGGED_MULTI_RECV, FI_MULTI_RECV, FI_SOURCE,
+FI_RMA_EVENT, FI_SOURCE_ERR, FI_COLLECTIVE, and FI_XPU.
 .PP
 Many applications will be able to ignore this field and rely solely on
 the fi_info::caps field.

man/man3/fi_getinfo.3

@@ -14,7 +14,7 @@
 . ftr VB CB
 . ftr VBI CBI
 .\}
-.TH "fi_getinfo" "3" "2024\-10\-11" "Libfabric Programmer\[cq]s Manual" "#VERSION#"
+.TH "fi_getinfo" "3" "2024\-10\-15" "Libfabric Programmer\[cq]s Manual" "#VERSION#"
 .hy
 .SH NAME
 .PP
@@ -337,6 +337,17 @@ incoming message when matching it with a receive buffer.
 If this capability is not set, then the src_addr parameter for msg and
 tagged receive operations is ignored.
 .TP
+\f[I]FI_TAGGED_DIRECTED_RECV\f[R]
+Similar to FI_DIRECTED_RECV, but only applies to tagged receive
+operations.
+.TP
+\f[I]FI_EXACT_DIRECTED_RECV\f[R]
+Similar to FI_DIRECTED_RECV, but requires the source address to be
+exact, i.e., FI_ADDR_UNSPEC is not allowed.
+This capability can be used alone, or in conjunction with
+FI_DIRECTED_RECV or FI_TAGGED_DIRECTED_RECV as a modifier to disallow
+FI_ADDR_UNSPEC being used as the source address.
+.TP
 \f[I]FI_FENCE\f[R]
 Indicates that the endpoint support the FI_FENCE flag on data transfer
 operations.
@@ -386,6 +397,10 @@ send-only or receive-only.
 Specifies that the endpoint must support the FI_MULTI_RECV flag when
 posting receive buffers.
 .TP
+\f[I]FI_TAGGED_MULTI_RECV\f[R]
+Specifies that the endpoint must support the FI_MULTI_RECV flag when
+posting tagged receive buffers.
+.TP
 \f[I]FI_NAMED_RX_CTX\f[R]
 Requests that endpoints which support multiple receive contexts allow an
 initiator to target (or name) a specific receive context as part of a
@@ -527,15 +542,16 @@ A provider may optionally report non-selected secondary capabilities if
 doing so would not compromise performance or security.
 .PP
 Primary capabilities: FI_MSG, FI_RMA, FI_TAGGED, FI_ATOMIC,
-FI_MULTICAST, FI_NAMED_RX_CTX, FI_DIRECTED_RECV, FI_HMEM, FI_COLLECTIVE,
-FI_XPU, FI_AV_USER_ID, FI_PEER
+FI_MULTICAST, FI_NAMED_RX_CTX, FI_DIRECTED_RECV,
+FI_TAGGED_DIRECTED_RECV, FI_HMEM, FI_COLLECTIVE, FI_XPU, FI_AV_USER_ID,
+FI_PEER
 .PP
 Primary modifiers: FI_READ, FI_WRITE, FI_RECV, FI_SEND, FI_REMOTE_READ,
 FI_REMOTE_WRITE
 .PP
-Secondary capabilities: FI_MULTI_RECV, FI_SOURCE, FI_RMA_EVENT,
-FI_SHARED_AV, FI_TRIGGER, FI_FENCE, FI_LOCAL_COMM, FI_REMOTE_COMM,
-FI_SOURCE_ERR, FI_RMA_PMEM.
+Secondary capabilities: FI_MULTI_RECV, FI_TAGGED_MULTI_RECV, FI_SOURCE,
+FI_RMA_EVENT, FI_SHARED_AV, FI_TRIGGER, FI_FENCE, FI_LOCAL_COMM,
+FI_REMOTE_COMM, FI_SOURCE_ERR, FI_RMA_PMEM.
 .SH MODE
 .PP
 The operational mode bits are used to convey requirements that an

man/man3/fi_mr.3

@@ -14,7 +14,7 @@
 . ftr VB CB
 . ftr VBI CBI
 .\}
-.TH "fi_mr" "3" "2024\-10\-11" "Libfabric Programmer\[cq]s Manual" "#VERSION#"
+.TH "fi_mr" "3" "2024\-10\-15" "Libfabric Programmer\[cq]s Manual" "#VERSION#"
 .hy
 .SH NAME
 .PP
@@ -152,6 +152,17 @@ mode bits, specified through the domain attributes (mr_mode field).
 Each mr_mode bit requires that an application take specific steps in
 order to use memory buffers with libfabric interfaces.
 .PP
+As a special case, a new memory region can be created from an existing
+memory region.
+Such a new memory region is called a sub-MR, and the existing memory
+region is called the base MR.
+Sub-MRs may be used to shared hardware resources, such as virtual to
+physical address translations and page pinning.
+This can improve performance when creating and destroying sub-regions
+that need different access rights.
+The base MR itself can also be a sub-MR, allowing for a hierarchy of
+memory regions.
+.PP
 The following apply to memory registration.
 .TP
 \f[I]Default Memory Registration\f[R]
@@ -630,8 +641,8 @@ passed directly into calls as function parameters.
 \f[C]
 struct fi_mr_attr {
     union {
-      const struct iovec *mr_iov;
-      const struct fi_mr_dmabuf *dmabuf;
+        const struct iovec *mr_iov;
+        const struct fi_mr_dmabuf *dmabuf;
     };
     size_t             iov_count;
     uint64_t           access;
@@ -650,6 +661,8 @@ struct fi_mr_attr {
     } device;
     void               *hmem_data;
     size_t             page_size;
+    const struct fid_mr *base_mr;
+    size_t             sub_mr_cnt;
 };
 
 struct fi_mr_auth_key {
@@ -880,6 +893,32 @@ failed transfers to or from the region.
 .PP
 Providers may choose to ignore page size.
 This will result in a provider selected page size always being used.
+.SS base_mr
+.PP
+If non-NULL, create a sub-MR from an existing memory region specified by
+the base_mr field.
+.PP
+The sub-MR must be fully contained within the base MR; however, the
+sub-MR has its own authorization keys and access rights.
+The following attributes are inherited from the base MR, and as a
+result, are ignored when creating the sub-MR:
+.PP
+iface, device, hmem_data, page_size
+.PP
+The sub-MR should hold a reference to the base MR.
+When fi_close is called on the base MR, the call would fail if there are
+any outstanding sub-MRs.
+.PP
+The base_mr field must be NULL if the FI_MR_DMABUF flag is set.
+.SS sub_mr_cnt
+.PP
+The number of sub-MRs expected to be created from the memory region.
+This value is not a limit.
+Instead, it is a hint to the provider to allow provider specific
+optimization for sub-MR creation.
+For example, the provider may reserve access keys or pre-allocation
+fid_mr objects.
+The provider may ignore this hint.
 .SS fi_hmem_ze_device
 .PP
 Returns an hmem device identifier for a level zero <driver, device>
@@ -979,6 +1018,13 @@ fi_mr_attr structure.
 This flag is only usable for domains opened with FI_HMEM capability
 support.
 .TP
+\f[I]FI_MR_SINGLE_USE\f[R]
+This flag indicates that the memory region is only used for a single
+operation.
+After the operation is complete, the key associated with the memory
+region is automatically invalidated and can no longer be used for remote
+access.
+.TP
 \f[I]FI_AUTH_KEY\f[R]
 Only valid with domains configured with FI_AV_AUTH_KEY.
 When used with fi_mr_regattr, this flag denotes that the

man/man3/fi_msg.3

@@ -14,7 +14,7 @@
 . ftr VB CB
 . ftr VBI CBI
 .\}
-.TH "fi_msg" "3" "2024\-10\-11" "Libfabric Programmer\[cq]s Manual" "#VERSION#"
+.TH "fi_msg" "3" "2024\-10\-15" "Libfabric Programmer\[cq]s Manual" "#VERSION#"
 .hy
 .SH NAME
 .PP
@@ -186,6 +186,12 @@ of the endpoint is to write CQ entries for all successful completions.
 See the flags discussion below for more details.
 The requested message size that can be used with fi_inject is limited by
 inject_size.
+.PP
+If FI_HMEM is enabled, the fi_inject call can only accept buffer with
+iface equal to FI_HMEM_SYSTEM if the provider requires the FI_MR_HMEM
+mr_mode.
+This limitation applies to all the fi_*inject* calls and does not affect
+how inject_size is reported.
 .SS fi_senddata
 .PP
 The send data call is similar to fi_send, but allows for the sending of

man/man3/fi_tagged.3

@@ -14,7 +14,7 @@
 . ftr VB CB
 . ftr VBI CBI
 .\}
-.TH "fi_tagged" "3" "2024\-10\-11" "Libfabric Programmer\[cq]s Manual" "#VERSION#"
+.TH "fi_tagged" "3" "2024\-10\-15" "Libfabric Programmer\[cq]s Manual" "#VERSION#"
 .hy
 .SH NAME
 .PP
@@ -283,6 +283,26 @@ This may require that the underlying provider implementation copy the
 data into a local buffer and transfer out of that buffer.
 This flag can only be used with messages smaller than inject_size.
 .TP
+\f[I]FI_MULTI_RECV\f[R]
+Applies to posted tagged receive operations when the
+FI_TAGGED_MULTI_RECV capability is enabled.
+This flag allows the user to post a single tagged receive buffer that
+will receive multiple incoming messages.
+Received messages will be packed into the receive buffer until the
+buffer has been consumed.
+Use of this flag may cause a single posted receive operation to generate
+multiple events as messages are placed into the buffer.
+The placement of received data into the buffer may be subjected to
+provider specific alignment restrictions.
+.PP
+The buffer will be released by the provider when the available buffer
+space falls below the specified minimum (see FI_OPT_MIN_MULTI_RECV).
+Note that an entry to the associated receive completion queue will
+always be generated when the buffer has been consumed, even if other
+receive completions have been suppressed (i.e.\ the Rx context has been
+configured for FI_SELECTIVE_COMPLETION).
+See the FI_MULTI_RECV completion flag \f[V]fi_cq\f[R](3).
+.TP
 \f[I]FI_INJECT_COMPLETE\f[R]
 Applies to fi_tsendmsg.
 Indicates that a completion should be generated when the source
@@ -315,7 +335,8 @@ operation (inclusive) to the posting of a subsequent fenced operation
 .TP
 \f[I]FI_AUTH_KEY\f[R]
 Only valid with domains configured with FI_AV_AUTH_KEY and
-connectionless endpoints configured with FI_DIRECTED_RECV.
+connectionless endpoints configured with FI_DIRECTED_RECV or
+FI_TAGGED_DIRECTED_RECV.
 When used with fi_trecvmsg, this flag denotes that the src_addr is an
 authorization key fi_addr_t instead of an endpoint fi_addr_t.
 .PP