Add specification for kernel binary update

sycl/doc/extensions/experimental/sycl_ext_oneapi_graph.asciidoc

@@ -556,6 +556,95 @@ Parameters:
 
 |===
 
+==== Dynamic Command Groups
+
+[source,c++]
+----
+namespace ext::oneapi::experimental{
+class dynamic_command_group {
+public:
+ dynamic_command_group(
+ command_graph<graph_state::modifiable> graph, std::vector<std::function<void(handler &)>> cgfList);
+
+ void set_active_cgf(int cgfIndex);
+};
+----
+
+Dynamic command-groups can be added as nodes to a graph. They provide a mechanism that
+allows updating the command-group function of a node after the graph is finalized.
+There is always one command-group function in the dynamic command-group that is set as active.
+
+===== Limitations
+
+Dynamic command-groups can only be used to update kernels. Trying to update a command group
+function that contains memory operations will result in an error.
+
+All the command-group functions in a dynamic command group must have identical dependencies.
+It is not allowed for a dynamic command group to have command-group functions that would
+result in a change to the graph topology when set to active. In practice, this means that
+any calls to `handler.depends_on()` must be identical for all the command-group functions
+in a dynamic command-group.
+
+See <<executable-graph-update, Executable Graph Update>> for more information
+about updating command-groups.
+
+Table {counter: tableNumber}. Member functions of the `dynamic_command_group` class.
+[cols="2a,a"]
+|===
+|Member Function|Description
+
+|
+[source,c++]
+----
+dynamic_command_group(command_graph<graph_state::modifiable> graph, std::vector<std::function<void(handler &)>> cgfList);
+----
+|Constructs a dynamic command group object that can be added as a node to a command_graph.
+
+Parameters:
+
+* `graph` - Graph to be associated with this `dynamic_command_group`.
+* `cgfList` - The list of command-group functions that can be activated for this dynamic command group.
+ The command-group function at index 0 will be active by default.
+
+Exceptions:
+
+* Throws synchronously with error code `invalid` if the graph wasn't created with
+ the `property::graph::assume_buffer_outlives_graph` property and the `dynamic_command_group`
+ is created with command-group functions that use buffers. See the
+ <<assume-buffer-outlives-graph-property, Assume-Buffer-Outlives-Graph>>
+ property for more information.
+
+* Throws with error code `invalid` if the `dynamic_command_group` is created with
+ command-group functions that are not kernel executions.
+
+|
+[source,c++]
+----
+void set_active_cgf(int cgfIndex);
+----
+
+| Sets the command-group function with index `cgfIndex` as active. The index of the
+command-group function in a `dynamic_command_group` is identical to its index in the
+`cgfList` vector when it was passed to the `dynamic_command_group` constructor.
+
+This change will be reflected immediately in the modifiable graph which contains this
+`dynamic_command_group`. The new value will not be reflected in any executable graphs
+created from that modifiable graph until `command_graph::update()` is called, passing
+the modified nodes, or a new executable graph is finalized from the modifiable graph.
+
+Setting `cgfIndex` to the index of the currently active command-group function is
+a no-op.
+
+Parameters:
+
+* `cgfIndex` - The index of the command-group function that should be set as active.
+
+Exceptions:
+
+* Throw with error code `invalid` if `cgfIndex` is not a valid index.
+
+|===
+
 ==== Depends-On Property
 
 [source,c++]
@@ -631,6 +720,8 @@ public:
  template<typename T>
  node add(T cgf, const property_list& propList = {});
 
+ node add(dynamic_command_group& dcg, const property_list& propList = {});
+
  void make_edge(node& src, node& dest);
 
  void print_graph(std::string path, bool verbose = false) const;
@@ -711,21 +802,39 @@ Updates to a graph will be scheduled after any in-flight executions of the same
 graph and will not affect previous submissions of the same graph. The user is
 not required to wait on any previous submissions of a graph before updating it.
 
-The only type of nodes that are currently able to be updated in a graph are
-kernel execution nodes.
-
-The aspects of a kernel execution node that can be configured during update are:
-
-* Parameters to the kernel.
-* Execution ND-Range of the kernel.
-
 To update an executable graph, the `property::graph::updatable` property must
 have been set when the graph was created during finalization. Otherwise, an
 exception will be thrown if a user tries to update an executable graph. This
 guarantee allows the backend to provide a more optimized implementation, if
 possible.
 
-===== Individual Node Update
+===== Supported Features:
+
+The only type of nodes that are currently able to be updated in a graphs are
+kernel execution nodes.
+
+There is two different API's that can be used to update a graph:
+
+* <individual-node-update, Individual Node Update>> which allows updating
+individual nodes of a command-graph.
+* <<whole-graph-update, Whole Graph Update>> which allows updating the
+entirety of the graph simultaneously by using another graph as a
+reference.
+
+The aspects of a kernel execution node that can be changed during update are
+different depending on the API used to perform the update:
+
+* For the <<individual-node-update, Individual Node Update>> API it's possible to update
+the kernel function, the parameters to the kernel, and the ND-Range.
+* For the <<whole-graph-update, Whole Graph Update>> API, only the parameters of the kernel
+and the ND-Range can be updated.
+
+===== Individual Node Update [[individual-node-update]]
+
+Individual nodes of an executable graph can be updated directly. Depending on the attribute
+of the node that requires updating, different API's should be used:
+
+===== Parameter Updates:
 
 Parameters to individual nodes in a graph in the `executable` state can be
 updated between graph executions using dynamic parameters. A `dynamic_parameter`
@@ -739,14 +848,6 @@ Parameter updates are performed using a `dynamic_parameter` instance by calling
 not registered, even if they use the same parameter value as a
 `dynamic_parameter`.
 
-The other node configuration that can be updated is the execution range of the
-kernel, this can be set through `node::update_nd_range()` or
-`node::update_range()` but does not require any prior registration.
-
-The executable graph can then be updated by passing the updated nodes to
-`command_graph<graph_state::executable>::update(node& node)` or
-`command_graph<graph_state::executable>::update(const std::vector<node>& nodes)`.
-
 Since the structure of the graph became fixed when finalizing, updating
 parameters on a node will not change the already defined dependencies between
 nodes. This is important to note when updating buffer parameters to a node,
@@ -762,6 +863,41 @@ dynamic parameter for the buffer can be registered with all the nodes which
 use the buffer as a parameter. Then a single `dynamic_parameter::update()` call
 will maintain the graphs data dependencies.
 
+===== Execution Range Updates:
+
+Another configuration that can be updated is the execution range of the
+kernel, this can be set through `node::update_nd_range()` or
+`node::update_range()` but does not require any prior registration.
+
+An alternative way to update the execution range of a node is to do so while
+updating command groups as described in the next section.
+
+===== Command Group Updates:
+
+The command-groups of a kernel node can be updated using dynamic command-groups.
+Dynamic command-groups allow replacing the command-group function of a kernel
+node with a different one. This effectively allows updating the kernel binaries, the
+argument types, the number of arguments and/or the execution ranges.
+
+Command-group updates are performed by creating an instance of the
+`dynamic_command_group` class. A dynamic command-group is created with a modifiable
+state graph and a list of possible command-group functions. Command-group functions
+within a dynamic command-group can then be set to active by using the member function
+`dynamic_command_group::setActiveCgf()`.
+
+Dynamic command-groups are compatible with dynamic parameters. This means that
+dynamic parameters can be used in command-group functions that are part of
+dynamic command-groups. Updates to such dynamic parameters will be reflected
+in the command-group functions once they are activated.
+
+===== Finalizing updates:
+
+Updating a node using the methods mentioned above will take effect immediately
+for nodes in modifiable command-graphs. However, for graphs that are in the executable
+state, in order to finalize the update, the updated nodes must be passed to
+`command_graph<graph_state::executable>::update(node& node)` or
+`command_graph<graph_state::executable>::update(const std::vector<node>& nodes)`.
+
 ===== Whole Graph Update [[whole-graph-update]]
 
 A graph in the executable state can have all of its nodes updated using the
@@ -1042,6 +1178,42 @@ Exceptions:
 |
 [source,c++]
 ----
+node add(dynamic_command_group& dcg, const property_list& propList = {});
+----
+
+| Adds the dynamic command-group `dcg` as a node to the graph and sets the
+current active command-group function in `dcg` as the executable for future
+executions of this graph node.
+
+The current active command-group function in `dcg` will be executed asynchronously
+when the graph is submitted to a queue. The requisites of this command-group
+function will be used to identify any dependent nodes in the graph
+to form edges with. The other command-group functions in `dcg` will be captured
+into the graph but will not be executed in a graph submission unless they are
+set to active.
+
+Constraints:
+
+* This member function is only available when the `command_graph` state is
+ `graph_state::modifiable`.
+
+Parameters:
+
+* `dcg` - Dynamic command-group object to be added as a node.
+
+* `propList` - Zero or more properties can be provided to the constructed node
+ via an instance of `property_list`. The `property::node::depends_on` property
+ can be passed here with a list of nodes to create dependency edges on.
+
+Returns: The dynamic command-group object node which has been added to the graph.
+
+Exceptions:
+
+* Throws synchronously with error code `invalid` if a queue is recording
+ commands to the graph.
+|
+[source,c++]
+----
 void make_edge(node& src, node& dest);
 ----
 
@@ -1157,8 +1329,9 @@ void update(node& node);
 ----
 
 | Updates an executable graph node that corresponds to `node`. `node` must be a
-kernel execution node. Kernel arguments and the ND-range of the node will be
-updated inside the executable graph to reflect the current values in `node`.
+kernel execution node. Kernel arguments and the command-group function of the 
+node will be updated inside the executable graph to reflect the current values
+in `node`.
 
 Updating these values will not change the structure of the graph.
 
@@ -1190,9 +1363,9 @@ void update(const std::vector<node>& nodes);
 ----
 
 | Updates all executable graph nodes that corresponds to the nodes contained in
-`nodes`. All nodes must be kernel nodes. Kernel arguments and the ND-range of
-each node will be updated inside the executable graph to reflect the current
-values in each node in `nodes`.
+`nodes`. All nodes must be kernel nodes. Kernel arguments and the
+command-group function of each node will be updated inside the executable graph
+to reflect the current values in each node in `nodes`.
 
 Updating these values will not change the structure of the graph.
 
@@ -1712,6 +1885,10 @@ the call to `queue::submit()` or `command_graph::add()` along with the calls to
 handler functions and this will not be reflected on future executions of the
 graph.
 
+Similarly, any command-group function inside a `dynamic_command_group` will be
+evaluated once, in index order, when submitted to the graph using
+`command_graph::add()`.
+
 Any code like this should be moved to a separate host-task and added to the
 graph via the recording or explicit APIs in order to be compatible with this
 extension.