[OpenMP][OMPT] Add grouping mechanism to libomptest

Added 'grouping' to assertion variants.
  tl;dr: Allows to correlate OMPT Target DataOp and Submit to their
         surrounding Target Region using the 'target_id'.

Improved QoL by adding aliases
  For example: H2D = `ompt_target_data_transfer_to_device`

Updated documentation (covers grouping)

Fixed small issues (e.g. Init toString)

Change-Id: I5934ff8912248f2dfa946d1c60170955dfee085d

openmp/libomptarget/test/ompTest/CMakeLists.txt

@@ -14,6 +14,7 @@ SHARED
 ./include/OmptTester.h
 ./src/InternalEvent.cpp
 ./src/InternalEventOperators.cpp
+./src/OmptAsserter.cpp
 ./src/OmptAssertEvent.cpp
 ./src/OmptCallbackHandler.cpp
 ./src/OmptTester.cpp

openmp/libomptarget/test/ompTest/README.md

@@ -6,6 +6,70 @@ Introduction
 ============
 TBD
 
+Testing macros
+==============
+
+## OMPT_SEQ_ASSERT
+TBD
+
+## OMPT_ASSERT_GROUPED_LIST
+
+## OMPT_ASSERT_GROUPED_LIST
+This macro will act like `OMPT_SEQ_ASSERT` with the addition of grouping.
+
+### Grouping
+
+This allows to generate and verify data during runtime of a test.
+Currently, we only use target region information which manifests into groups.
+This allows to correlate multiple events to a certain target region without
+manual interaction just by specifying a groupname for these events.
+
+When a target region is encountered and we are about to enter it, we will gather
+the `target_id` (non-EMI) -OR- `target_data->value` (EMI). This value is stored
+along the groupname for future reference. Upon target region end, we will erase
+the corresponding group. (Note: This will make the groupname available again.)
+
+Other asserted callbacks which may occur within target regions, will query their
+groupname: retrieving and verifying the value of the group vs. the observed
+event's own value.
+
+## Shorthands
+To allow for easier writing of tests and enhanced readability, the following set
+of aliases is introduced. The left hand side represents the original value,
+while the right hand side depicts the shorthand version.
+
+  |=================================================================|
+  | ompt_scope_endpoint_t                                           |
+  |-----------------------------------------------------------------|
+  | ompt_scope_begin                            | BEGIN             |
+  | ompt_scope_end                              | END               |
+  | ompt_scope_beginend                         | BEGINEND          |
+  |=================================================================|
+  | ompt_target_t                                                   |
+  |-----------------------------------------------------------------|
+  | ompt_target                                 | TARGET            |
+  | ompt_target_enter_data                      | ENTER_DATA        |
+  | ompt_target_exit_data                       | EXIT_DATA         |
+  | ompt_target_update                          | UPDATE            |
+  | ompt_target_nowait                          | TARGET_NOWAIT     |
+  | ompt_target_enter_data_nowait               | ENTER_DATA_NOWAIT |
+  | ompt_target_exit_data_nowait                | EXIT_DATA_NOWAIT  |
+  | ompt_target_update_nowait                   | UPDATE_NOWAIT     |
+  |=================================================================|
+  | ompt_target_data_op_t                                           |
+  |-----------------------------------------------------------------|
+  | ompt_target_data_alloc                      | ALLOC             |
+  | ompt_target_data_transfer_to_device         | H2D               |
+  | ompt_target_data_transfer_from_device       | D2H               |
+  | ompt_target_data_delete                     | DELETE            |
+  | ompt_target_data_associate                  | ASSOCIATE         |
+  | ompt_target_data_disassociate               | DISASSOCIATE      |
+  | ompt_target_data_alloc_async                | ALLOC_ASYNC       |
+  | ompt_target_data_transfer_to_device_async   | H2D_ASYNC         |
+  | ompt_target_data_transfer_from_device_async | D2H_ASYNC         |
+  | ompt_target_data_delete_async               | DELETE_ASYNC      |
+  |=================================================================|
+
 Limitations
 ===========
 Currently, there are some peculiarities which have to be kept in mind when using
@@ -22,3 +86,19 @@ this library:
   * It is not possible to check for device finalization callbacks, as libomptest
     is un-loaded before this callback occurs. Same holds true for the final
     ThreadEnd event(s).
+
+Miscellaneous
+=============
+
+## Default values
+
+To allow for easier writing of tests, many OMPT events may be created using less
+parameters than actually requested by the spec -- by using default values. These
+defaults are currently set to the corresponding data type's minimum as follows,
+for example integers will use: `std::numeric_limits<int>::min()`.
+
+When an expected / user-specified event has certain values set to the
+corresponding default, these values will be ignored. That is, when compared to
+an observed event, this property will be considered as 'equal' regardless of
+their actual equality relation.
+

openmp/libomptarget/test/ompTest/include/InternalEventCommon.h

@@ -31,9 +31,9 @@ enum class EventTy {
   DeviceFinalize,
   DeviceLoad,
   DeviceUnload,
-  BufferRequest,  // not part of OpenMP spec, used for implementation
-  BufferComplete, // not part of OpenMP spec, used for implementation
-  BufferRecord    // not part of OpenMP spec, used for implementation
+  BufferRequest,
+  BufferComplete,
+  BufferRecord // not part of OpenMP spec, used for implementation
 };
 
 struct InternalEvent {

openmp/libomptarget/test/ompTest/include/OmptAliases.h

@@ -0,0 +1,35 @@
+#ifndef OPENMP_LIBOMPTARGET_TEST_OMPTEST_OMPTALIASES_H
+#define OPENMP_LIBOMPTARGET_TEST_OMPTEST_OMPTALIASES_H
+
+#include <omp-tools.h>
+
+/// Aliases for enum: ompt_target_data_op_t
+constexpr ompt_target_data_op_t ALLOC = ompt_target_data_alloc;
+constexpr ompt_target_data_op_t H2D = ompt_target_data_transfer_to_device;
+constexpr ompt_target_data_op_t D2H = ompt_target_data_transfer_from_device;
+constexpr ompt_target_data_op_t DELETE = ompt_target_data_delete;
+constexpr ompt_target_data_op_t ASSOCIATE = ompt_target_data_associate;
+constexpr ompt_target_data_op_t DISASSOCIATE = ompt_target_data_disassociate;
+constexpr ompt_target_data_op_t ALLOC_ASYNC = ompt_target_data_alloc_async;
+constexpr ompt_target_data_op_t H2D_ASYNC =
+    ompt_target_data_transfer_to_device_async;
+constexpr ompt_target_data_op_t D2H_ASYNC =
+    ompt_target_data_transfer_from_device_async;
+constexpr ompt_target_data_op_t DELETE_ASYNC = ompt_target_data_delete_async;
+
+/// Aliases for enum: ompt_scope_endpoint_t
+constexpr ompt_scope_endpoint_t BEGIN = ompt_scope_begin;
+constexpr ompt_scope_endpoint_t END = ompt_scope_end;
+constexpr ompt_scope_endpoint_t BEGINEND = ompt_scope_beginend;
+
+/// Aliases for enum: ompt_scope_endpoint_t
+constexpr ompt_target_t TARGET = ompt_target;
+constexpr ompt_target_t ENTER_DATA = ompt_target_enter_data;
+constexpr ompt_target_t EXIT_DATA = ompt_target_exit_data;
+constexpr ompt_target_t UPDATE = ompt_target_update;
+constexpr ompt_target_t TARGET_NOWAIT = ompt_target_nowait;
+constexpr ompt_target_t ENTER_DATA_NOWAIT = ompt_target_enter_data_nowait;
+constexpr ompt_target_t EXIT_DATA_NOWAIT = ompt_target_exit_data_nowait;
+constexpr ompt_target_t UPDATE_NOWAIT = ompt_target_update_nowait;
+
+#endif // include guard

openmp/libomptarget/test/ompTest/include/OmptAssertEvent.h

@@ -14,64 +14,74 @@ namespace omptest{
 enum class AssertState { pass, fail };
 
 struct OmptAssertEvent {
-  static OmptAssertEvent ThreadBegin(ompt_thread_t ThreadType,
-                                     const std::string &Name);
+  static OmptAssertEvent ThreadBegin(const std::string &Name,
+                                     const std::string &Group,
+                                     ompt_thread_t ThreadType);
 
-  static OmptAssertEvent ThreadEnd(const std::string &Name);
+  static OmptAssertEvent ThreadEnd(const std::string &Name,
+                                   const std::string &Group);
 
-  static OmptAssertEvent ParallelBegin(int NumThreads, const std::string &Name);
+  static OmptAssertEvent ParallelBegin(const std::string &Name,
+                                       const std::string &Group,
+                                       int NumThreads);
 
-  static OmptAssertEvent ParallelEnd(const std::string &Name);
+  static OmptAssertEvent ParallelEnd(const std::string &Name,
+                                     const std::string &Group);
 
-  static OmptAssertEvent TaskCreate(const std::string &Name);
+  static OmptAssertEvent TaskCreate(const std::string &Name,
+                                    const std::string &Group);
 
-  static OmptAssertEvent TaskSchedule(const std::string &Name);
+  static OmptAssertEvent TaskSchedule(const std::string &Name,
+                                      const std::string &Group);
 
-  static OmptAssertEvent ImplicitTask(const std::string &Name);
+  static OmptAssertEvent ImplicitTask(const std::string &Name,
+                                      const std::string &Group);
 
   static OmptAssertEvent
-  Target(ompt_target_t Kind, ompt_scope_endpoint_t Endpoint,
+  Target(const std::string &Name, const std::string &Group, ompt_target_t Kind,
+         ompt_scope_endpoint_t Endpoint,
          int DeviceNum = std::numeric_limits<int>::min(),
          ompt_data_t *TaskData = std::numeric_limits<ompt_data_t *>::min(),
          ompt_id_t TargetId = std::numeric_limits<ompt_id_t>::min(),
-         const void *CodeptrRA = std::numeric_limits<void *>::min(),
-         const std::string &Name = "");
+         const void *CodeptrRA = std::numeric_limits<void *>::min());
 
   static OmptAssertEvent TargetEmi(
-      ompt_target_t Kind, ompt_scope_endpoint_t Endpoint, int DeviceNum,
+      const std::string &Name, const std::string &Group, ompt_target_t Kind,
+      ompt_scope_endpoint_t Endpoint, int DeviceNum,
       ompt_data_t *TaskData = std::numeric_limits<ompt_data_t *>::min(),
       ompt_data_t *TargetTaskData = std::numeric_limits<ompt_data_t *>::min(),
       ompt_data_t *TargetData = std::numeric_limits<ompt_data_t *>::min(),
-      const void *CodeptrRA = std::numeric_limits<void *>::min(),
-      const std::string &Name = "");
+      const void *CodeptrRA = std::numeric_limits<void *>::min());
 
-  /// Create a DataAlloc Event
-  static OmptAssertEvent TargetDataOp(ompt_id_t TargetId, ompt_id_t HostOpId,
+  static OmptAssertEvent TargetDataOp(const std::string &Name,
+                                      const std::string &Group,
+                                      ompt_id_t TargetId, ompt_id_t HostOpId,
                                       ompt_target_data_op_t OpType,
                                       void *SrcAddr, int SrcDeviceNum,
                                       void *DstAddr, int DstDeviceNum,
-                                      size_t Bytes, const void *CodeptrRA,
-                                      const std::string &Name);
+                                      size_t Bytes, const void *CodeptrRA);
 
   static OmptAssertEvent
-  TargetDataOp(ompt_target_data_op_t OpType, size_t Bytes,
+  TargetDataOp(const std::string &Name, const std::string &Group,
+               ompt_target_data_op_t OpType, size_t Bytes,
                void *SrcAddr = std::numeric_limits<void *>::min(),
                void *DstAddr = std::numeric_limits<void *>::min(),
                int SrcDeviceNum = std::numeric_limits<int>::min(),
                int DstDeviceNum = std::numeric_limits<int>::min(),
                ompt_id_t TargetId = std::numeric_limits<ompt_id_t>::min(),
                ompt_id_t HostOpId = std::numeric_limits<ompt_id_t>::min(),
-               const void *CodeptrRA = std::numeric_limits<void *>::min(),
-               const std::string &Name = "");
+               const void *CodeptrRA = std::numeric_limits<void *>::min());
 
   static OmptAssertEvent
-  TargetDataOpEmi(ompt_scope_endpoint_t Endpoint, ompt_data_t *TargetTaskData,
+  TargetDataOpEmi(const std::string &Name, const std::string &Group,
+                  ompt_scope_endpoint_t Endpoint, ompt_data_t *TargetTaskData,
                   ompt_data_t *TargetData, ompt_id_t *HostOpId,
                   ompt_target_data_op_t OpType, void *SrcAddr, int SrcDeviceNum,
                   void *DstAddr, int DstDeviceNum, size_t Bytes,
-                  const void *CodeptrRA, const std::string &Name);
+                  const void *CodeptrRA);
 
   static OmptAssertEvent TargetDataOpEmi(
+      const std::string &Name, const std::string &Group,
       ompt_target_data_op_t OpType, ompt_scope_endpoint_t Endpoint,
       size_t Bytes, void *SrcAddr = std::numeric_limits<void *>::min(),
       void *DstAddr = std::numeric_limits<void *>::min(),
@@ -80,84 +90,94 @@ struct OmptAssertEvent {
       ompt_data_t *TargetTaskData = std::numeric_limits<ompt_data_t *>::min(),
       ompt_data_t *TargetData = std::numeric_limits<ompt_data_t *>::min(),
       ompt_id_t *HostOpId = std::numeric_limits<ompt_id_t *>::min(),
-      const void *CodeptrRA = std::numeric_limits<void *>::min(),
-      const std::string &Name = "");
+      const void *CodeptrRA = std::numeric_limits<void *>::min());
 
-  static OmptAssertEvent TargetSubmit(ompt_id_t TargetId, ompt_id_t HostOpId,
-                                      unsigned int RequestedNumTeams,
-                                      const std::string &Name);
+  static OmptAssertEvent TargetSubmit(const std::string &Name,
+                                      const std::string &Group,
+                                      ompt_id_t TargetId, ompt_id_t HostOpId,
+                                      unsigned int RequestedNumTeams);
 
   static OmptAssertEvent
-  TargetSubmit(unsigned int RequestedNumTeams,
+  TargetSubmit(const std::string &Name, const std::string &Group,
+               unsigned int RequestedNumTeams,
                ompt_id_t TargetId = std::numeric_limits<ompt_id_t>::min(),
-               ompt_id_t HostOpId = std::numeric_limits<ompt_id_t>::min(),
-               const std::string &Name = "");
+               ompt_id_t HostOpId = std::numeric_limits<ompt_id_t>::min());
 
-  static OmptAssertEvent TargetSubmitEmi(ompt_scope_endpoint_t Endpoint,
-                                         ompt_data_t *TargetData,
-                                         ompt_id_t *HostOpId,
-                                         unsigned int RequestedNumTeams,
-                                         const std::string &Name);
+  static OmptAssertEvent
+  TargetSubmitEmi(const std::string &Name, const std::string &Group,
+                  ompt_scope_endpoint_t Endpoint, ompt_data_t *TargetData,
+                  ompt_id_t *HostOpId, unsigned int RequestedNumTeams);
 
   static OmptAssertEvent TargetSubmitEmi(
+      const std::string &Name, const std::string &Group,
       unsigned int RequestedNumTeams, ompt_scope_endpoint_t Endpoint,
       ompt_data_t *TargetData = std::numeric_limits<ompt_data_t *>::min(),
-      ompt_id_t *HostOpId = std::numeric_limits<ompt_id_t *>::min(),
-      const std::string &Name = "");
+      ompt_id_t *HostOpId = std::numeric_limits<ompt_id_t *>::min());
 
-  static OmptAssertEvent ControlTool(std::string &Name);
+  static OmptAssertEvent ControlTool(const std::string &Name,
+                                     const std::string &Group);
 
   static OmptAssertEvent DeviceInitialize(
-      int DeviceNum,
+      const std::string &Name, const std::string &Group, int DeviceNum,
       const char *Type = std::numeric_limits<const char *>::min(),
       ompt_device_t *Device = std::numeric_limits<ompt_device_t *>::min(),
       ompt_function_lookup_t LookupFn =
           std::numeric_limits<ompt_function_lookup_t>::min(),
-      const char *DocumentationStr = std::numeric_limits<const char *>::min(),
-      const std::string &Name = "");
+      const char *DocumentationStr = std::numeric_limits<const char *>::min());
 
-  static OmptAssertEvent DeviceFinalize(int DeviceNum, const std::string &Name);
+  static OmptAssertEvent DeviceFinalize(const std::string &Name,
+                                        const std::string &Group,
+                                        int DeviceNum);
 
   static OmptAssertEvent
-  DeviceLoad(int DeviceNum,
+  DeviceLoad(const std::string &Name, const std::string &Group, int DeviceNum,
              const char *Filename = std::numeric_limits<const char *>::min(),
              int64_t OffsetInFile = std::numeric_limits<int64_t>::min(),
              void *VmaInFile = std::numeric_limits<void *>::min(),
              size_t Bytes = std::numeric_limits<size_t>::min(),
              void *HostAddr = std::numeric_limits<void *>::min(),
              void *DeviceAddr = std::numeric_limits<void *>::min(),
-             uint64_t ModuleId = std::numeric_limits<int64_t>::min(),
-             const std::string &Name = "");
+             uint64_t ModuleId = std::numeric_limits<int64_t>::min());
 
-  static OmptAssertEvent DeviceUnload(const std::string &Name);
+  static OmptAssertEvent DeviceUnload(const std::string &Name,
+                                      const std::string &Group);
 
-  static OmptAssertEvent BufferRequest(int DeviceNum, ompt_buffer_t **Buffer,
-                                       size_t *Bytes, const std::string &Name);
+  static OmptAssertEvent BufferRequest(const std::string &Name,
+                                       const std::string &Group, int DeviceNum,
+                                       ompt_buffer_t **Buffer, size_t *Bytes);
 
-  static OmptAssertEvent BufferComplete(int DeviceNum, ompt_buffer_t *Buffer,
-                                        size_t Bytes,
+  static OmptAssertEvent BufferComplete(const std::string &Name,
+                                        const std::string &Group, int DeviceNum,
+                                        ompt_buffer_t *Buffer, size_t Bytes,
                                         ompt_buffer_cursor_t Begin,
-                                        int BufferOwned,
-                                        const std::string &Name);
+                                        int BufferOwned);
 
-  static OmptAssertEvent BufferRecord(ompt_record_ompt_t *Record,
-                                      const std::string &Name);
+  static OmptAssertEvent BufferRecord(const std::string &Name,
+                                      const std::string &Group,
+                                      ompt_record_ompt_t *Record);
 
   /// Allow move construction (due to std::unique_ptr)
   OmptAssertEvent(OmptAssertEvent &&o) = default;
   OmptAssertEvent &operator=(OmptAssertEvent &&o) = default;
 
   std::string getEventName() const;
 
+  std::string getEventGroup() const;
+
   internal::EventTy getEventType() const;
 
+  // ToDo: Maybe look for another way to get access to or copy the underlying
+  // event. Note: This can easily result in slicing!
+  internal::InternalEvent *getEvent() const;
+
   /// Make events comparable
   friend bool operator==(const OmptAssertEvent &A, const OmptAssertEvent &B);
 
   std::string toString(bool PrefixEventName = false) const;
 
 private:
-  OmptAssertEvent(const std::string &Name, internal::InternalEvent *IE);
+  OmptAssertEvent(const std::string &Name, const std::string &Group,
+                  internal::InternalEvent *IE);
   OmptAssertEvent(const OmptAssertEvent &o) = delete;
 
   static std::string getName(const std::string &Name,
@@ -169,10 +189,23 @@ struct OmptAssertEvent {
     return EName;
   }
 
+  static std::string getGroup(const std::string &Group) {
+    if (Group.empty())
+      return "default";
+
+    return Group;
+  }
+
   std::string Name;
+  std::string Group;
   std::unique_ptr<internal::InternalEvent> TheEvent;
 };
 
+struct AssertEventGroup {
+  AssertEventGroup(uint64_t TargetRegion) : TargetRegion(TargetRegion) {}
+  uint64_t TargetRegion;
+};
+
 bool operator==(const OmptAssertEvent &A, const OmptAssertEvent &B);
 
 } // namespace omptest