Major refactor of the code to use the new version of Trieste.

**New Features**
- Updated to latest version of Trieste
- Uses Trieste's YAML and JSON parsers and emitters
- Uses Trieste's more robust Unicode support
- Passes broken up into a `Reader`, which parses Rego files, and a `Rewriter` which performs unification
- Added the `encoding` and `graphs` builtins
- Added `set_log_level_from_string` and `setLogLevelFromString` to make it easier to set the log level. All executables now use this option.

**Improvements**
- The Interpreter caches intermediate versions of all modules, reducing query execution time
- All of the early passes have been rewritten and are more efficient, resulting in reduced parsing time
- The reduction of the input passes (and their restriction to a single file) along with other optimizations to header files has resulted in reduced compile times
- The `rego_fuzzer` xecutable allows for more fine-grained fuzzing and testing of the Rego toolchain
- Multiple bug fixes

**Deprecation**
- The `regoSetInputJSON` method is a misnomer, as the input does not strictly need to be JSON. This has been replaced with an (otherwise identical) function called
  `regoSetInputTerm` which does the same thing, and the `JSON` version has been deprecated.

**Breaking Changes**
- The `rego_trieste` executable has been replaced by the new `rego_fuzzer` executable

**New Test Coverage**
- `base64builtins`
- `base64urlbuiltins`
- `hexbuiltins`
- `jsonbuiltins`
- `reachable`
- `urlbuiltins`

Signed-off-by: Matthew A Johnson <matjoh@microsoft.com>

CHANGELOG

@@ -1,5 +1,38 @@
 # Changelog
 
+## 2024-06-04 - Version 0.4.0
+Major refactor of the code to use the new version of Trieste.
+
+**New Features**
+- Updated to latest version of Trieste
+- Uses Trieste's YAML and JSON parsers and emitters
+- Uses Trieste's more robust Unicode support
+- Passes broken up into a `Reader`, which parses Rego files, and a `Rewriter` which performs unification
+- Added the `encoding` and `graphs` builtins
+- Added `set_log_level_from_string` and `setLogLevelFromString` to make it easier to set the log level. All executables now use this option.
+
+**Improvements**
+- The Interpreter caches intermediate versions of all modules, reducing query execution time
+- All of the early passes have been rewritten and are more efficient, resulting in reduced parsing time
+- The reduction of the input passes (and their restriction to a single file) along with other optimizations to header files has resulted in reduced compile times
+- The `rego_fuzzer` xecutable allows for more fine-grained fuzzing and testing of the Rego toolchain
+- Multiple bug fixes
+
+**Deprecation**
+- The `regoSetInputJSON` method is a misnomer, as the input does not strictly need to be JSON. This has been replaced with an (otherwise identical) function called
+  `regoSetInputTerm` which does the same thing, and the `JSON` version has been deprecated.
+
+**Breaking Changes**
+- The `rego_trieste` executable has been replaced by the new `rego_fuzzer` executable
+
+**New Test Coverage**
+- `base64builtins`
+- `base64urlbuiltins`
+- `hexbuiltins`
+- `jsonbuiltins`
+- `reachable`
+- `urlbuiltins`
+
 ## 2024-01-19 - Version 0.3.11
 Minor improvements and bug fixes.
 

CMakeLists.txt

@@ -80,16 +80,29 @@ set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
 
 # -------------------- Find packages --------------------------
 
+# Used to provide
+# FetchContent_MakeAvailable_ExcludeFromAll
+FetchContent_Declare(
+    cmake_utils
+    GIT_REPOSITORY https://github.com/mjp41/cmake_utils
+    GIT_TAG 2bf98b5773ea7282197c823e205547d8c2e323c0
+    GIT_SHALLOW FALSE
+)
+
 FetchContent_Declare(
   trieste
   GIT_REPOSITORY https://github.com/microsoft/trieste
-  GIT_TAG        58a6eeeaeda4f5c96dad30c23b2d4e3feae7a60c
+  GIT_TAG        27b11771a0c07fca676e5d515f6cde61ece5117c
 )
 
+FetchContent_MakeAvailable(cmake_utils)
+
 set(CMAKE_POLICY_DEFAULT_CMP0077 NEW) 
 
 set(TRIESTE_USE_CXX17 ${REGOCPP_USE_CXX17})
 set(TRIESTE_BUILD_SAMPLES OFF)
+set(TRIESTE_BUILD_PARSERS ON)
+set(TRIESTE_CLEAN_INSTALL ON)
 
 if (REGOCPP_SPECULATIVE_LOAD_HARDENING)
   if (NOT CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
@@ -104,7 +117,7 @@ if (REGOCPP_SPECULATIVE_LOAD_HARDENING)
   set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -mllvm -x86-speculative-load-hardening")
 endif()
 
-FetchContent_MakeAvailable(trieste)
+FetchContent_MakeAvailable_ExcludeFromAll(trieste)
 
 if (REGOCPP_SPECULATIVE_LOAD_HARDENING)
   set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS_SAVED}")
@@ -119,7 +132,8 @@ else()
   file(GLOB_RECURSE ALL_SOURCE_FILES CONFIGURE_DEPENDS
         src/*.cc
         src/*.hh
-        src/passes/*.cc
+        src/reader/*.cc
+        src/unify/*.cc
         src/*.h
         include/rego/*.h
         include/rego/*.hh
@@ -177,7 +191,7 @@ set(INSTALL_CONFIGDIR cmake)
 set(INSTALL_LIBDIR lib)
 set(INSTALL_INCLUDEDIR include)
 
-install(TARGETS rego
+install(TARGETS rego snmalloc trieste snmallocshim-static re2
   EXPORT ${PROJECT_NAME}_Targets
   ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
   LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
@@ -208,16 +222,16 @@ install(FILES ${PROJECT_BINARY_DIR}/${PROJECT_NAME}Config.cmake
   DESTINATION ${CMAKE_INSTALL_DATAROOTDIR}/${PROJECT_NAME}/cmake)
 
 if (REGOCPP_BUILD_TOOLS)
-  install(TARGETS rego_interpreter rego_trieste RUNTIME)
+  install(TARGETS rego_interpreter rego_fuzzer RUNTIME)
 endif()
 
 if (REGOCPP_COPY_EXAMPLES)
   install(DIRECTORY tools/examples DESTINATION .)
 endif()
 
 if (REGOCPP_BUILD_TESTS)
-  install(TARGETS rego_test yaml_trieste RUNTIME)
-  install(FILES tests/regocpp.yaml tests/bigint.yaml DESTINATION tests)
+  install(TARGETS rego_test rego_test_c_api RUNTIME)
+  install(FILES tests/regocpp.yaml tests/bigint.yaml tests/bugs.yaml DESTINATION tests)
   install(DIRECTORY tests/aci DESTINATION tests)
 endif()
 

README.md

@@ -154,6 +154,8 @@ all the standard builtins. The following builtins are currently supported:
 - `arrays`
 - `bits`
 - `casts`
+- `encoding`
+- `graphs`
 - `numbers`
 - `objects`
 - `regex`
@@ -163,9 +165,6 @@ all the standard builtins. The following builtins are currently supported:
 - `types`
 - `units`
 - miscellaneous
-    * `base64_encode`
-    * `base64_decode`
-    * `json.marshal`
     * `opa.runtime`
     * `print`
     * `time.now_ns`
@@ -182,18 +181,16 @@ To build with the OPA tests available for testing, use one of the following pres
 - `release-opa`
 
 At present, we are **NOT** passing the following test suites in full:
-- `base64*`
 - `crypto*`
 - `glob*`
 - `graphql`
 - `invalidkeyerror`
-- `json*`
+- `json*` (except `jsonbuiltins`)
 - `jwt*`
 - `net*`
 - `planner-ir`
 - `providers-aws`
-- `reachable`
-- `urlbuiltins`
+- `time`
 - `walkbuiltin`
 
 ## Contributing

VERSION

@@ -1 +1 @@
-0.3.11
+0.4.0

cmake/regocppConfig.cmake.in

@@ -4,7 +4,7 @@ include(FetchContent)
 FetchContent_Declare(
   trieste
   GIT_REPOSITORY https://github.com/microsoft/trieste
-  GIT_TAG        c23669df20956a19c263ae1638e6fa976c31eec5
+  GIT_TAG        58a6eeeaeda4f5c96dad30c23b2d4e3feae7a60c
 )
 
 FetchContent_GetProperties(trieste)

examples/cpp/custom_builtin.cc

@@ -29,5 +29,5 @@ int main()
   Interpreter rego;
   rego.builtins().register_builtin(
     BuiltInDef::create(Location("myadd"), 2, add));
-  std::cout << rego.query("myadd(2, 3)");
+  logging::Output() << rego.query("myadd(2, 3)");
 }