Completed documentation.

1. Cleaned up versioning and copyright tags.
2. Added homepage for API documentation.
3. Added documentation for Optimizer classes.

CMakeLists.txt

@@ -5,6 +5,9 @@ add_compile_options(-Wall -Wextra -Wpedantic)
 
 set(CMAKE_CXX_STANDARD 17)
 
+# Define current version
+set(SVGDCPP_VERSION "1.0.0")
+
 # Define variables
 set(SVGDCPP_CMAKE_INSTALL_DIR "${CMAKE_INSTALL_PREFIX}/share/${PROJECT_NAME}/cmake")
 set(SVGDCPP_INCLUDE_INSTALL_DIR "${CMAKE_INSTALL_PREFIX}/include/${PROJECT_NAME}") # assign include destination to variable

README.md

@@ -10,7 +10,8 @@ This library provides the [Stein Variational Gradient Descent](https://arxiv.org
     - `-D include_eigen=true`
     - `-D CMAKE_BUILD_TYPE=Release`
 - [OpenMP v4.5+ (201511)](https://www.openmp.org/) - _should be shipped with your compiler_
-- [Doxygen](https://www.doxygen.nl/) - _required only if documentation is desired_
+- [GraphViz](https://graphviz.org/) - _required only if documentation is desired; can be installed using `apt install graphviz`_
+- [Doxygen](https://www.doxygen.nl/) - _required only if documentation is desired; can be installed using `apt install doxygen`_
 
 ## Installation
 1. Clone this repository and enter it.
@@ -32,7 +33,7 @@ This library provides the [Stein Variational Gradient Descent](https://arxiv.org
     ```
 
 ## Getting Started
-See the [examples directory](examples/) for tutorials on how to use them and see [here](doc/instructions.md) for detailed instructions.
+See the [examples directory](./examples/) for tutorials on how to use them and see [here](./doc/instructions.md) for detailed instructions.
 
 ## Tests
 Unit tests have been provided to aid source code development. Besides identifying the kinds of testing imposed on the source code, looking into the test files can help you understand how the algorithm works. All you need to do is build the code with `-D CMAKE_BUILD_TYPE=Debug` and then run the tests either using `CTest`:

doc/Doxyfile.in

@@ -38,7 +38,7 @@ PROJECT_NAME           = "SVGDCpp"
 # could be handy for archiving the generated documentation or if some version
 # control system is used.
 
-PROJECT_NUMBER         = "0.1"
+PROJECT_NUMBER         = @SVGDCPP_VERSION@
 
 # Using the PROJECT_BRIEF tag one can provide an optional one line description
 # for a project that appears at the top of each page and should give viewer a
@@ -830,7 +830,8 @@ WARN_LOGFILE           =
 # spaces. See also FILE_PATTERNS and EXTENSION_MAPPING
 # Note: If this tag is empty the current directory is searched.
 
-INPUT                  = @CMAKE_SOURCE_DIR@/include/
+INPUT                  = @CMAKE_SOURCE_DIR@/include/ \
+                         ../README.md
 
 # This tag can be used to specify the character encoding of the source files
 # that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses
@@ -982,7 +983,7 @@ FILTER_SOURCE_PATTERNS =
 # (index.html). This can be useful if you have a project on for instance GitHub
 # and want to reuse the introduction page also for the doxygen output.
 
-USE_MDFILE_AS_MAINPAGE =
+USE_MDFILE_AS_MAINPAGE = ../README.md
 
 #---------------------------------------------------------------------------
 # Configuration options related to source browsing
@@ -1251,7 +1252,7 @@ HTML_COLORSTYLE_GAMMA  = 80
 # The default value is: NO.
 # This tag requires that the tag GENERATE_HTML is set to YES.
 
-HTML_TIMESTAMP         = NO
+HTML_TIMESTAMP         = YES
 
 # If the HTML_DYNAMIC_MENUS tag is set to YES then the generated HTML
 # documentation will contain a main index with vertical navigation menus that

doc/instructions.md

@@ -30,7 +30,7 @@ Mainly, there are 5 elements required to run a minimally working SVGD algorithm:
 3. `Optimizer` object;
 4. `SVGD` object.
 
-The first four are required by the fifth as shared pointer arguments.
+The first four are required by the fifth as shared pointer arguments. The `Model` and `Kernel` classes are used to compute $\hat{\phi}^*$ in Equation (8) of the [SVGD paper](https://arxiv.org/abs/1608.04471), and the step size $\epsilon$ is provided with the `Optimizer` class.
 
 ### Basic usage
 This example uses the provided classes `MultivariateNormal`, `GaussianRBF`, and `Adam` optimizer to estimate a multivariate normal distribution.

include/Optimizer

@@ -1,3 +1,13 @@
+/**
+ * @defgroup Optimizer_Module Optimizer Module
+ * @brief Provides functionality to define optimizer functions.
+ * @details This module provides the base and derived @ref Optimizer classes.
+ *
+ * @code{.cpp}
+ * #include <SVGDCpp/Optimizer>
+ * @endcode
+ */
+
 #ifndef SVGDCPP_OPTIMIZER_MODULE_HPP
 #define SVGDCPP_OPTIMIZER_MODULE_HPP
 

include/SVGDCpp/Core.hpp

@@ -2,10 +2,8 @@
  * @file Core.hpp
  * @author Khai Yi Chin (khaiyichin@gmail.com)
  * @brief Core header to provide common types and functions.
- * @version 0.1
- * @date 2024-03-23
  *
- * @copyright Copyright (c) 2024
+ * @copyright Copyright (c) 2024 Khai Yi Chin
  *
  */
 

include/SVGDCpp/Exceptions.hpp

@@ -2,10 +2,8 @@
  * @file Exceptions.hpp
  * @author Khai Yi Chin (khaiyichin@gmail.com)
  * @brief Exceptions header to provide some commonly used custom exceptions.
- * @version 0.1
- * @date 2024-10-16
  * 
- * @copyright Copyright (c) 2024
+ * @copyright Copyright (c) 2024 Khai Yi Chin
  * 
  */
 
@@ -15,8 +13,13 @@
 #include <exception>
 #include <string>
 
-#define SVGDCPP_LOG_PREFIX std::string("SVGDCpp: ")
+#define SVGDCPP_LOG_PREFIX std::string("SVGDCpp: ") ///< Convenience macro to prefix output logs.
 
+/**
+ * @class DimensionMismatchException
+ * @brief Exception for dimension mismatch type errors.
+ * @ingroup Core_Module
+ */
 class DimensionMismatchException : public std::exception
 {
 public:
@@ -32,6 +35,11 @@ class DimensionMismatchException : public std::exception
     std::string message_;
 };
 
+/**
+ * @class UnsetException
+ * @brief Exception for unset value type errors.
+ * @ingroup Core_Module
+ */
 class UnsetException : public std::exception
 {
 public:

include/SVGDCpp/Kernel/GaussianRBFKernel.hpp

@@ -2,10 +2,8 @@
  * @file GaussianRBFKernel.hpp
  * @author Khai Yi Chin (khaiyichin@gmail.com)
  * @brief Gaussian RBF kernel class header.
- * @version 0.1
- * @date 2024-03-23
  *
- * @copyright Copyright (c) 2024
+ * @copyright Copyright (c) 2024 Khai Yi Chin
  *
  */
 

include/SVGDCpp/Kernel/Kernel.hpp

@@ -1,11 +1,9 @@
 /**
  * @file Kernel.hpp
  * @author Khai Yi Chin (khaiyichin@gmail.com)
- * @brief Kernel class header
- * @version 0.1
- * @date 2024-03-23
+ * @brief Kernel class header.
  *
- * @copyright Copyright (c) 2024
+ * @copyright Copyright (c) 2024 Khai Yi Chin
  *
  */
 #ifndef SVGDCPP_KERNEL_HPP
@@ -350,7 +348,7 @@ class Kernel
     }
 
     /**
-     * @brief Execute methods required for each step.
+     * @brief Execute methods required at each step.
      * @details Override this method to include methods that you need to have run every step of the iteration
      * @a e.g., computing the scale parameter of the kernel function.
      *

include/SVGDCpp/Model/Model.hpp

@@ -2,10 +2,8 @@
  * @file Model.hpp
  * @author Khai Yi Chin (khaiyichin@gmail.com)
  * @brief Model class header
- * @version 0.1
- * @date 2024-03-22
  *
- * @copyright Copyright (c) 2024
+ * @copyright Copyright (c) 2024 Khai Yi Chin
  *
  */
 
@@ -408,7 +406,7 @@ class Model
     }
 
     /**
-     * @brief Execute methods required for each step.
+     * @brief Execute methods required at each step.
      * @details Override this method to include methods that you need to have run every step of the iteration.
      *
      */

include/SVGDCpp/Model/MultivariateNormal.hpp

@@ -2,10 +2,8 @@
  * @file MultivariateNormal.hpp
  * @author Khai Yi Chin (khaiyichin@gmail.com)
  * @brief Multivariate normal model class header.
- * @version 0.1
- * @date 2024-03-22
  *
- * @copyright Copyright (c) 2024
+ * @copyright Copyright (c) 2024 Khai Yi Chin
  *
  */
 

include/SVGDCpp/Optimizer/AdaGrad.hpp

@@ -1,12 +1,33 @@
+/**
+ * @file AdaGrad.hpp
+ * @author Khai Yi Chin (khaiyichin@gmail.com)
+ * @brief AdaGrad optimizer class header.
+ *
+ * @copyright Copyright (c) 2024 Khai Yi Chin
+ *
+ */
 #ifndef SVGDCPP_ADAGRAD_HPP
 #define SVGDCPP_ADAGRAD_HPP
 
 #include "../Core.hpp"
 #include "Optimizer.hpp"
 
+/**
+ * @class AdaGrad
+ * @brief This class provides the AdaGrad optimizer.
+ * @ingroup Optimizer_Module
+ */
 class AdaGrad : public Optimizer
 {
 public:
+    /**
+     * @brief Default constructor.
+     *
+     * @param dimension Dimension of the problem.
+     * @param num_particles Number of particles used in the problem.
+     * @param lr Learning rate.
+     * @param epsilon Numerical stabilizer.
+     */
     AdaGrad(const size_t &dimension,
             const size_t &num_particles,
             const double &lr,
@@ -15,13 +36,27 @@ class AdaGrad : public Optimizer
           dimension_(dimension),
           num_particles_(num_particles) {}
 
+    /**
+     * @brief Default destructor.
+     *
+     */
     virtual ~AdaGrad() {}
 
+    /**
+     * @brief Initialize the optimizer.
+     *
+     */
     virtual void Initialize() override
     {
         sum_of_sq_grad_ = Eigen::MatrixXd::Zero(dimension_, num_particles_).array();
     }
 
+    /**
+     * @brief Execute the optimization of the gradient at each step.
+     *
+     * @param grad_matrix Gradient matrix for each particle coordinate.
+     * @return Optimized gradient matrix.
+     */
     virtual Eigen::MatrixXd Step(const Eigen::MatrixXd &grad_matrix)
     {
         sum_of_sq_grad_ += grad_matrix.array().square();
@@ -30,13 +65,13 @@ class AdaGrad : public Optimizer
     }
 
 protected:
-    size_t dimension_;
+    size_t dimension_; ///< Dimension of the problem.
 
-    size_t num_particles_;
+    size_t num_particles_; ///< Number of particles used in the problem.
 
-    double decay_rate_;
+    double decay_rate_; ///< Decay rate (unused).
 
-    Eigen::ArrayXXd sum_of_sq_grad_;
+    Eigen::ArrayXXd sum_of_sq_grad_; ///< Sum of squares gradient matrix.
 };
 
 #endif

include/SVGDCpp/Optimizer/Adam.hpp

@@ -1,12 +1,35 @@
+/**
+ * @file Adam.hpp
+ * @author Khai Yi Chin (khaiyichin@gmail.com)
+ * @brief Adam optimizer class header.
+ *
+ * @copyright Copyright (c) 2024 Khai Yi Chin
+ *
+ */
 #ifndef SVGDCPP_ADAM_HPP
 #define SVGDCPP_ADAM_HPP
 
 #include "../Core.hpp"
 #include "Optimizer.hpp"
 
+/**
+ * @class Adam
+ * @brief This class provides the Adam optimizer.
+ * @ingroup Optimizer_Module
+ */
 class Adam : public Optimizer
 {
 public:
+    /**
+     * @brief Default constructor.
+     *
+     * @param dimension Dimension of the problem.
+     * @param num_particles Number of particles used in the problem.
+     * @param lr Learning rate.
+     * @param beta1 Exponential decay rate for the 1st moment estimates.
+     * @param beta2 Exponential decay rate for the 2nd moment estimates.
+     * @param epsilon Numerical stabilizer.
+     */
     Adam(const size_t &dimension,
          const size_t &num_particles,
          const double &lr,
@@ -25,8 +48,16 @@ class Adam : public Optimizer
         }
     }
 
+    /**
+     * @brief Default destructor.
+     *
+     */
     virtual ~Adam() {}
 
+    /**
+     * @brief Initialize the optimizer.
+     *
+     */
     virtual void Initialize() override
     {
         sum_of_sq_grad_ = Eigen::MatrixXd::Zero(dimension_, num_particles_).array();
@@ -35,6 +66,12 @@ class Adam : public Optimizer
         counter_ = 0;
     }
 
+    /**
+     * @brief Execute the optimization of the gradient at each step.
+     *
+     * @param grad_matrix Gradient matrix for each particle coordiniate.
+     * @return Optimized gradient matrix.
+     */
     virtual Eigen::MatrixXd Step(const Eigen::MatrixXd &grad_matrix)
     {
         sum_of_grad_ = decay_rate_1_ * sum_of_grad_ + (1 - decay_rate_1_) * grad_matrix.array();                // Momentum part
@@ -46,24 +83,31 @@ class Adam : public Optimizer
     }
 
 protected:
+    /**
+     * @brief Correct moment estimate for bias.
+     *
+     * @param arr Biased gradient estimate array.
+     * @param decay Decay rate.
+     * @return Bias-corrected moment estimate.
+     */
     Eigen::ArrayXXd CorrectForBias(const Eigen::ArrayXXd arr, const double &decay)
     {
         return arr / (1.0 - std::pow(decay, counter_));
     }
 
-    size_t counter_ = 0;
+    size_t counter_ = 0; ///< Time step counter.
 
-    size_t dimension_;
+    size_t dimension_; ///< Dimension of the problem.
 
-    size_t num_particles_;
+    size_t num_particles_; ///< Number of particles used in the problem.
 
-    double decay_rate_1_;
+    double decay_rate_1_; ///< Exponential decay rate for the 1st moment estimate.
 
-    double decay_rate_2_;
+    double decay_rate_2_; ///< Exponential decay rate for the 2nd moment estimate.
 
-    Eigen::ArrayXXd sum_of_sq_grad_;
+    Eigen::ArrayXXd sum_of_sq_grad_; ///< 2nd biased moment estimate.
 
-    Eigen::ArrayXXd sum_of_grad_;
+    Eigen::ArrayXXd sum_of_grad_; ///< 1st biased moment estimate
 };
 
 #endif