MDPSolver v0.9.7

MDPSolver v0.9.7 Release Notes

Highlights

• Users can now select the average reward optimality criteria.

Coming up

• We are currently testing and developing the implementation of the built-in models.

Parallel computing for Markov Decision Processes

MDPSolver is a Python package for Markov Decision Processes (MDPs) with discounted rewards and infinite-horizon.

Features

• Fast solver: Our C++-based solver is substantially faster than other MDP packages available for Python. See details in the documentation.
• Three optimization algorithms: Value iteration, Policy iteration, and Modified policy iteration.
• Three value-update methods: Standard, Gauss–Seidel, Successive over-relaxation.
• Supports sparse matrices.
• Employs parallel computing.

A C++based solver for MDP problems

Initial pre-release of our C++based solver for Markov Decision Process optimization problems. The solver is based on a Modified Policy Iteration (MPI) algorithm, which derives an epsilon-optimal policy that maximizes the expected total discounted reward, where epsilon is a tolerance parameter given to the algorithm. We further provide the user with the option to choose between three different value update methods as well as switching to an epsilon-optimal Value Iteration or Policy Iteration algorithm. See the Readme-file for further information.