README

This repository contains code for

Christian Fiedler, Carsten W. Scherer, Sebastian Trimpe, Practical and Rigorous Uncertainty Bounds for Gaussian Process Regression, AAAI21

Note that a version of this article with corrections is available at

https://arxiv.org/abs/2105.02796

The code in this repository has been updated accordingly.

Introduction

This directory contains the Python source code to reproduce all the experiments in the main text and the supplementary text.

Setup

All experiments were performed with Python 3.8 in a Miniconda environment running on Ubuntu 18.04.4 LTS. More details can be found in the supplementary text.

The file environment.yml contains the exact environment used for the experiments (the file was generated by running conda env export > environment.yml). In order to run also the Jupyter Notebooks, use extended_environment.yml instead.

Running the experiments

We assume now that a Python environment as described in environment.yml or extended_environment.yml is available. Furthermore, we assume that the files utilities.py and experiments.py are in the same directory as the scripts.

Experiments with synthetic data

All experiments with synthetic data were performed by running each of the scripts exp_1_1_a.py up to exp_1_4_c.py. Note that each script expects a directory called outputs in which the script's output is stored. The experimental settings can be easily adapted in the script files. In particular, the number of threads used by joblib can be adjusted by setting n_jobs, the number of functions generated for each of the experiments by n_rep_funcs, the number of learning instances by n_rep_training and the number of training samples in each run by n_samples.

Reproducing the figures and tables

• Figure 1, LEFT, in the main text was generated with the script illustrating_nominal.py Running it with python illustrating_nominal.py produced the graphic as file figure_example_nominal.pdf in the same directory.
• Figure 1, RIGHT, in the main text was generated with the script illustrating_misspec.py Running it with python illustrating_nominal.py produced the graphic as file figure_example_misspec.pdf in the same directory.
• Table 1 in the main text summerises the results of the scripts exp_1_1_a.py and exp_1_1_b.py. Data has been analysed with the Jupyter notebook AnalysingPlottingExperiments.ipynb
• Table 2 in the main text summerises the results of the scripts exp_1_4_a.py and exp_1_4_b.py. Data has been analysed with the Jupyter notebook AnalysingPlottingExperiments.ipynb
• Table 3 in the main text summerises the results of the script exp_1_4_c.py. Data has been analysed with the Jupyter notebook AnalysingPlottingExperiments.ipynb