Team name: Ristretto Learning

The team:

• Martorella Tommaso
• Ramírez Contreras Héctor Manuel
• Cerezo García Daniel

EPFL Machine Learning course 2022

Project 2: Reproducibility Challenge 🚀️

Reproducing the paper: Numerical influence of ReLU’(0) on backpropagation.

This a project for the Machine Learning course at EPFL. The reproducibility challenge. We reproduce a part of the paper Numerical influence of ReLU’(0) on backpropagation.

The goals of this project are:

• Reproduce part of the paper to better understand state of the art problems in Machine Learning
• As there's already a well documented code, we will go beyond the stated results of the paper and try some other things to get better insights of the former research.
• We use different subgradient of ReLU'(0) to see how it can affect the training and test loss and accuracy.
• We did some a different architecture of the code to run experiments so that we could have more flexibility when trying new things on models and datasets. (next steps could be test with more regularizations, optimizers and data sets.)

Create a conda environment using the file environment.yml and simply run the desired notebook to reproduce the experiments.