-
Demo of a Denoising Autoencoder (DAE) learning vector fields for one-dimensional manifolds.
-
Programmed for the seminar 'Concepts in Deep Learning' at the University of Osnabrück (2024/2025)
-
By the group DataSippers
In this demo, we will see how a denoising autoencoder will learn the vector field for three examples of one-dimensional manifolds: a line, a circle and a spiral.
But a circle is not one but two-dimensional! you may correctly say. That is where the concept of manifolds comes in. In mathematics, a manifold is a topological space that locally resembles euclidean space. For example, we all know the earth is a sphere (I hope) but if we stand in an open field, it looks like we are standing on a plane, a two-dimensional space.
For our circle let's imagine we are a little bug that is walking along the edge. From our point of view all we see is the point in front of us: a one-dimensional space.
Now, why is this interesting? According to the manifold hypothesis, "high dimensional data tends to lie in the vicinity of a low dimensional manifold" (Fefferman et al., 2016). That mean that our machine learning models would be able to fit "simpler" subspaces instead of having to fit high-dimensional input space and dimension reduction is always a win.
What does our denoising autoencoder (DAE) have to do with all of this? Let us remember how it looks like:
Our DAE has to learn how to map corrupted inputs
When putting in new inputs coordinates to the trained DAE, we see where the DAE maps them to -> closer to the manifold!
That means our DAE not only approximates the function of a circle, but also of the vector field around it.
- install the conda environment with
conda env create -f env.yml - Clone the Repository or Download the ZIP File and extract.
- Open the demo.ipynb by using your preferred code editor or jupyter
Fefferman, C., Mitter, S., & Narayanan, H. (2016). Testing the manifold hypothesis. Journal of the American Mathematical Society, 29(4), 983-1049. https://arxiv.org/pdf/1310.0425