03_Autoencoder.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Welcome to your third assignment!\n",
    "\n",
    "Please submit your solution of this notebook in the Whiteboard at the corresponding Assignment entry. We need you to upload the .ipynb-file and the exported .pdf of this notebook.\n",
    "\n",
    "If you have any questions, ask them in either in the tutorials or in the \"Mattermost\" channel: https://mattermost.imp.fu-berlin.de/biorobotics/channels/ssl_ws_2324\n",
    "\n",
    "\n",
    "In this assignment, we want you to explore Autoencoders."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Slide Review\n",
    "\n",
    "[Google Form](https://forms.gle/ZgL1p9g1T9GgtAkVA) for the slide review. Please take a minute to scroll over the slides again and improve your lecture.\n",
    "\n",
    "Please make sure to only choose your top 5 slides per lecture!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PapagAI\n",
    "\n",
    "From the second week onwards we started the reflective study.\n",
    "Register on the [PapagAI website](https://www.papag.ai) and write your first reflection about your impressions and challenges in the context of the lectures and tutorials you had this and previous week. The size of reflection can be anywhere bigger than 100 words. You can check out this [YouTube video](https://www.youtube.com/watch?v=QdmZHocZQBk&ab_channel=FernandoRamosL%C3%B3pez) with instructions on how to register, create a reflection and get an ai feedback.\n",
    "\n",
    "Please note, that this task is an obligatory one for this course and make sure each of you does the reflection, not only one person per group."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Please state both names of your group members here:\n",
    "Authors: Jane and John Doe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Assignment 3: Autoencoder\n",
    "\n",
    "This week's lecture introduced Autoencoders (AE), Denoising Autoencoders (DAE), and Masked Autoencoders (MAE). We want you to implement solutions for each approach in the following exercises. They will all use the MNIST images dataset, like the previous assignments. Here are the paper-links for you to read up on them:\n",
    "\n",
    "AE - [Paper](https://www.cs.toronto.edu/~hinton/absps/science.pdf) <br>\n",
    "DAE - [Paper](https://www.cs.toronto.edu/~larocheh/publications/icml-2008-denoising-autoencoders.pdf) <br>\n",
    "MAE - [Paper](https://arxiv.org/pdf/2111.06377.pdf) <br>\n",
    "\n",
    "\n",
    "## Ex. 3.1 Autoencoder for Compression\n",
    "\n",
    "Build an Autoencoder with a compressed latent space. You should use Convolution for downscaling and Transposed Convolution for upscaling. **(RESULT)**\n",
    "\n",
    "Train your model on MNIST images and compare at least 5 original and reconstructed images visually. **(RESULT)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ex. 3.2 Denoising AE\n",
    "\n",
    "As a second exercise, we want you to build a denoising autoencoder (DAE). The DAE may have more latent dimensions than input dimensions. The training on noisy inputs prevents it from learning the identity function. Therefore, using a bigger latent space is possible, too. Make sure to use the original inputs for computing the loss.\n",
    "\n",
    "- Build a DAE with more hidden dimensions than input dimensions. **(RESULT)** Usually a higher channel count provides a bigger latent space, while the spatial dimentsion are reduced.\n",
    "\n",
    "- Train it on MNIST samples that have two different flavours of noise applied to them. Show at least 5 denoised samples with your model and compare them visually to the respective input  **(RESULT)**\n",
    "\n",
    "- Compare two different loss functions and their effect on the model's reconstruction performance. **(RESULT)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ex. 3.3 Masked AE (BONUS)\n",
    "\n",
    "Implement a Masked Autoencoder (MAE) model for image data, i.e. the MNIST data. Can you build representations using fully connected or convolutional layers? You don't have to implement a Transformer as an Encoder or Decoder for this exercise. **(RESULT)**\n",
    "\n",
    "Make sure to utilize image-patching. The masking token can be fixed and does not have to be learnable by the Decoder.\n",
    "\n",
    "Check the performance of your Autoencoder on a the finetuning classification task on the MNIST test dataset. Report on the accuracy using your learned representations. **(RESULT)**\n",
    "\n",
    "Bonus question: What can you do to account for information leekage in the MAE that uses Convolutional Layers?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code here"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "uptodate",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}