A One-Week University Course on Deep Learning using PyTorch

Rodrigo da Motta C. de Carvalho
Data Scientist | M.Sc. Researcher in Neuroscience | Content Creator

Introduction

A hot topic in Data Science is how to teach it. This repository presents the content of a 20-hour course on Deep Learning at USP - Universidade de São Paulo, the best university in Latin America. The course aimed to provide an introduction to Deep Learning, covering essential topics, and offering hands-on experience using PyTorch, the main framework in academia.

Course Goals

The course was designed for students with prior knowledge of Python programming, particularly with Numpy arrays, and a basic understanding of linear algebra and probability. The main topics covered were:

• Fundamentals
• Computer Vision
• Geometric Deep Learning
• Natural Language Processing
• Large Language Models

Each session lasted 4 hours, consisting of theoretical instruction (1h45), a coffee break (30 minutes), and hands-on activities (1h45).

Detailed Course Outline

The following is a breakdown of the macro-areas and sub-topics covered, along with the hands-on exercises associated with each:

Fundamentals

• Machine Learning basics, Multi-Layer Perceptron, Gradient Descent, Hyper-parameters, Representation learning.
• Hands-on: MLP Classification.

Computer Vision

• Image data structure, Convolutional Operations, Filters, Auto-Encoder, Latent Space, Fine Tuning.
• Hands-on: Implementing an Auto-Encoder.

Geometric Deep Learning

• Graph Basics, Message Passing Algorithm, Graph Convolutional Layer, Self-Attention, Node & Graph classification.
• Hands-on: Document classification.

Natural Language Processing (NLP)

• Background, Tokens, TF-IDF, Embeddings, Word2Vec, CBOW, Skip Gram, BERT.
• Hands-on: Sentiment Analysis.

Large Language Models (LLM)

• Tokens, Embeddings, RNN and LSTM, Self-Attention, Auto-encoder, Sentence transformers, LLMs Architecture, training LLMs.
• Hands-on: Vector Databases.

Acknowledgments

Special thanks to the Institute of Physics | University of São Paulo (IFUSP) for the infrastructure, the instructors for their time and effort, and the support team at HackerSpace IFUSP.

Course Material

• The course is in Portuguese, with subtitles available in other languages.
• Google Drive Link - Contains slides, hands-on exercises, and additional resources.

Repository Contents

This repository contains Jupyter Notebooks and other material related to the course:

• /notebooks: The Jupyter notebooks used in the hands-on sessions.
• /slides: Presentation slides for the theoretical part of the course.
• /resources: Additional references, datasets, and reading materials.