ML at University Ramon Llull-2019
Agenda:
- Overview
- Introduction into ML, why it works
- Pizzeria example, feature and target space
- Linear Regression
- Numpy, Matplotlib, Pandas, Seaborn
- Weather data sample
- Kaggle
- House prices dataset (https://www.kaggle.com/c/house-prices-advanced-regression-techniques)
- EDA: exploratory data analysis (https://www.kaggle.com/pmarcelino/comprehensive-data-exploration-with-python)
- Linear regression example by hand (https://www.kaggle.com/tentotheminus9/linear-regression-from-scratch-gradient-descent)
- Linear regression example by SciKit-Learn
- Linear regression
- Scikit-learn
- Gradient descent
- Overfitting
- Logistic regression
- Figures of merit, ROC
- K-fold cross-validation
- K-neares neighbors
- Regularization
- https://www.kaggle.com/juliencs/a-study-on-regression-applied-to-the-ames-dataset
- Multilayer Perceptron
- Intro into PyTorch, https://pytorch.org/tutorials/beginner/blitz/tensor_tutorial.html, https://pytorch.org/get-started/locally/
- Computer vision
- Convolutional, pooling layers
- Dropout, Batch normalisation
- https://www.comet.ml/
- Inofrmation Bottleneck - https://github.com/ravidziv/IDNNs
- NN interpretation: heatmap, [simple one](https://towardsdatascience.com/understanding-convolutional-neural-networks-through-visualizations-in-pytorch-b5444de08b91
-
Topics:
- GridSearchCV (https://scikit-learn.org/stable/auto_examples/model_selection/plot_grid_search_digits.html)
- https://github.com/skorch-dev/skorch
- Bayesian optimisation https://github.com/scikit-optimize/scikit-optimize/blob/master/examples/bayesian-optimization.ipynb
- https://github.com/scikit-optimize/scikit-optimize/blob/master/examples/sklearn-gridsearchcv-replacement.ipynb
- comparison https://scikit-optimize.github.io/notebooks/strategy-comparison.html
- DARTS https://github.com/dragen1860/DARTS-PyTorch, https://github.com/quark0/darts
-
Exercises:
- finish day7 practice.ipynb
- scikit-optimize notebook (bayesian_optimization.ipynb)
- skorch: pytorch to scikit (skorch_FMNIST.ipynb)
- comet.ml optimisation (optimise_FMNIST.ipynb)
- Deep Learning Book (http://www.deeplearningbook.org)
- Recurrent neural network (https://towardsdatascience.com/recurrent-neural-networks-and-lstm-4b601dd822a5)
- ResNet https://medium.com/@14prakash/understanding-and-implementing-architectures-of-resnet-and-resnext-for-state-of-the-art-image-cf51669e1624
- Generative nets (https://towardsdatascience.com/understanding-generative-adversarial-networks-gans-cd6e4651a29)
- StyleTransfer https://pytorch.org/tutorials/advanced/neural_style_tutorial.html, https://medium.com/@purnasaigudikandula/artistic-neural-style-transfer-with-pytorch-1543e08cc38f
- Reinforcement Learning (https://towardsdatascience.com/introduction-to-various-reinforcement-learning-algorithms-i-q-learning-sarsa-dqn-ddpg-72a5e0cb6287)
- Geometrical Deep Learning (http://geometricdeeplearning.com)
- https://www.deeplearning.ai