!! Check the Project Report. Score: 8/10
The course aims at providing a solid introduction to machine learning, a branch of artificial intelligence that deals with the development of algorithms able to extract knowledge from data, with a focus on pattern recognition and classification problems.
The course will cover the basic concepts of statistical machine learning and will concentrate on the broad class of generative linear Gaussian models and discriminative classifiers based on logistic regression and support vector machines.
The objective of the course is to provide the students with solid theoretical bases that will allow them to select, apply and evaluate different classification methods on real tasks.
The course will also allow the students to acquire the required competencies to devise novel approaches based on the frameworks that will be presented during the classes.
Introduction
- Machine learning and pattern recognition
- Probability theory concepts
- Python: language, main numerical libraries
Decision Theory
- Inference and decisions
- Model taxonomy: generative and discriminative approaches
- Model optimization, hyperparameter selection, cross-validation
Model evaluation
- Classification scores and log-likelihood ratios
- Detection Cost Functions and optimal Bayes decisions
Dimensionality reduction
- Principal Component Analysis (PCA)
- Linear Discriminant Analysis (LDA)
Generative models
- Generative Gaussian classifiers: univariate Gaussian, Naive Bayes, multivariate Gaussian (MVG)
- Tied covariance MVG and LDA
- Categorical and Multinomial classifiers
Logistic Regression (LR)
- Tied MVG and LR
- LR as Maximum Likelihood solution for class labels
- Binary and multiclass cross-entropy
- LR as empirical risk minimization
- Overfitting and regularization
- MVG and Quadratic LR
Support Vector Machines (SVM)
- Optimal classification hyperplane: the maximum margin definition
- Soft margin and L2 regularization
- Primal and dual SVM formulation
- Non linear extension: brief introduction to kernels
Density estimation and latent variable models
- Gaussian mixture models (GMM)
- The Expectation Maximization (EM) algorithm