Skip to content

Personal reimplementation of some ML algorithms for learning purposes

License

Notifications You must be signed in to change notification settings

MaxenceGiraud/MachineLearningAlgos

Repository files navigation

MachineLearningAlgos

Personal reimplementation of some ML algorithm for learning purposes

Contribution/ Author

This repo is only made by myself (Maxence Giraud), I do not seek additional contributors (although you can open an issue if you find a mistake or have a constructive comment) as it has only learning purposes and no other intentions.

If you want to see the work to come and in progress, you can go to the project tab.

Requirements

The requirements can be installed through this simple command :

pip install -r requirements.txt

Some readme have written LaTeX, you can view them locally with some capable reader (e.g. VSCode) or using extensions (e.g. here for Firefox).

Installation

To install, simply clone the project (don't forget the requirements first) :

git clone https://github.com/MaxenceGiraud/MachineLearningAlgos
cd MachineLearningAlgos/

Usage

  • The machine learning algorithms are programmed in a similar fashion as sklearn.
  • The deep learning framework is conceived in a Keras-like manner.
  • The Reinforcement Learning take as input gym-like environments.
import mla
X,y =  load_your_data()

clf = QDA()
clf.fit(X,y)
clf.score(X,y)

##### Unsupevized learning
clust = mla.DBSCAN(eps=5)
clust.fit_predict(X)

##### Deep Learning
from mla import dl

nn = dl.NeuralNetwork(X.shape[1:],loss=dl.MSE())
nn.add(dl.Dense(10,activation=dl.activation.Relu()))
nn.add(dl.Dense(4,activation=dl.activation.Relu()))
nn.add(dl.Dense(1))

nn.fit(X,y)

##### Multi-armed bandits
from mla import mab 

arms = [mab.arms.Bernoulli(0.8),mab.arms.Exponential(2),mab.arms.Gaussian(2.4),mab.arms.Gaussian(1.5)]
bandit = mab.MAB(arms)

T = 100 # Number of trials
ucb = mab.UCB(bandit.nbArms)
ts = mab.ThompsonSampling(bandit.nbArms)
etc = mab.ETC(bandit.nbArms,T)

mab.bandit_env.RunExpes([ucb,ts,etc],bandit,N_exp=10,timeHorizon=T) # Compare the different algorithms

Other examples on how to use the algorithms can be found in the file example.py

To run unittest :

python -m unittest tests/*/*_test.py

Implemented Algorithms

Some explainations of the algorithms can be found in the readme of their folders, for more thorough reviews check the references.
Algorithms that I plan to implement are written in the projects tab.

Semi Supzevized Learning

Wrappers

All the following loss support weights (if given to fit method, it is done automatically)

Credits to Emilie Kaufmann for the bandit environment.

For non stationary bandits, see my other repo here.

A KernelFusion is also available that allows to combine kernels w.r.t. differents features of the data

  • MSE
  • RMSE
  • MAE
  • Median Absolute error
  • R2 score
  • Accuracy score
  • Zero one loss

References

The references of the algorithms themselves are for the most part written in the docstring of the corresponding class

[1] M. BISHOP, Christopher. Pattern Recognition and Machine Learning. Springer, 2011.
[2] TIBSHIRANI, Robert, HASTIE, Trevor y FRIEDMAN JEROME, . The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition. Springer, 2016.
[3] P. MURPHY, Kevin. Machine Learning: A Probabilistic Perspective. The MIT Press, 2012.
[4] Scikit-learn: Machine Learning in Python, Pedregosa et al., JMLR 12, pp. 2825-2830, 2011.
[5] Courses from Master Data Science held at the University of Lille, France