diff --git a/README.md b/README.md index e0ca2cc..c533b51 100755 --- a/README.md +++ b/README.md @@ -29,7 +29,116 @@ print("Lambda =", model.best_lambda_) You should always try to give normalized data to LassoNet as it uses neural networks under the hood. -You can read the full [documentation](https://lasso-net.github.io//lassonet/api/) or read the [examples](https://github.com/lasso-net/lassonet/tree/master/examples) that cover most features. +You can read the full [documentation](https://lasso-net.github.io//lassonet/api/) or read the [examples](https://github.com/lasso-net/lassonet/tree/master/examples) that cover most features. We also provide a Quickstart section below. + + + +## Quickstart + +Here we guide you through the features of LassoNet and how you typically use them. + +### Task + +LassoNet is based on neural networks and can be used for any kind of data. Currently, we have implemented losses for the following tasks: + +- regression: `LassoNetRegressor` +- classification: `LassoNetClassifier` +- Cox regression: `LassoNetCoxRegressor` +- interval-censored Cox regression: `LassoNetIntervalRegressor` + +If features naturally belong to groups, you can use the `groups` parameter to specify them. This will allow the model to put a penalty on groups of features instead of each feature individually. + +### Data preparation + +You should always normalize your data before passing it to the model to avoid too large (or too small) values in the data. + +### What do you want to do? + +The LassoNet family of models do a lot of things. + +Here are some examples of what you can do with LassoNet. Note that you can switch `LassoNetRegressor` with any of the other models to perform the same operations. + +#### Using the base interface + +The base interface implements a `.fit()` method that is not very useful as it computes a path but does not store any intermediate result. + +Usually, you want to store the intermediate results (with `return_state_dicts=True`) and then load one of the models from the path into the model to inspect it. + +```python +from lassonet import LassoNetRegressor, plot_path + +model = LassoNetRegressor() +path = model.path(X_train, y_train, return_state_dicts=True) +plot_path(model, X_test, y_test) + +# choose `best_id` based on the plot +model.load(path[best_id].state_dict) +print(model.score(X_test, y_test)) +``` + +You can also retrieve the mask of the selected features and train a dense model on the selected features. + +```python +selected = path[best_id].selected +model.fit(X_train[:, selected], y_train, dense_only=True) +print(model.score(X_test[:, selected], y_test)) +``` + +You get a `model.feature_importances_` attribute that is the value of the L1 regularization parameter at which each feature is removed. This can give you an idea of the most important features but is very unstable across different runs. You should use stability selection to select the most stable features. + +#### Using the cross-validation interface + +The cross-validation interface computes validation scores on multiple folds before running a final path on the whole training dataset with the best regularization parameter. + +```python +model = LassoNetRegressorCV() +model.fit(X_train, y_train) +model.score(X_test, y_test) +``` + +You can also use the `plot_cv` method to get more information. + +Some attributes give you more information about the best model, like `best_lambda_`, `best_selected_` or `best_cv_score_`. + +This information is useful to pass to a base model to train it from scratch with the best regularization parameter or the best subset of features. + +#### Using the stability selection interface + + +[Stability selection](https://arxiv.org/abs/0809.2932) is a method to select the most stable features when running the model multiple times on different random subsamples of the data. It is probably the best way to select the most important features. + +```python +model = LassoNetRegressor() +oracle, order, wrong, paths, prob = model.stability_selection(X_train, y_train) +``` + +- `oracle` is a heuristic that can detect the most stable features when introducing noise. +- `order` sorts the features by their decreasing importance. +- `wrong[k]` is a measure of error when selecting the k+1 first features (read [this paper](https://arxiv.org/pdf/2206.06885) for more details). You can `plt.plot(wrong)` to see the error as a function of the number of selected features. +- `paths` stores all the computed paths. +- `prob` is the probability that a feature is selected at each value of the regularization parameter. + +In practice, you might want to train multiple dense models on different subsets of features to get a better understanding of the importance of each feature. + +For example: + +```python +for i in range(10): + selected = order[:i] + model.fit(X_train[:, selected], y_train, dense_only=True) + print(model.score(X_test[:, selected], y_test)) +``` + +### Important parameters + +Here are the most important parameters you should be aware of: + +- `hidden_dims`: the number of neurons in each hidden layer. The default value is `(100,)` but you might want to try smaller and deeper networks like `(10, 10)`. +- `path_multiplier`: the number of lambda values to compute on the path. The lower it is, the more precise the model is but the more time it takes. The default value is a trade-off to get a fast training but you might want to try smaller values like `1.01` or `1.005` to get a better model. +- `lambda_start`: the starting value of the regularization parameter. The default value is `"auto"` and the model will try to select a good starting value according to an unpublised heuristic (read the code to know more). You can identify a bad `lambda_start` by plotting the path. If `lambda_start` is too small, the model will stay dense for a long time, which does not affect performance but takes longer. If `lambda_start` is too large, the number of features with decrease very fast and the path will not be accurate. In that case you might also want to decrease `lambda_start`. +- `gamma`: puts some L2 penalty on the network. The default is `0.0` which means no penalty but some small value can improve the performance, especially on small datasets. +- more standard MLP training parameters are accessible: `dropout`, `batch_size`, `optim`, `n_iters`, `patience`, `tol`, `backtrack`, `val_size`. In particular, `batch_size` can be useful to do stochastic gradient descent instead of full batch gradient descent and to avoid memory issues on large datasets. +- `M`: this parameter has almost no effect on the model. ## Features diff --git a/docs/api/index.html b/docs/api/index.html index 6ed4dc8..142d27b 100644 --- a/docs/api/index.html +++ b/docs/api/index.html @@ -1490,16 +1490,16 @@

API

-lassonet.plot_path(model, path, X_test, y_test, *, score_function=None)
+lassonet.plot_path(model, X_test, y_test, *, score_function=None)

Plot the evolution of the model on the path, namely: - lambda - number of selected variables - score

+

Requires to have called model.path(return_state_dicts=True) beforehand.

Parameters:
  • model (LassoNetClassifier or LassoNetRegressor)

  • -
  • path – output of model.path

  • X_test (array-like)

  • y_test (array-like)

  • score_function (function or None) – if None, use score_function=model.score diff --git a/docs/api/searchindex.js b/docs/api/searchindex.js index 23ed79f..1c5f596 100644 --- a/docs/api/searchindex.js +++ b/docs/api/searchindex.js @@ -1 +1 @@ -Search.setIndex({"alltitles": {"API": [[0, "api"]], "Installation": [[0, "installation"]], "Welcome to LassoNet\u2019s documentation!": [[0, null]]}, "docnames": ["index"], "envversion": {"sphinx": 63, "sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2}, "filenames": ["index.rst"], "indexentries": {"fit() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.fit", false]], "fit() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.fit", false]], "fit() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.fit", false]], "fit() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.fit", false]], "fit() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.fit", false]], "fit() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.fit", false]], "get_metadata_routing() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.get_metadata_routing", false]], "get_metadata_routing() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.get_metadata_routing", false]], "get_metadata_routing() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.get_metadata_routing", false]], "get_metadata_routing() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.get_metadata_routing", false]], "get_metadata_routing() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.get_metadata_routing", false]], "get_metadata_routing() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.get_metadata_routing", false]], "get_params() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.get_params", false]], "get_params() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.get_params", false]], "get_params() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.get_params", false]], "get_params() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.get_params", false]], "get_params() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.get_params", false]], "get_params() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.get_params", false]], "lassonet_path() (in module lassonet)": [[0, "lassonet.lassonet_path", false]], "lassonetclassifier (class in lassonet)": [[0, "lassonet.LassoNetClassifier", false]], "lassonetclassifiercv (class in lassonet)": [[0, "lassonet.LassoNetClassifierCV", false]], "lassonetcoxregressor (class in lassonet)": [[0, "lassonet.LassoNetCoxRegressor", false]], "lassonetcoxregressorcv (class in lassonet)": [[0, "lassonet.LassoNetCoxRegressorCV", false]], "lassonetregressor (class in lassonet)": [[0, "lassonet.LassoNetRegressor", false]], "lassonetregressorcv (class in lassonet)": [[0, "lassonet.LassoNetRegressorCV", false]], "path() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.path", false]], "path() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.path", false]], "path() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.path", false]], "path() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.path", false]], "path() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.path", false]], "path() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.path", false]], "plot_path() (in module lassonet)": [[0, "lassonet.plot_path", false]], "score() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.score", false]], "score() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.score", false]], "score() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.score", false]], "score() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.score", false]], "score() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.score", false]], "score() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.score", false]], "set_fit_request() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.set_fit_request", false]], "set_fit_request() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.set_fit_request", false]], "set_fit_request() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.set_fit_request", false]], "set_fit_request() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.set_fit_request", false]], "set_fit_request() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.set_fit_request", false]], "set_fit_request() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.set_fit_request", false]], "set_params() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.set_params", false]], "set_params() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.set_params", false]], "set_params() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.set_params", false]], "set_params() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.set_params", false]], "set_params() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.set_params", false]], "set_params() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.set_params", false]], "set_score_request() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.set_score_request", false]], "set_score_request() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.set_score_request", false]], "set_score_request() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.set_score_request", false]], "set_score_request() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.set_score_request", false]], "set_score_request() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.set_score_request", false]], "set_score_request() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.set_score_request", false]], "stability_selection() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.stability_selection", false]], "stability_selection() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.stability_selection", false]], "stability_selection() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.stability_selection", false]], "stability_selection() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.stability_selection", false]], "stability_selection() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.stability_selection", false]], "stability_selection() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.stability_selection", false]]}, "objects": {"lassonet": [[0, 0, 1, "", "LassoNetClassifier"], [0, 0, 1, "", "LassoNetClassifierCV"], [0, 0, 1, "", "LassoNetCoxRegressor"], [0, 0, 1, "", "LassoNetCoxRegressorCV"], [0, 0, 1, "", "LassoNetRegressor"], [0, 0, 1, "", "LassoNetRegressorCV"], [0, 2, 1, "", "lassonet_path"], [0, 2, 1, "", "plot_path"]], "lassonet.LassoNetClassifier": [[0, 1, 1, "", "fit"], [0, 1, 1, "", "get_metadata_routing"], [0, 1, 1, "", "get_params"], [0, 1, 1, "", "path"], [0, 1, 1, "", "score"], [0, 1, 1, "", "set_fit_request"], [0, 1, 1, "", "set_params"], [0, 1, 1, "", "set_score_request"], [0, 1, 1, "", "stability_selection"]], "lassonet.LassoNetClassifierCV": [[0, 1, 1, "", "fit"], [0, 1, 1, "", "get_metadata_routing"], [0, 1, 1, "", "get_params"], [0, 1, 1, "", "path"], [0, 1, 1, "", "score"], [0, 1, 1, "", "set_fit_request"], [0, 1, 1, "", "set_params"], [0, 1, 1, "", "set_score_request"], [0, 1, 1, "", "stability_selection"]], "lassonet.LassoNetCoxRegressor": [[0, 1, 1, "", "fit"], [0, 1, 1, "", "get_metadata_routing"], [0, 1, 1, "", "get_params"], [0, 1, 1, "", "path"], [0, 1, 1, "", "score"], [0, 1, 1, "", "set_fit_request"], [0, 1, 1, "", "set_params"], [0, 1, 1, "", "set_score_request"], [0, 1, 1, "", "stability_selection"]], "lassonet.LassoNetCoxRegressorCV": [[0, 1, 1, "", "fit"], [0, 1, 1, "", "get_metadata_routing"], [0, 1, 1, "", "get_params"], [0, 1, 1, "", "path"], [0, 1, 1, "", "score"], [0, 1, 1, "", "set_fit_request"], [0, 1, 1, "", "set_params"], [0, 1, 1, "", "set_score_request"], [0, 1, 1, "", "stability_selection"]], "lassonet.LassoNetRegressor": [[0, 1, 1, "", "fit"], [0, 1, 1, "", "get_metadata_routing"], [0, 1, 1, "", "get_params"], [0, 1, 1, "", "path"], [0, 1, 1, "", "score"], [0, 1, 1, "", "set_fit_request"], [0, 1, 1, "", "set_params"], [0, 1, 1, "", "set_score_request"], [0, 1, 1, "", "stability_selection"]], "lassonet.LassoNetRegressorCV": [[0, 1, 1, "", "fit"], [0, 1, 1, "", "get_metadata_routing"], [0, 1, 1, "", "get_params"], [0, 1, 1, "", "path"], [0, 1, 1, "", "score"], [0, 1, 1, "", "set_fit_request"], [0, 1, 1, "", "set_params"], [0, 1, 1, "", "set_score_request"], [0, 1, 1, "", "stability_selection"]]}, "objnames": {"0": ["py", "class", "Python class"], "1": ["py", "method", "Python method"], "2": ["py", "function", "Python function"]}, "objtypes": {"0": "py:class", "1": "py:method", "2": "py:function"}, "terms": {"0": 0, "02": 0, "1": 0, "10": 0, "100": 0, "1000": 0, "1e": 0, "2": 0, "20": 0, "23": 0, "3": 0, "5": 0, "9": 0, "99": 0, "A": 0, "For": 0, "If": 0, "In": 0, "The": 0, "There": 0, "To": 0, "__": 0, "accord": 0, "accuraci": 0, "ad": 0, "adam": 0, "alia": 0, "all": 0, "allow": 0, "alwai": 0, "an": 0, "ani": 0, "anymor": 0, "approxim": 0, "ar": 0, "arbitrarili": 0, "arrai": 0, "auto": 0, "automat": 0, "avail": 0, "backtrack": 0, "baselassonet": 0, "batch": 0, "batch_siz": 0, "becaus": 0, "being": 0, "best": 0, "bool": 0, "bound": 0, "breslow": 0, "call": 0, "callback": 0, "can": 0, "chang": 0, "check": 0, "check_cv": 0, "class": 0, "class_weight": 0, "classif": 0, "classifi": 0, "coeffici": 0, "compon": 0, "comput": 0, "concord": 0, "connect": 0, "consist": 0, "constant": 0, "contain": 0, "correctli": 0, "cox": 0, "cpu": 0, "cross": 0, "cv": 0, "data": 0, "decreas": 0, "deep": 0, "default": 0, "defin": 0, "dens": 0, "dense_onli": 0, "determin": 0, "devic": 0, "dict": 0, "differ": 0, "dimens": 0, "disabl": 0, "disable_lambda_warn": 0, "disregard": 0, "doe": 0, "dropout": 0, "durat": 0, "dure": 0, "e": 0, "each": 0, "earli": 0, "effect": 0, "efron": 0, "els": 0, "enable_metadata_rout": 0, "encapsul": 0, "ensur": 0, "epoch": 0, "epsilon": 0, "error": 0, "estim": 0, "event": 0, "evolut": 0, "except": 0, "exist": 0, "expect": 0, "factor": 0, "fals": 0, "featur": 0, "first": 0, "fit": 0, "float": 0, "fold": 0, "form": 0, "frac": 0, "from": 0, "function": 0, "g": 0, "gamma": 0, "gamma_skip": 0, "gener": 0, "get": 0, "get_metadata_rout": 0, "get_param": 0, "given": 0, "go": 0, "gpu": 0, "group": 0, "guid": 0, "ha": 0, "harsh": 0, "have": 0, "hidden": 0, "hidden_dim": 0, "hierarchi": 0, "histori": 0, "historyitem": 0, "how": 0, "html": 0, "http": 0, "i": 0, "ideal": 0, "ignor": 0, "improv": 0, "increas": 0, "increment": 0, "index": 0, "indic": 0, "inf": 0, "influenc": 0, "inform": 0, "initi": 0, "input": 0, "insid": 0, "instanc": 0, "instead": 0, "int": 0, "iter": 0, "keep": 0, "kernel": 0, "kwarg": 0, "l2": 0, "label": 0, "lambda": 0, "lambda_": 0, "lambda_max": 0, "lambda_seq": 0, "lambda_start": 0, "lassonet_path": 0, "lassonetclassifi": 0, "lassonetclassifiercv": 0, "lassonetcoxregressor": 0, "lassonetcoxregressorcv": 0, "lassonetregressor": 0, "lassonetregressorcv": 0, "latter": 0, "layer": 0, "learn": 0, "leav": 0, "like": 0, "list": 0, "longtensor": 0, "lr": 0, "m": 0, "mai": 0, "main": 0, "map": 0, "matrix": 0, "maximum": 0, "mean": 0, "mechan": 0, "meta": 0, "metadata": 0, "metadata_rout": 0, "metadatarequest": 0, "method": 0, "metric": 0, "minimum": 0, "model": 0, "model_select": 0, "modul": 0, "momentum": 0, "most": 0, "multi": 0, "multioutput": 0, "multioutputregressor": 0, "multipl": 0, "must": 0, "n_featur": 0, "n_iter": 0, "n_model": 0, "n_output": 0, "n_sampl": 0, "n_samples_fit": 0, "n_step": 0, "name": 0, "neg": 0, "nest": 0, "network": 0, "new": 0, "none": 0, "note": 0, "number": 0, "object": 0, "old": 0, "one": 0, "onli": 0, "optim": 0, "option": 0, "oracl": 0, "order": 0, "org": 0, "origin": 0, "other": 0, "otherwis": 0, "output": 0, "over": 0, "pair": 0, "param": 0, "paramet": 0, "pass": 0, "path": 0, "path_multipli": 0, "patienc": 0, "penal": 0, "penalti": 0, "per": 0, "pip": 0, "pipelin": 0, "pleas": 0, "plot": 0, "plot_path": 0, "possibl": 0, "precomput": 0, "predict": 0, "prob": 0, "probabl": 0, "proport": 0, "provid": 0, "pytorch": 0, "r": 0, "r2_score": 0, "rais": 0, "random": 0, "random_st": 0, "regress": 0, "regressor": 0, "regular": 0, "relev": 0, "request": 0, "requir": 0, "residu": 0, "retain": 0, "return": 0, "return_state_dict": 0, "rout": 0, "sampl": 0, "sample_weight": 0, "scikit": 0, "score": 0, "score_funct": 0, "see": 0, "select": 0, "selection_prob": 0, "self": 0, "sequenc": 0, "set": 0, "set_config": 0, "set_fit_request": 0, "set_param": 0, "set_score_request": 0, "sgd": 0, "shape": 0, "should": 0, "shuffl": 0, "simpl": 0, "sinc": 0, "skip": 0, "sklearn": 0, "so": 0, "some": 0, "specifi": 0, "split": 0, "squar": 0, "stabil": 0, "stability_select": 0, "stabl": 0, "start": 0, "state": 0, "step": 0, "stop": 0, "str": 0, "strategi": 0, "sub": 0, "subobject": 0, "subset": 0, "sum": 0, "t": 0, "take": 0, "target": 0, "task": 0, "tensor": 0, "test": 0, "th": 0, "thi": 0, "tie": 0, "tie_approxim": 0, "tol": 0, "torch": 0, "torch_se": 0, "total": 0, "train": 0, "true": 0, "tupl": 0, "two": 0, "type": 0, "u": 0, "unchang": 0, "uniform_averag": 0, "until": 0, "updat": 0, "upper": 0, "us": 0, "user": 0, "util": 0, "v": 0, "val_siz": 0, "valid": 0, "valu": 0, "variabl": 0, "verbos": 0, "version": 0, "w": 0, "wait": 0, "websit": 0, "weight": 0, "well": 0, "when": 0, "where": 0, "which": 0, "without": 0, "work": 0, "wors": 0, "would": 0, "wrong": 0, "x": 0, "x_test": 0, "x_val": 0, "y": 0, "y_pred": 0, "y_test": 0, "y_true": 0, "y_val": 0, "you": 0, "zero": 0}, "titles": ["Welcome to LassoNet\u2019s documentation!"], "titleterms": {"": 0, "api": 0, "document": 0, "instal": 0, "lassonet": 0, "welcom": 0}}) \ No newline at end of file +Search.setIndex({"alltitles": {"API": [[0, "api"]], "Installation": [[0, "installation"]], "Welcome to LassoNet\u2019s documentation!": [[0, null]]}, "docnames": ["index"], "envversion": {"sphinx": 63, "sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2}, "filenames": ["index.rst"], "indexentries": {"fit() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.fit", false]], "fit() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.fit", false]], "fit() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.fit", false]], "fit() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.fit", false]], "fit() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.fit", false]], "fit() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.fit", false]], "get_metadata_routing() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.get_metadata_routing", false]], "get_metadata_routing() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.get_metadata_routing", false]], "get_metadata_routing() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.get_metadata_routing", false]], "get_metadata_routing() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.get_metadata_routing", false]], "get_metadata_routing() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.get_metadata_routing", false]], "get_metadata_routing() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.get_metadata_routing", false]], "get_params() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.get_params", false]], "get_params() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.get_params", false]], "get_params() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.get_params", false]], "get_params() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.get_params", false]], "get_params() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.get_params", false]], "get_params() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.get_params", false]], "lassonet_path() (in module lassonet)": [[0, "lassonet.lassonet_path", false]], "lassonetclassifier (class in lassonet)": [[0, "lassonet.LassoNetClassifier", false]], "lassonetclassifiercv (class in lassonet)": [[0, "lassonet.LassoNetClassifierCV", false]], "lassonetcoxregressor (class in lassonet)": [[0, "lassonet.LassoNetCoxRegressor", false]], "lassonetcoxregressorcv (class in lassonet)": [[0, "lassonet.LassoNetCoxRegressorCV", false]], "lassonetregressor (class in lassonet)": [[0, "lassonet.LassoNetRegressor", false]], "lassonetregressorcv (class in lassonet)": [[0, "lassonet.LassoNetRegressorCV", false]], "path() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.path", false]], "path() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.path", false]], "path() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.path", false]], "path() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.path", false]], "path() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.path", false]], "path() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.path", false]], "plot_path() (in module lassonet)": [[0, "lassonet.plot_path", false]], "score() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.score", false]], "score() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.score", false]], "score() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.score", false]], "score() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.score", false]], "score() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.score", false]], "score() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.score", false]], "set_fit_request() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.set_fit_request", false]], "set_fit_request() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.set_fit_request", false]], "set_fit_request() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.set_fit_request", false]], "set_fit_request() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.set_fit_request", false]], "set_fit_request() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.set_fit_request", false]], "set_fit_request() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.set_fit_request", false]], "set_params() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.set_params", false]], "set_params() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.set_params", false]], "set_params() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.set_params", false]], "set_params() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.set_params", false]], "set_params() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.set_params", false]], "set_params() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.set_params", false]], "set_score_request() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.set_score_request", false]], "set_score_request() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.set_score_request", false]], "set_score_request() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.set_score_request", false]], "set_score_request() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.set_score_request", false]], "set_score_request() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.set_score_request", false]], "set_score_request() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.set_score_request", false]], "stability_selection() (lassonet.lassonetclassifier method)": [[0, "lassonet.LassoNetClassifier.stability_selection", false]], "stability_selection() (lassonet.lassonetclassifiercv method)": [[0, "lassonet.LassoNetClassifierCV.stability_selection", false]], "stability_selection() (lassonet.lassonetcoxregressor method)": [[0, "lassonet.LassoNetCoxRegressor.stability_selection", false]], "stability_selection() (lassonet.lassonetcoxregressorcv method)": [[0, "lassonet.LassoNetCoxRegressorCV.stability_selection", false]], "stability_selection() (lassonet.lassonetregressor method)": [[0, "lassonet.LassoNetRegressor.stability_selection", false]], "stability_selection() (lassonet.lassonetregressorcv method)": [[0, "lassonet.LassoNetRegressorCV.stability_selection", false]]}, "objects": {"lassonet": [[0, 0, 1, "", "LassoNetClassifier"], [0, 0, 1, "", "LassoNetClassifierCV"], [0, 0, 1, "", "LassoNetCoxRegressor"], [0, 0, 1, "", "LassoNetCoxRegressorCV"], [0, 0, 1, "", "LassoNetRegressor"], [0, 0, 1, "", "LassoNetRegressorCV"], [0, 2, 1, "", "lassonet_path"], [0, 2, 1, "", "plot_path"]], "lassonet.LassoNetClassifier": [[0, 1, 1, "", "fit"], [0, 1, 1, "", "get_metadata_routing"], [0, 1, 1, "", "get_params"], [0, 1, 1, "", "path"], [0, 1, 1, "", "score"], [0, 1, 1, "", "set_fit_request"], [0, 1, 1, "", "set_params"], [0, 1, 1, "", "set_score_request"], [0, 1, 1, "", "stability_selection"]], "lassonet.LassoNetClassifierCV": [[0, 1, 1, "", "fit"], [0, 1, 1, "", "get_metadata_routing"], [0, 1, 1, "", "get_params"], [0, 1, 1, "", "path"], [0, 1, 1, "", "score"], [0, 1, 1, "", "set_fit_request"], [0, 1, 1, "", "set_params"], [0, 1, 1, "", "set_score_request"], [0, 1, 1, "", "stability_selection"]], "lassonet.LassoNetCoxRegressor": [[0, 1, 1, "", "fit"], [0, 1, 1, "", "get_metadata_routing"], [0, 1, 1, "", "get_params"], [0, 1, 1, "", "path"], [0, 1, 1, "", "score"], [0, 1, 1, "", "set_fit_request"], [0, 1, 1, "", "set_params"], [0, 1, 1, "", "set_score_request"], [0, 1, 1, "", "stability_selection"]], "lassonet.LassoNetCoxRegressorCV": [[0, 1, 1, "", "fit"], [0, 1, 1, "", "get_metadata_routing"], [0, 1, 1, "", "get_params"], [0, 1, 1, "", "path"], [0, 1, 1, "", "score"], [0, 1, 1, "", "set_fit_request"], [0, 1, 1, "", "set_params"], [0, 1, 1, "", "set_score_request"], [0, 1, 1, "", "stability_selection"]], "lassonet.LassoNetRegressor": [[0, 1, 1, "", "fit"], [0, 1, 1, "", "get_metadata_routing"], [0, 1, 1, "", "get_params"], [0, 1, 1, "", "path"], [0, 1, 1, "", "score"], [0, 1, 1, "", "set_fit_request"], [0, 1, 1, "", "set_params"], [0, 1, 1, "", "set_score_request"], [0, 1, 1, "", "stability_selection"]], "lassonet.LassoNetRegressorCV": [[0, 1, 1, "", "fit"], [0, 1, 1, "", "get_metadata_routing"], [0, 1, 1, "", "get_params"], [0, 1, 1, "", "path"], [0, 1, 1, "", "score"], [0, 1, 1, "", "set_fit_request"], [0, 1, 1, "", "set_params"], [0, 1, 1, "", "set_score_request"], [0, 1, 1, "", "stability_selection"]]}, "objnames": {"0": ["py", "class", "Python class"], "1": ["py", "method", "Python method"], "2": ["py", "function", "Python function"]}, "objtypes": {"0": "py:class", "1": "py:method", "2": "py:function"}, "terms": {"0": 0, "02": 0, "1": 0, "10": 0, "100": 0, "1000": 0, "1e": 0, "2": 0, "20": 0, "23": 0, "3": 0, "5": 0, "9": 0, "99": 0, "A": 0, "For": 0, "If": 0, "In": 0, "The": 0, "There": 0, "To": 0, "__": 0, "accord": 0, "accuraci": 0, "ad": 0, "adam": 0, "alia": 0, "all": 0, "allow": 0, "alwai": 0, "an": 0, "ani": 0, "anymor": 0, "approxim": 0, "ar": 0, "arbitrarili": 0, "arrai": 0, "auto": 0, "automat": 0, "avail": 0, "backtrack": 0, "baselassonet": 0, "batch": 0, "batch_siz": 0, "becaus": 0, "beforehand": 0, "being": 0, "best": 0, "bool": 0, "bound": 0, "breslow": 0, "call": 0, "callback": 0, "can": 0, "chang": 0, "check": 0, "check_cv": 0, "class": 0, "class_weight": 0, "classif": 0, "classifi": 0, "coeffici": 0, "compon": 0, "comput": 0, "concord": 0, "connect": 0, "consist": 0, "constant": 0, "contain": 0, "correctli": 0, "cox": 0, "cpu": 0, "cross": 0, "cv": 0, "data": 0, "decreas": 0, "deep": 0, "default": 0, "defin": 0, "dens": 0, "dense_onli": 0, "determin": 0, "devic": 0, "dict": 0, "differ": 0, "dimens": 0, "disabl": 0, "disable_lambda_warn": 0, "disregard": 0, "doe": 0, "dropout": 0, "durat": 0, "dure": 0, "e": 0, "each": 0, "earli": 0, "effect": 0, "efron": 0, "els": 0, "enable_metadata_rout": 0, "encapsul": 0, "ensur": 0, "epoch": 0, "epsilon": 0, "error": 0, "estim": 0, "event": 0, "evolut": 0, "except": 0, "exist": 0, "expect": 0, "factor": 0, "fals": 0, "featur": 0, "first": 0, "fit": 0, "float": 0, "fold": 0, "form": 0, "frac": 0, "from": 0, "function": 0, "g": 0, "gamma": 0, "gamma_skip": 0, "gener": 0, "get": 0, "get_metadata_rout": 0, "get_param": 0, "given": 0, "go": 0, "gpu": 0, "group": 0, "guid": 0, "ha": 0, "harsh": 0, "have": 0, "hidden": 0, "hidden_dim": 0, "hierarchi": 0, "histori": 0, "historyitem": 0, "how": 0, "html": 0, "http": 0, "i": 0, "ideal": 0, "ignor": 0, "improv": 0, "increas": 0, "increment": 0, "index": 0, "indic": 0, "inf": 0, "influenc": 0, "inform": 0, "initi": 0, "input": 0, "insid": 0, "instanc": 0, "instead": 0, "int": 0, "iter": 0, "keep": 0, "kernel": 0, "kwarg": 0, "l2": 0, "label": 0, "lambda": 0, "lambda_": 0, "lambda_max": 0, "lambda_seq": 0, "lambda_start": 0, "lassonet_path": 0, "lassonetclassifi": 0, "lassonetclassifiercv": 0, "lassonetcoxregressor": 0, "lassonetcoxregressorcv": 0, "lassonetregressor": 0, "lassonetregressorcv": 0, "latter": 0, "layer": 0, "learn": 0, "leav": 0, "like": 0, "list": 0, "longtensor": 0, "lr": 0, "m": 0, "mai": 0, "main": 0, "map": 0, "matrix": 0, "maximum": 0, "mean": 0, "mechan": 0, "meta": 0, "metadata": 0, "metadata_rout": 0, "metadatarequest": 0, "method": 0, "metric": 0, "minimum": 0, "model": 0, "model_select": 0, "modul": 0, "momentum": 0, "most": 0, "multi": 0, "multioutput": 0, "multioutputregressor": 0, "multipl": 0, "must": 0, "n_featur": 0, "n_iter": 0, "n_model": 0, "n_output": 0, "n_sampl": 0, "n_samples_fit": 0, "n_step": 0, "name": 0, "neg": 0, "nest": 0, "network": 0, "new": 0, "none": 0, "note": 0, "number": 0, "object": 0, "old": 0, "one": 0, "onli": 0, "optim": 0, "option": 0, "oracl": 0, "order": 0, "org": 0, "origin": 0, "other": 0, "otherwis": 0, "output": [], "over": 0, "pair": 0, "param": 0, "paramet": 0, "pass": 0, "path": 0, "path_multipli": 0, "patienc": 0, "penal": 0, "penalti": 0, "per": 0, "pip": 0, "pipelin": 0, "pleas": 0, "plot": 0, "plot_path": 0, "possibl": 0, "precomput": 0, "predict": 0, "prob": 0, "probabl": 0, "proport": 0, "provid": 0, "pytorch": 0, "r": 0, "r2_score": 0, "rais": 0, "random": 0, "random_st": 0, "regress": 0, "regressor": 0, "regular": 0, "relev": 0, "request": 0, "requir": 0, "residu": 0, "retain": 0, "return": 0, "return_state_dict": 0, "rout": 0, "sampl": 0, "sample_weight": 0, "scikit": 0, "score": 0, "score_funct": 0, "see": 0, "select": 0, "selection_prob": 0, "self": 0, "sequenc": 0, "set": 0, "set_config": 0, "set_fit_request": 0, "set_param": 0, "set_score_request": 0, "sgd": 0, "shape": 0, "should": 0, "shuffl": 0, "simpl": 0, "sinc": 0, "skip": 0, "sklearn": 0, "so": 0, "some": 0, "specifi": 0, "split": 0, "squar": 0, "stabil": 0, "stability_select": 0, "stabl": 0, "start": 0, "state": 0, "step": 0, "stop": 0, "str": 0, "strategi": 0, "sub": 0, "subobject": 0, "subset": 0, "sum": 0, "t": 0, "take": 0, "target": 0, "task": 0, "tensor": 0, "test": 0, "th": 0, "thi": 0, "tie": 0, "tie_approxim": 0, "tol": 0, "torch": 0, "torch_se": 0, "total": 0, "train": 0, "true": 0, "tupl": 0, "two": 0, "type": 0, "u": 0, "unchang": 0, "uniform_averag": 0, "until": 0, "updat": 0, "upper": 0, "us": 0, "user": 0, "util": 0, "v": 0, "val_siz": 0, "valid": 0, "valu": 0, "variabl": 0, "verbos": 0, "version": 0, "w": 0, "wait": 0, "websit": 0, "weight": 0, "well": 0, "when": 0, "where": 0, "which": 0, "without": 0, "work": 0, "wors": 0, "would": 0, "wrong": 0, "x": 0, "x_test": 0, "x_val": 0, "y": 0, "y_pred": 0, "y_test": 0, "y_true": 0, "y_val": 0, "you": 0, "zero": 0}, "titles": ["Welcome to LassoNet\u2019s documentation!"], "titleterms": {"": 0, "api": 0, "document": 0, "instal": 0, "lassonet": 0, "welcom": 0}}) \ No newline at end of file diff --git a/examples/cox_regression.py b/examples/cox_regression.py index c14e68d..7b8024d 100644 --- a/examples/cox_regression.py +++ b/examples/cox_regression.py @@ -23,9 +23,9 @@ tie_approximation="breslow", ) -path = model.path(X_train, y_train, return_state_dicts=True) +model.path(X_train, y_train, return_state_dicts=True) -plot_path(model, path, X_test, y_test) +plot_path(model, X_test, y_test) plt.savefig("cox_regression.png") @@ -40,5 +40,5 @@ path = model.path(X_train, y_train) -plot_path(model, path, X_test, y_test) +plot_path(model, X_test, y_test) plt.savefig("cox_regression_efron.png") diff --git a/examples/diabetes.py b/examples/diabetes.py index 7705432..2fa8fa8 100644 --- a/examples/diabetes.py +++ b/examples/diabetes.py @@ -29,7 +29,7 @@ ) path = model.path(X_train, y_train, return_state_dicts=True) -plot_path(model, path, X_test, y_test) +plot_path(model, X_test, y_test) plt.savefig("diabetes.png") diff --git a/examples/friedman.py b/examples/friedman.py index 260b99e..0e48fd0 100644 --- a/examples/friedman.py +++ b/examples/friedman.py @@ -50,7 +50,7 @@ def rrmse(y, y_pred): "rrmse:", min(rrmse(y_test, model.load(save).predict(X_test)) for save in path), ) - plot_path(model, path, X_test, y_test, score_function=rrmse) + plot_path(model, X_test, y_test, score_function=rrmse) plt.savefig(f"friedman_path({path_multiplier})_M({M}).jpg") path_multiplier = 1.001 @@ -70,7 +70,7 @@ def rrmse(y, y_pred): "rrmse:", min(rrmse(y_test, model.load(save).predict(X_test)) for save in path), ) - plot_path(model, path, X_test, y_test, score_function=rrmse) + plot_path(model, X_test, y_test, score_function=rrmse) plt.savefig(f"friedman_path({path_multiplier})_M({M})_backtrack.jpg") @@ -92,7 +92,7 @@ def rrmse(y, y_pred): "rrmse:", min(rrmse(y_test, model.load(save).predict(X_test)) for save in path), ) - plot_path(model, path, X_test, y_test, score_function=rrmse) + plot_path(model, X_test, y_test, score_function=rrmse) plt.savefig(f"friedman_path({path_multiplier})_M({M})_long.jpg") # if __name__ == "__main__": @@ -100,5 +100,5 @@ def rrmse(y, y_pred): # model = LassoNetRegressor(verbose=True, path_multiplier=1.01, hidden_dims=(10, 10)) # path = model.path(X_train, y_train) -# plot_path(model, path, X_test, y_test, score_function=rrmse) +# plot_path(model, X_test, y_test, score_function=rrmse) # plt.show() diff --git a/examples/friedman/main.py b/examples/friedman/main.py index 833a1ff..12ea49e 100644 --- a/examples/friedman/main.py +++ b/examples/friedman/main.py @@ -49,5 +49,5 @@ def rrmse(y, y_pred): "rrmse:", min(rrmse(y_test, model.load(save).predict(X_test)) for save in path), ) - plot_path(model, path, X_test, y_test, score_function=rrmse) + plot_path(model, X_test, y_test, score_function=rrmse) plt.show() diff --git a/examples/generated.py b/examples/generated.py index e590820..4fd6985 100644 --- a/examples/generated.py +++ b/examples/generated.py @@ -66,5 +66,5 @@ def score(self, X, y, sample_weight=None): model.score = partial(score, model) - plot_path(model, path, X_test, y_test) + plot_path(model, X_test, y_test) plt.show() diff --git a/examples/madelon.py b/examples/madelon.py new file mode 100644 index 0000000..b3afd1e --- /dev/null +++ b/examples/madelon.py @@ -0,0 +1,25 @@ +# %% +import openml + +dataset = openml.datasets.get_dataset(1485) +X, *_ = dataset.get_data(dataset_format="dataframe") +# %% +# class is last column +X, y = X.iloc[:, :-1], X.iloc[:, -1] + +# %% +X = X.to_numpy() / 500 +# %% +y = (y == "2").to_numpy() +# %% + +from lassonet import LassoNetClassifier + +# %% +model = LassoNetClassifier(hidden_dims=(10, 10)) + +oracle, order, wrong, paths, prob = model.stability_selection(X, y, n_models=500) + +# %% +sorted(model.feature_importances_) +# %% diff --git a/examples/miceprotein.py b/examples/miceprotein.py index af45db0..b47abbf 100755 --- a/examples/miceprotein.py +++ b/examples/miceprotein.py @@ -38,60 +38,60 @@ model = LassoNetClassifier() path = model.path(X_train, y_train, return_state_dicts=True) -plot_path(model, path, X_test, y_test) +plot_path(model, X_test, y_test) plt.savefig("miceprotein.png") model = LassoNetClassifier(dropout=0.5) path = model.path(X_train, y_train, return_state_dicts=True) -plot_path(model, path, X_test, y_test) +plot_path(model, X_test, y_test) plt.savefig("miceprotein_dropout.png") model = LassoNetClassifier(hidden_dims=(100, 100)) path = model.path(X_train, y_train, return_state_dicts=True) -plot_path(model, path, X_test, y_test) +plot_path(model, X_test, y_test) plt.savefig("miceprotein_deep.png") model = LassoNetClassifier(hidden_dims=(100, 100), gamma=0.01) path = model.path(X_train, y_train, return_state_dicts=True) -plot_path(model, path, X_test, y_test) +plot_path(model, X_test, y_test) plt.savefig("miceprotein_deep_l2_weak.png") model = LassoNetClassifier(hidden_dims=(100, 100), gamma=0.1) path = model.path(X_train, y_train, return_state_dicts=True) -plot_path(model, path, X_test, y_test) +plot_path(model, X_test, y_test) plt.savefig("miceprotein_deep_l2_strong.png") model = LassoNetClassifier(hidden_dims=(100, 100), gamma=1) path = model.path(X_train, y_train, return_state_dicts=True) -plot_path(model, path, X_test, y_test) +plot_path(model, X_test, y_test) plt.savefig("miceprotein_deep_l2_super_strong.png") model = LassoNetClassifier(hidden_dims=(100, 100), dropout=0.5) path = model.path(X_train, y_train, return_state_dicts=True) -plot_path(model, path, X_test, y_test) +plot_path(model, X_test, y_test) plt.savefig("miceprotein_deep_dropout.png") model = LassoNetClassifier(hidden_dims=(100, 100), backtrack=True, dropout=0.5) path = model.path(X_train, y_train, return_state_dicts=True) -plot_path(model, path, X_test, y_test) +plot_path(model, X_test, y_test) plt.savefig("miceprotein_deep_dropout_backtrack.png") model = LassoNetClassifier(batch_size=64) path = model.path(X_train, y_train, return_state_dicts=True) -plot_path(model, path, X_test, y_test) +plot_path(model, X_test, y_test) plt.savefig("miceprotein_64.png") model = LassoNetClassifier(backtrack=True) path = model.path(X_train, y_train, return_state_dicts=True) -plot_path(model, path, X_test, y_test) +plot_path(model, X_test, y_test) plt.savefig("miceprotein_backtrack.png") model = LassoNetClassifier(batch_size=64, backtrack=True) path = model.path(X_train, y_train, return_state_dicts=True) -plot_path(model, path, X_test, y_test) +plot_path(model, X_test, y_test) plt.savefig("miceprotein_backtrack_64.png") model = LassoNetClassifier(class_weight=[0.1, 0.2, 0.3, 0.1, 0.3, 0, 0, 0]) path = model.path(X_train, y_train, return_state_dicts=True) -plot_path(model, path, X_test, y_test) +plot_path(model, X_test, y_test) plt.savefig("miceprotein_weighted.png") diff --git a/examples/spinet b/examples/spinet index a88dba7..d9c7d19 160000 --- a/examples/spinet +++ b/examples/spinet @@ -1 +1 @@ -Subproject commit a88dba7d7e400eb47f380645b38983bde63138fe +Subproject commit d9c7d195e0c8b511f7ae4dcef9dad962576f0ec7 diff --git a/lassonet/interfaces.py b/lassonet/interfaces.py index 46e086b..d8585d9 100755 --- a/lassonet/interfaces.py +++ b/lassonet/interfaces.py @@ -504,6 +504,8 @@ def _lambda_seq(start): self.feature_importances_ = self._compute_feature_importances(hist) """When does each feature disappear on the path?""" + self.path_ = hist + return hist def _stability_selection_path(self, X, y, lambda_seq=None) -> List[HistoryItem]: @@ -918,7 +920,6 @@ def callback(model, hist): lambda_seq=[h.lambda_ for h in path[1:-1]], return_state_dicts=return_state_dicts, ) - self.path_ = path self.best_selected_ = path[-1].selected return path diff --git a/lassonet/plot.py b/lassonet/plot.py index 4747476..36a36a3 100644 --- a/lassonet/plot.py +++ b/lassonet/plot.py @@ -4,19 +4,19 @@ from .utils import confidence_interval, eval_on_path -def plot_path(model, path, X_test, y_test, *, score_function=None): +def plot_path(model, X_test, y_test, *, score_function=None): """ Plot the evolution of the model on the path, namely: - lambda - number of selected variables - score + Requires to have called model.path(return_state_dicts=True) beforehand. + Parameters ========== model : LassoNetClassifier or LassoNetRegressor - path - output of model.path X_test : array-like y_test : array-like score_function : function or None @@ -24,9 +24,11 @@ def plot_path(model, path, X_test, y_test, *, score_function=None): score_function must take as input X_test, y_test """ # TODO: plot with manually computed score - score = eval_on_path(model, path, X_test, y_test, score_function=score_function) - n_selected = [save.selected.sum() for save in path] - lambda_ = [save.lambda_ for save in path] + score = eval_on_path( + model, model.path_, X_test, y_test, score_function=score_function + ) + n_selected = [save.selected.sum() for save in model.path_] + lambda_ = [save.lambda_ for save in model.path_] plt.figure(figsize=(16, 16)) diff --git a/setup.py b/setup.py index fdd4e95..7b4cc7e 100755 --- a/setup.py +++ b/setup.py @@ -9,7 +9,7 @@ def read(fname): setup( name="lassonet", - version="0.0.19", + version="0.0.20", author="Louis Abraham, Ismael Lemhadri", author_email="louis.abraham@yahoo.fr, lemhadri@stanford.edu", license="MIT",