-
Notifications
You must be signed in to change notification settings - Fork 5
/
TODO
28 lines (23 loc) · 1.32 KB
/
TODO
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
Backlog:
-----------
Small:
** [Done] raise insights to user when there are important insights he should look into
** [Done] provide calibration curve for uncertainty models
** make separation quality into a report, and add sensitivity analysis for the separation classifier, and connect it to insights.
** add cross-report insights - e.g. features sensitive to missing values that also has many missing values
** add uncertainty of model to uncertainty predictions
** analyze which areas of the feature space is more prone to errors (either using uncertainty estimator, or basic analysis)
** Metrics - micro / binary / additional
** Uncertainty methods - prediction intervals regression - xgboost quantiles
** add meta-data to dataset analysis report (feature types, names, base types, etc.)
Big:
** Readable report (e.g. pdf)
** UI - widgets for notebooks
** Explainability - shap / DT / KNN / linear
** Dataset quality - values distribution in feature, recognize complex feature types
- Check if more data is required, Find areas with not enough samples
- Find anomalies / outliers in data
- Find anomalies / outliers in target
** Uncertainty methods - sample out of distribution
** Advanced ML - group id, sample weight
** problem types - vision / time series / anomaly detection