Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

Roadmap

Jump to bottom
rabbit008 edited this page Aug 29, 2019 · 59 revisions

NNI Roadmap 2019

We typically look out 6 to 12 months and establish topics we want to work on. As we go we learn and our assessment of some of the topics listed changes. Thus, we may add or drop topics as we go.

Items with checkbox [] are work items in progress or finished. Items in bullets * are still in backlog need triage.

We describe some initiatives as "investigations" which means our goal in the next few months is to better understand the problem and potential solutions before scheduling actual feature work. Once an investigation is done, we will update our plan, either deferring the initiative or committing to it.

Iteration plans can be found --> Iteration Plans.

As always, we will listen to your feedback and adapt our plans if needed.

New Features and Improvements

Tuners and Assessors

  • Enable ealy stopped trials data for tuners. #843
  • New Tuner: BOHB. #844
  • Provide easy-to-use interface for neural architecture search #869
  • New Tuner: Design a general and efficient tuner for NAS #870
  • Paralize TPE #1052

Trials, Metrics, Experiements and Examples

  • Ability to compare experiments #881
  • Ability to generate analysis results report after the experiement execution #880
  • Retrieve the values of the metrics in a programmatic way #868
  • Support search space recommendation #84 #878
  • Examples: Add RL example on NNI, and think about optimization opportunities #872

Training Service

  • Work out a plan for HDFS quota handling and authentication in NNI. #836
  • Assign dedicated resources (such as GPU) to Tuner #874
  • Show PAI job link in NNI trial section. #837
  • NNI CTRL support HDFS trial query from PAI #884
  • PAI Integration: fault tolerance for trial failure (when trials having dependency) #883
  • Assign dedicated resources (such as GPU) to Tuner #874
  • Speed up HPO with intermediate result #1229

Web Portal

  • add intermediate result graph #839
  • optimize hyper-parameter graph #840
  • support intermediate result is dict #824
  • (In Progress)Support resubmit the same trial again #1268

Onboarding and Upgrading Experience

  • Support NNI local mode on Windows #875
  • Support NNI remote mode on Windows #1053
  • Support NNI PAI mode on Windows #1051
  • Inconsistent version on different machine. #859
  • investigations: A better tuner installation experience. #858
  • (In Progress)Support automatic model compression on NNI #876

Docs,Blogs,Samples,Articles

  • Doc for "how to implement your own training service" #838
  • Latest paper for NAS (AutoML blog/analysis). #1160

Long term Architecture Investigation

  • investigations: NNI for tuning system parameters #860
  • Auto feature extraction #877

Engineering efforts like CICD

  • (in progress) Pass pylint and tslint #882

This wiki is a journal that tracks the development of NNI. It's not guaranteed to be up-to-date. Read NNI documentation for latest information: https://nni.readthedocs.io/en/latest/

Clone this wiki locally