Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

Roadmap

Jump to bottom
Scarlett Li edited this page Jun 4, 2019 · 59 revisions

NNI Roadmap 2019

We typically look out 6 to 12 months and establish topics we want to work on. As we go we learn and our assessment of some of the topics listed changes. Thus, we may add or drop topics as we go.

Items with checkbox [] are work items in progress or finished. Items in bullets * are still in backlog need triage.

We describe some initiatives as "investigations" which means our goal in the next few months is to better understand the problem and potential solutions before scheduling actual feature work. Once an investigation is done, we will update our plan, either deferring the initiative or committing to it.

Iteration plans can be found --> Iteration Plans.

As always, we will listen to your feedback and adapt our plans if needed.

New Features and Improvements

Tuners and Assessors

  • Tuner and assessor compete for the same message queue, assessor may starve tuner. #841
  • Enable ealy stopped trials data for tuners. #843
  • New Tuner: BOHB. #844
  • Assessor knows trials’ status (successful/failed), does tuner also have this requirement? #857
  • Provide easy-to-use interface for neural architecture search #869
  • New Tuner: Design a general and efficient tuner for NAS #870
  • Paralize TPE

Trials, Metrics, Experiements and Examples

  • retrieve the values of the metrics in a programmatic way #868
  • Ability to generate analysis results report after the experiement execution #880
  • investigations: Ability to compare experiements #881
  • support search space recommendation #84 #878
  • Examples: Add RL example on NNI, and think about optimization opportunities #872

Training Service

  • Doc for "how to implement your own training service" #838
  • Work out a plan for HDFS quota handling and authentication in NNI. #836
  • Show PAI job link in NNI trial section. #837
  • Assign dedicated resources (such as GPU) to Tuner #874
  • investigations: NNI CTRL support HDFS trial query from PAI #884
  • investigations: PAI Integration: fault tolerance for trial failure (when trials having dependency) #883

Web Portal

  • add intermediate result graph #839
  • optimize hyper-parameter graph #840
  • support intermediate result is dict #824

Onboarding and Upgrading Experience

  • Support NNI local mode on Windows #875
  • Support NNI remote mode on Windows
  • Support NNI PAI mode on Windows
  • investigations: A better tuner installation experience. #858
  • investigations: Inconsistent version on different machine. #859

Long term Architecture Investigation

  • investigations: NNI for tuning system parameters #860
  • investigations: Auto feature extraction #877
  • investigations: Checkpoint support to resume from any previous experiment #859
  • Support automatic model compression on NNI #876

Engineering efforts like CICD

  • Pass pylint and tslint #882

This wiki is a journal that tracks the development of NNI. It's not guaranteed to be up-to-date. Read NNI documentation for latest information: https://nni.readthedocs.io/en/latest/

Clone this wiki locally