Robot open-Ended Autonomous Learning Starter Kit

Instructions to make submissions to the Robot open-Ended Autonomous Learning.

Participants will have to submit their controllers, with packaging specifications, and the evaluator will automatically build a docker image and execute their controllers in two phases an Intrinsic Phase and an Extrinsic Phase.

In the first round of the competition, the Intrinsic Phase will be skipped.

Dependencies

• Anaconda (By following instructions here) At least version 4.5.11 is required to correctly populate environment.yml.
• real-robots PyPI

Setup

• Clone the repository

git clone git@github.com:AIcrowd/neurips_goal_real_robots_starter_kit.git
cd neurips_goal_real_robots_starter_kit

• Create a conda environment from the provided environment.yml

conda env create -f environment.yml

• Activate the conda environment and install your code specific dependencies

conda activate real_robots
# If say you want to install PyTorch
# conda install pytorch torchvision -c pytorch
#
# or you can even use pip to install any additional packages
# for example : 
# pip install -U real-robots
# which updates the real-robots package to the latest version

Test Submission Locally

• First lets download a sample goals.npy.npz file by doing :

wget https://aicrowd-production.s3.eu-central-1.amazonaws.com/misc/REAL-Robots/goals.npy.npz -O data/goals.npy.npz

• test locally

python local_evaluation.py

• (optional) build docker image locally and run docker container

pip install -U aicrowd-repo2docker
# This also expects that you have Docker, Nvidia-Docker installed

./debug.sh

How do I specify my software runtime ?

The software runtime is specified by exporting your conda env to the root of your repository by doing :

# The included environment.yml is generated by the command below, and you do not need to run it again
# if you did not add any custom dependencies

conda env export --no-build > environment.yml

# Note the `--no-build` flag, which is important if you want your anaconda env to be replicable across all

This environment.yml file will be used to recreate the conda environment inside the Docker container. This repository includes an example environment.yml

You can specify your software environment by using all the available configuration options of repo2docker. (But please remember to use aicrowd-repo2docker to have GPU support)

What should my code structure be like ?

Please follow the structure documented in the included my_controller.py to adapt your already existing code to the required structure for this round.

Important Concepts

Repository Structure

• aicrowd.json Each repository should have a aicrowd.json with the following content :

{
  "challenge_id": "goal_real_robots_challenge_2019",
  "grader_id": "goal_real_robots_challenge_2019",
  "authors": ["mohanty"],
  "description": "Robot open-Ended Autonomous Learning (REAL) Challenge.",
  "license" : "MIT",
  "debug": true
}

This is used to map your submission to the said challenge, so please remember to use the correct challenge_id and grader_id as specified above.

If you set debug to true, then the evaluation will run on a separate set of 20 environments, and the logs from your submitted code (if it fails), will be made available to you to help you debug. NOTE : IMPORTANT : By default we have set debug:false, so when you have done the basic integration testing of your code, and are ready to make a final submission, please do make sure to set debug to true in aicrowd.json.

• my_controller.py The task is to implement your own controller my following the template provided in my_controller.py. The my_controller.py file should finally reference the implemented class as SubmittedPolicy

Submission

To make a submission, you will have to create a private repository on https://gitlab.aicrowd.com/.

You will have to add your SSH Keys to your GitLab account by following the instructions here. If you do not have SSH Keys, you will first need to generate one.

Then you can create a submission by making a tag push to your repository on https://gitlab.aicrowd.com/. Any tag push (where the tag name begins with "submission-") to your private repository is considered as a submission
Then you can add the correct git remote, and finally submit by doing :

cd neurips_goal_real_robots_starter_kit
# Add AIcrowd git remote endpoint
git remote add aicrowd git@gitlab.aicrowd.com:<YOUR_AICROWD_USER_NAME>/neurips_goal_real_robots_starter_kit.git
git push aicrowd master

# Create a tag for your submission and push
git tag -am "submission-v0.1" submission-v0.1
git push aicrowd master
git push aicrowd submission-v0.1

# Note : If the contents of your repository (latest commit hash) does not change,
# then pushing a new tag will **not** trigger a new evaluation.

You now should be able to see the details of your submission at : https://gitlab.aicrowd.com/<YOUR_AICROWD_USER_NAME>/neurips_goal_real_robots_starter_kit/issues

NOTE: Remember to update your username in the link above 😉

In the link above, you should start seeing something like this take shape (the whole evaluation can take a bit of time, so please be a bit patient too 😉 ) :

Best of Luck 🎉 🎉

Author

Sharada Mohanty