aicrowd

A CLI app to interact with AIcrowd

How to install the CLI for development

• Install the API locally https://github.com/AIcrowd/aicrowd-api
• Install python client for Evaluations API https://gitlab.aicrowd.com/aicrowd/evaluations-api-clients/tree/dev/python (for grader commands only)
• Clone this repository https://github.com/AIcrowd/aicrowd-cli.git
• inside the repository pip install -r requirements.txt
• then do python setup.py develop
• And now you are good to start developing/using the features

Project Features

Before using the CLI one has to generate the ssh keys and access token required to interact with Gitlab.

• To generate ssh keys and upload them to Gitlab:

aicrowd ssh create

• To genereate personal access token from Gitlab:

aicrowd access_token create

• To create a folder for a challenge with challenge spec:

aicrowd challenge init

The console will ask for the challenge id (which will be provided on the challenge home page)

• To view all the datasets related to a challenge, (has to be run inside the challenge folder):

aicrowd dataset list

• To download a dataset:

aicrowd dataset download

• To Start with a template for the challenge:

aicrowd template list
aicrowd template download

• To quickly start working on a baseline for the challenge:

aicrowd baseline list
aicrowd baseline download

• To make a submission:

aicrowd submit

(Submit requires one to have already run aicrowd ssh create)

• To check the proogress of the submission once submitted:

aicrowd status

AIcrowd Evaluations API Commands

• Login and save the auth token locally.

aicrowd evaluations login -e <email> -p <password>

• Validate the grader repository

git clone <grader_repository>
cd <repository>
aicrowd evaluations grader create --repo <grader_repository> --validate

• Creating a grader

git clone <grader_repository>
cd <repository>
aicrowd evaluations grader create --repo <grader_repository>

• Creating a grader and deploying to AIcrowd

aicrowd keys add -k AICROWD_API_KEY=<aicrowd_api_key>
git clone <grader_repository>
cd <repository>
aicrowd evaluations grader create --repo <grader_repository> --deploy

• Passing the secrets when creating grader

aicrowd evaluations grader create \ 
    --repo <grader_repository> \ 
    -s secret1=value \ 
    -s secret2=value

Many more commands like aicrowd convert and aicrowd capture yet to come. Stay tuned.

• aicrowd
• a starter Click command-line application
• automated unit tests you can run with pytest
• a Sphinx documentation project

Getting Started

The project's documentation contains a section to help you get started as a developer or user of the library.

Development Prerequisites

If you're going to be working in the code (rather than just using the library), you'll want a few utilities.

• GNU Make
• Pandoc

Resources

Below are some handy resource links.

• Project Documentation
• Click is a Python package for creating beautiful command line interfaces in a composable way with as little code as necessary.
• Sphinx is a tool that makes it easy to create intelligent and beautiful documentation, written by Geog Brandl and licnsed under the BSD license.
• pytest helps you write better programs.
• GNU Make is a tool which controls the generation of executables and other non-source files of a program from the program's source files.

Authors

• S.P. Mohanty - Initial work - github
• Nikhil Rayaprolu - Initial work (Current Maintainer) - github

See also the list of contributors who participated in this project.

LicenseCopyright (c) AIcrowd SA

All rights reserved.