Debugging Locally Orchestrator

Tutorial on debugging locally

Although K-FTLK is written to run experiments at scale, debugging locally is possible as well. This debuggin approach allows you to run the Orchestrator locally, while deploying the experiments on your cluster.

Pre-requisites

The following entries must be present on the system/computer you are intending to debug/develop with, this can be a single machine.

• Cloned the repository in a directory, we will refer to this directory as content_root.
• Created a working venv, during all times it is assumed that this virtual environment is active.
  • python3.9 -m venv venv
  • source venv/bin/activate
  • pip install -r requirements-cpu.txt
• Preferably use PyCharm to run the configurations files.
  • We provide configuration files that will allow to 'Run' and 'Debug' in PyCharm directly:
    • Download the configuration orchestrator.xml and place under .idea/runConfigurations in your content_root directory (i.e. the project in PyCharm).

In addition, a valid kube configuration needs to be available on your file-system at ~/.kube/config.

When not using PyCharm

When you are not using PyCharm, the following settings/environmental variables need to be set, assuming that you are running locally on a machine. Make sure that no process is bound to the port 12345, otherwise change the environment variables accordingly.

For the Orchestrator, the following command will be run:

python3 -m fltk cluster configs/example_cloud_experiment.json configs/distributed_tasks/example_arrival_config.json -l True

Getting started

Simply run the command, in PyCharm or shell. You can use a debugger to inspect code/add breakpoints.