Project Coriolis

Cloud Migration as a Service

Migrating existing workloads between clouds is a necessity for a large number of use cases, especially for user moving from traditional virtualization technologies like VMware vSphere or Microsoft System Center VMM to Azure / AzureStack, OpenStack, Amazon AWS or Google Cloud. Furthermore, cloud to cloud migrations, like AWS to Azure are also a common requirement.

Project Coriolis(R) addresses exactly those requirements, in particular migrating Linux (Ubuntu, Red Hat / CentOS, SUSE, Debian, Fedora) and Windows virtual machine, templates, storage and networking configurations.

There are some tricky scenarios where Coriolis excels: to begin with, virtual machines need to be moved between different hypervisors, which means including new operating system drivers and tools, for example cloud-init / cloudbase-init in the OpenStack use case, LIS kernel modules on Hyper-V and Azure and so on.

The project is largely using Oslo libraries with an architecture meant to be familiar for devops used to OpenStack, as shown in the diagram below. A clear separation between stateless microservices and proper usage of queues allows scalability and fault tolerance from the start. For PoCs and small environments, all components can be deployed on a single host / VM / container.

Authentication and endpoint discovery is based on Keystone (in the typical OpenStack way), which means that passing an X-Auth-Token header from an existing Keystone session allows an easy integration with other components, e.g. in Horizon's console. The same token is passed to other components along the pipeline, which also implies that importing virtual machines and other resources to the same OpenStack infrastructure doesn't require further authentication.

Authentication to external clouds (Azure, AWS, etc) or virtualisation solutions (vSphere, SCVMM, etc) in order to export virtual resources requires credentials that can be saved in Barbican, thus avoiding the need to pass secrets directly to the API.

Cloud resources that can be migrated:

• Virtual machines
• Virtual Machine templates
• Storage
• Network configurations

VM disks are converted to the desired target format and drivers / tools are automatically added where appropriate during the process (e. cloud-init on OpenStack, KVM Windows VirtIO drivers, etc).

The migration jobs are split in import / export tasks with a scheduler taking care of choosing a worker node where this can be executed. Each task contains progress update info that the client can poll to follow the progress of the operations. Tasks can be relatively long running, depending on the storage size, so proper status reporting was included in the design from the start.

Keystone configuration

Here's an example Keystone service and endpoints configuration:

openstack service create --name coriolis --description "Cloud Migration as a Service" migration

ENDPOINT_URL="http://hostname:7667/v1/%(tenant_id)s"
openstack endpoint create --region RegionOne migration `
--publicurl $ENDPOINT_URL `
--internalurl $ENDPOINT_URL `
--adminurl $ENDPOINT_URL

openstack user create --password-prompt coriolis
openstack role add --project service --user coriolis admin

API

The API is also very straightforward, here's a complete example available on Postman: https://api.postman.com/collections/2414861-766ace3d-b46c-4510-bbaf-e1f001a8be75?access_key=PMAT-01GJTF9HV7MY7YMHZW7Q79N5V2

Create a migration job:

POST http://server:7667/v1/%project_id/migrations

Example request body:

{
    "migration": {
        "origin": {
            "type": "vmware_vsphere",
            "connection_info": {
                "secret_id": "ebe69d82-da6f-451e-a0f6-3551d0f7ef85"
            }
        },
        "destination": {
            "type": "openstack",
            "target_environment": {
                "flavor_name": "m1.small",
                "network_map": {
                    "VM Network": "private",
                    "VM Network Local": "public"
                }
            }
        },
        "instances": ["CentOS 7", "RHEL 7.2", "Ubuntu 14.04", "WS 2012 R2"]
    }
}

Note: here's an example secret stored in Barbican with vSphere connection info:

{
    "host": "10.0.0.10",
    "username": "user@vsphere.local",
    "password": "Password",
    "allow_untrusted": true
}

List migrations:

GET http://server:7667/v1/%(project_id)s/migrations GET http://server:7667/v1/%(project_id)s/migrations/detail

Get a migration job info:

GET http://server:7667/v1/%(project_id)s/migrations/%(migration_id)s

Cancel a migration job:

This API allows the user to interrupt any running job.

POST http://server:7667/v1/%(project_id)s/migrations/%(migration_id)s/action

Request body:

{ "cancel": null }

Delete a migration job:

DELETE http://server:7667/v1/%(project_id)s/migrations/%(migration_id)s

Note: only completed, failed or cancelled jobs can be deleted.

API Documentation

To build the API documentation, while in the repository root directory, run:

sphinx-build -W -b html coriolis/api-refs/source $DOCS_PATH

API bindings

A reference Python client library implementation is available at: https://github.com/cloudbase/python-coriolisclient

Web UI

The official Web-based GUI for Coriolis is available at: https://github.com/cloudbase/coriolis-web