Skip to content

Roadmap to v1.0

Jump to bottom Edit New page
Rob Parrott edited this page Nov 1, 2013 · 10 revisions

Refactoring and additional features for a usable "1.0" release. More functionality will come in post-1.0 versions.

High-Level Design Goals

  • Refactor Nepho core to be more object oriented & modular, to encourage outside collaboration
  • Refactor Nepho into a client/server model, to allow asynchronous build monitoring and actions - is this necessary for 1.0?
  • Break orchestration out into providers
    • Build Vagrant provider
    • Build CloudFormation provider
    • Build OpenStack/Rackspace provider - after 1.0
    • Build Google Compute provider - after 1.0
  • Remove all scenarios/patterns/deployments from Nepho core
    • Create a new structure for "cloudlet" bundles
    • Package no more than 3-5 best-of-breed "cloudlets"
    • Create a web-accessible registry/listing of approved cloudlets
    • Allow for pulling in cloudlets from non-standard repos
  • Build cloudlet management into Nepho core, modeled on "puppet module" command
  • Build scenario management (listing, merging YAMLs, etc.) into Nepho core
  • Build interactive prompting for parameters into Nepho core
  • Provide multiple display/output options (pp, json, yaml) - expand this

Refactoring

We want to adopt a more capable CLI framwork, and create a suitable OOP class structure to model the various objects we want to create. We will use Python Cement 2.0 as the basis.

The class structure represent the various textual objects that are assembled into a representation of the desired pattern and configuration, with super classes that are subclassed based on provider and config system.

Client/Server

Currently nepho is completely synchronous, and once a build is started, there's no way to monitor the progress through the tool, or start other processes, without interacting with provider-specific consoles. This is fine with AWS, but we have limited feedback on puppet builds, and if we wanted to use ansible, this would be more difficult. Instead, we want a nanny process to spawn off and monitor/manage the build process to completion.

This nanny process in its simplest form could be a forked process that would communicate back to the command ine client through a simple RPC service, possibly as a message queue. By ensuring that all objects are serializable, this would be straightforward. A further improvement would be for the nanny process to be launched on a remote host, so that longer builds could allow the user to disconnect and reconnect without affecting the build. Lastly, we could have a "foreground" mode that would not fork, but run the same process in the CLI's memory and process space.

Break Orchestration out into providers

Create a clearer seperation of concerns between Nepho components:

  • Nepho Core should manage the scenario (YAML) config file, and pass along options to the provider.
  • Provider should handle all CloudFormation API pieces, and then hand off to the provisioner.
  • Provisioner should handle system bootstrapping.

Note: While provider/provisioner boundaries are fuzzy, we should work to make them more clear. As an initial proposal/discussion point, I suggest that the provider does only the bare minimum cloud-init and then runs a handoff from provider to provisioner, which is always via a boostrap shell script, rather than Puppet management built into the provider. This would provide maximum flexibility for any given application, be it Puppet, Chef, Ansible, Salt, or just basic shell commands.

Create Cloudlet bundles

Superceded by Structure of a Cloudlet.

Example Manpage

Simplified cloud orchestration tool for constructing virtual data centers.

Usage: nepho <command> [<args>...]

Configuration commands::
	register      set clould provider credentials

Shortcut commands::
	create        create a new formation from a scenario
	destroy       destroy a running formation
	open          open the public URL endpoint for the formation
	ssh           connect to the formation's public IP or bastion host

Subcommands, use ``nepho help [subcommand]`` to learn more::
	cloudlet      find, download, and manage cloudlets
	scenario      view and manage cloudlet deployment scenarios
	formation     create and manage cloudlet instances

Example Usage Sequence

  • Search for cloudlets
$ nepho cloudlet search mediawiki
huit/mediawiki - generic mediawiki awesomeness
tfhartmann/mediawiki - like huit but even better
  • Learn about a cloudlet
$ nepho cloudlet describe huit/mediawiki
Name: huit/mediawiki
Summary: generic mediawiki awesomeness
Description: This is a generic mediawiki module that supports both single-site and multi-site configurations.
Project URL: https://github.com/huit/nepho-mediawiki
Repository: git@github.com:huit/nepho-mediawiki.git
  • Install a cloudlet
$ nepho cloudlet install huit/mediawiki
Notice: Preparing to install into /Users/zeno/.nepho/cloudlets ...
Notice: Cloning from git@github.com:huit/nepho-mediawiki.git ...
/Users/zeno/.nepho/cloudlets
└── huit-mediawiki (v0.7.2)
  • List installed cloudlets
$ nepho cloudlet list
/Users/zeno/.nepho/cloudlets
└── huit-mediawiki (v0.7.2)
  • List scenarios in a cloudlet
$ nepho scenario list mediawiki
default        a single-host, single-tenant setup
multi-tenant   elastic scaling multi-tenant load-balanced service
  • Describe a scenario
$ nepho scenario:describe mediawiki/multi-tenant
Name: multi-tenant
Summary: elastic scaling multi-tenant load-balanced service
Description: Blah blah blah.

------------------------------------------------
Scenario Configuration:
------------------------------------------------
{ lots of YAML output }
------------------------------------------------

THESE COMMANDS NEED UPDATES BELOW ....
Modify these options by editing the scenario file:
/Users/zeno/.nepho/cloudlets/mediawiki/scenarios/multi-tenant.yaml
  • Create a formation from a scenario
$ nepho formation:create mediawiki/multi-tenant --name agmedia
<what should output be here?>
  • List formations
$ nepho formation:list
<what should output be here?>
  • Get info on a formation
$ nepho formation:describe agmedia
<what should output be here?>
  • Open the URL of a formation
$ nepho formation:open agmedia
<what should output be here?>

Complete list of nepho commands/subcommands

Note: I have removed all commands that are currently implemented or stubbed out, since the wiki documentation of commands is already getting stale, and the internal documentation will remain up to date.

The following commands have not yet been stubbed out/finalized:

  • formation:list - list all running nepho-tagged cloudformations in the AWS account
  • formation:create - create a new formation from a given cloudlet/scenario combo (nepho formation:create mediawiki/multi-tier or nepho formation:create mediawiki - implies "default"). give some useful processiong output? the stack id to query later? subscribe to sns topic?
  • formation:destroy - destroy a formation by name (if it is unique) or stack id
  • formation:open - open the tier 1 ELB CNAME in a browser (and print it to CLI)
  • formation:ssh - open a SSH connection to the bastion host (or if its public, the host specified as primary/bastion in the scenario) and print the address to CLI
  • formation:describe - query AWS and describe the state of the running formation
Add a custom footer
Add a custom sidebar
Clone this wiki locally