Roadmap to v1.0

Refactoring and additional features for a usable "1.0" release. More functionality will come in post-1.0 versions.

High-Level Design Goals

• Refactor Nepho core to be more object oriented & modular, to encourage outside collaboration
• Break orchestration out into providers
  • Build CloudFormation provider
  • Build Vagrant provider
  • Build OpenStack/Rackspace provider
  • Build Google Compute provider
• Remove all scenarios/patterns/deployments from Nepho core
  • Package no more than 3-5 "cloudlet" bundles of scenarios/patterns/deployments into their own repositories (i.e. nepho-mediawiki)
  • Create a web-accessible registry/listing of approved cloudlets
  • Allow for pulling in cloudlets from non-standard repos
• Build cloudlet management into Nepho core, modeled on "puppet module" command
• Build scenario management (listing, merging YAMLs, etc.) into Nepho core
• Provide multiple display/output options (pp, json, yaml)

Break Orchestration out into providers

Create a clearer seperation of concerns between Nepho components:

• Nepho Core should manage the scenario (YAML) config file, and pass along options to the provider.
• Provider should handle all CloudFormation API pieces, and then hand off to the provisioner.
• Provisioner should handle system bootstrapping.

Note: While provider/provisioner boundaries are fuzzy, we should work to make them more clear. As an initial proposal/discussion point, I suggest that the provider does only the bare minimum cloud-init and then runs a handoff from provider to provisioner, which is always via a boostrap shell script, rather than Puppet management built into the provider. This would provide maximum flexibility for any given application, be it Puppet, Chef, Ansible, Salt, or just basic shell commands.

Create Cloudlet bundles

A cloudlet consists of the following complete set of components in a single git repository

• A cloudlet info file, in YAML format. Same content as a puppet Modulefile (name, version, source, author, license, summary, description, projett page)
• Scenario File(s) - One per scenario, minimum of one, named "default". Should the concept of "environments" apply here, or should that be incorporated into the scenario design?
• CloudFormation Pattern(s), under an "aws" tree for future expandability
• Provisioner bootstrap file(s) - TBD (one or more shell scripts, scenario chooses one?)

.nepho
└── cloudlets
    └── mediawiki
        ├── cloudlet.yaml
        ├── patterns
        │   └── aws
        │       ├── single-host.cf
        │       └── two-tier-with-vpc.cf
        ├── provisioners
        │   └── bootstrap.sh
        └── scenarios
            └── default.yaml
            └── multi-tenant.yaml

Questions:

• Should common CF templates be submoduled or subpathed?
• Likewise for Puppet bootstrap script?

Example Manpage

Simplified cloud orchestration tool for constructing virtual data centers.

Usage: nepho <command> [<args>...]

Configuration commands::
	register      set clould provider credentials

Shortcut commands::
	create        create a new formation from a scenario
	destroy       destroy a running formation
	open          open the public URL endpoint for the formation
	ssh           connect to the formation's public IP or bastion host

Subcommands, use ``nepho help [subcommand]`` to learn more::
	cloudlet      find, download, and manage cloudlets
	scenario      view and manage cloudlet deployment scenarios
	formation     create and manage cloudlet instances

Example Usage Sequence

• Search for cloudlets

$ nepho cloudlet:search mediawiki
huit/mediawiki - generic mediawiki awesomeness
tfhartmann/mediawiki - like huit but even better

• Learn about a cloudlet

$ nepho cloudlet:describe huit/mediawiki
Name: huit/mediawiki
Summary: generic mediawiki awesomeness
Description: This is a generic mediawiki module that supports both single-site and multi-site configurations.
Project URL: https://github.com/huit/nepho-mediawiki
Repository: git@github.com:huit/nepho-mediawiki.git

• Install a cloudlet

$ nepho cloudlet:install huit/mediawiki
Notice: Preparing to install into /Users/zeno/.nepho/cloudlets ...
Notice: Cloning from git@github.com:huit/nepho-mediawiki.git ...
/Users/zeno/.nepho/cloudlets
└── huit-mediawiki (v0.7.2)

• List installed cloudlets

$ nepho cloudlet:list
/Users/zeno/.nepho/cloudlets
└── huit-mediawiki (v0.7.2)

• List scenarios in a cloudlet

$ nepho scenario:list mediawiki
default        a single-host, single-tenant setup
multi-tenant   elastic scaling multi-tenant load-balanced service

• Describe a scenario

$ nepho scenario:describe mediawiki/multi-tenant
Name: multi-tenant
Summary: elastic scaling multi-tenant load-balanced service
Description: Blah blah blah.

------------------------------------------------
Scenario Configuration:
------------------------------------------------
{ lots of YAML output }
------------------------------------------------

Modify these options by editing the scenario file:
/Users/zeno/.nepho/cloudlets/mediawiki/scenarios/multi-tenant.yaml

• Create a formation from a scenario

$ nepho formation:create mediawiki/multi-tenant --name agmedia
<what should output be here?>

• List formations

$ nepho formation:list
<what should output be here?>

• Get info on a formation

$ nepho formation:describe agmedia
<what should output be here?>

• Open the URL of a formation

$ nepho formation:open agmedia
<what should output be here?>

Complete list of nepho commands/subcommands

• register - if AWS credentials are missing, present URL to a nepho wiki page about how to sign up for AWS, followed by prompting for region, key id, and secret key
• cloudlet:list - list installed cloudlets (see puppet module list)
• cloudlet:search - download/cache register of cloudlets (JSON), search and return results
• cloudlet:install - download cloudlet from URL in register, or provided URL (git repo) -- also instantiate submodules?
• cloudlet:uninstall - delete cloudlet files from local disk
• cloudlet:update - run git pull on specified cloudlet or all of them
• scenario:list - list all possible scenarios for a given cloudlet (may just be "default") along with the one-line summary of the scenario
• scenario:describe - list the given scenario, summary, long description, and YAML output
• formation:list - list all running nepho-tagged cloudformations in the AWS account
• formation:create - create a new formation from a given cloudlet/scenario combo (nepho formation:create mediawiki/multi-tier or nepho formation:create mediawiki - implies "default"). give some useful processiong output? the stack id to query later? subscribe to sns topic?
• formation:destroy - destroy a formation by name (if it is unique) or stack id
• formation:open - open the tier 1 ELB CNAME in a browser (and print it to CLI)
• formation:ssh - open a SSH connection to the bastion host (or if its public, the host specified as primary/bastion in the scenario) and print the address to CLI
• formation:describe - query AWS and describe the state of the running formation