-
Notifications
You must be signed in to change notification settings - Fork 6
Roadmap to v1.0
Refactoring and additional features for a usable "1.0" release. More functionality will come in post-1.0 versions.
- Refactor Nepho core to be more object oriented & modular, to encourage outside collaboration
- Refactor nepho into an client/server model, to allow asynchronous build monitoring and actions
- Break orchestration out into providers
- Build CloudFormation provider
- Build Vagrant provider
- Build OpenStack/Rackspace provider
- Build Google Compute provider
- Remove all scenarios/patterns/deployments from Nepho core
- Package no more than 3-5 "cloudlet" bundles of scenarios/patterns/deployments into their own repositories (i.e. nepho-mediawiki)
- Create a web-accessible registry/listing of approved cloudlets
- Allow for pulling in cloudlets from non-standard repos
- Build cloudlet management into Nepho core, modeled on "puppet module" command
- Build scenario management (listing, merging YAMLs, etc.) into Nepho core
- Provide multiple display/output options (pp, json, yaml)
We want to adopt a more capable CLI framwork, and create a suitable OOP class structure to model the various objects we want to create.
Consider using python-cement
for the CLI framework, and a basic object model for classes.
The class structure represent the various textual objects that are assembled into a representation of the desired pattern and configuration, with super classes that are subclassed based on provider and config system.
Currently nepho is completely synchronous, and once a build is started, there's no way to monitor the progress through the tool, or start other processes, without interacting with provider-specific consoles. This is fine with AWS, but we have limited feedback on puppet builds, and if we wanted to use ansible, this would be more difficult. Instead, we want a nanny process to spawn off and monitor/manage the build process to completion.
This nanny process in its simplest form could be a forked process that would communicate back to the command ine client through a simple RPC service, possibly as a message queue. By ensuring that all objects are serializable, this would be straightforward. A further improvement would be for the nanny process to be launched on a remote host, so that longer builds could allow the user to disconnect and reconnect without affecting the build. Lastly, we could have a "foreground" mode that would not fork, but run the same process in the CLI's memory and process space.
Create a clearer seperation of concerns between Nepho components:
- Nepho Core should manage the scenario (YAML) config file, and pass along options to the provider.
- Provider should handle all CloudFormation API pieces, and then hand off to the provisioner.
- Provisioner should handle system bootstrapping.
Note: While provider/provisioner boundaries are fuzzy, we should work to make them more clear. As an initial proposal/discussion point, I suggest that the provider does only the bare minimum cloud-init and then runs a handoff from provider to provisioner, which is always via a boostrap shell script, rather than Puppet management built into the provider. This would provide maximum flexibility for any given application, be it Puppet, Chef, Ansible, Salt, or just basic shell commands.
A cloudlet consists of the following complete set of components in a single git repository
- A cloudlet info file, in YAML format. Same content as a puppet Modulefile (name, version, source, author, license, summary, description, projett page)
- Scenario File(s) - One per scenario, minimum of one, named "default". Should the concept of "environments" apply here, or should that be incorporated into the scenario design?
- CloudFormation Pattern(s), under an "aws" tree for future expandability
- Provisioner bootstrap file(s) - TBD (one or more shell scripts, scenario chooses one?)
.nepho
└── cloudlets
└── mediawiki
├── cloudlet.yaml
├── patterns
│ └── aws
│ ├── single-host.cf
│ └── two-tier-with-vpc.cf
├── provisioners
│ └── bootstrap.sh
└── scenarios
└── default.yaml
└── multi-tenant.yaml
Questions:
- Should common CF templates be submoduled or subpathed?
- Likewise for Puppet bootstrap script?
Simplified cloud orchestration tool for constructing virtual data centers.
Usage: nepho <command> [<args>...]
Configuration commands::
register set clould provider credentials
Shortcut commands::
create create a new formation from a scenario
destroy destroy a running formation
open open the public URL endpoint for the formation
ssh connect to the formation's public IP or bastion host
Subcommands, use ``nepho help [subcommand]`` to learn more::
cloudlet find, download, and manage cloudlets
scenario view and manage cloudlet deployment scenarios
formation create and manage cloudlet instances
- Search for cloudlets
$ nepho cloudlet:search mediawiki
huit/mediawiki - generic mediawiki awesomeness
tfhartmann/mediawiki - like huit but even better
- Learn about a cloudlet
$ nepho cloudlet:describe huit/mediawiki
Name: huit/mediawiki
Summary: generic mediawiki awesomeness
Description: This is a generic mediawiki module that supports both single-site and multi-site configurations.
Project URL: https://github.com/huit/nepho-mediawiki
Repository: git@github.com:huit/nepho-mediawiki.git
- Install a cloudlet
$ nepho cloudlet:install huit/mediawiki
Notice: Preparing to install into /Users/zeno/.nepho/cloudlets ...
Notice: Cloning from git@github.com:huit/nepho-mediawiki.git ...
/Users/zeno/.nepho/cloudlets
└── huit-mediawiki (v0.7.2)
- List installed cloudlets
$ nepho cloudlet:list
/Users/zeno/.nepho/cloudlets
└── huit-mediawiki (v0.7.2)
- List scenarios in a cloudlet
$ nepho scenario:list mediawiki
default a single-host, single-tenant setup
multi-tenant elastic scaling multi-tenant load-balanced service
- Describe a scenario
$ nepho scenario:describe mediawiki/multi-tenant
Name: multi-tenant
Summary: elastic scaling multi-tenant load-balanced service
Description: Blah blah blah.
------------------------------------------------
Scenario Configuration:
------------------------------------------------
{ lots of YAML output }
------------------------------------------------
Modify these options by editing the scenario file:
/Users/zeno/.nepho/cloudlets/mediawiki/scenarios/multi-tenant.yaml
- Create a formation from a scenario
$ nepho formation:create mediawiki/multi-tenant --name agmedia
<what should output be here?>
- List formations
$ nepho formation:list
<what should output be here?>
- Get info on a formation
$ nepho formation:describe agmedia
<what should output be here?>
- Open the URL of a formation
$ nepho formation:open agmedia
<what should output be here?>
- register - if AWS credentials are missing, present URL to a nepho wiki page about how to sign up for AWS, followed by prompting for region, key id, and secret key
-
cloudlet:list - list installed cloudlets (see
puppet module list) - cloudlet:search - download/cache register of cloudlets (JSON), search and return results
- cloudlet:install - download cloudlet from URL in register, or provided URL (git repo) -- also instantiate submodules?
- cloudlet:uninstall - delete cloudlet files from local disk
- cloudlet:update - run git pull on specified cloudlet or all of them
- scenario:list - list all possible scenarios for a given cloudlet (may just be "default") along with the one-line summary of the scenario
- scenario:describe - list the given scenario, summary, long description, and YAML output
- formation:list - list all running nepho-tagged cloudformations in the AWS account
-
formation:create - create a new formation from a given cloudlet/scenario combo (
nepho formation:create mediawiki/multi-tierornepho formation:create mediawiki- implies "default"). give some useful processiong output? the stack id to query later? subscribe to sns topic? - formation:destroy - destroy a formation by name (if it is unique) or stack id
- formation:open - open the tier 1 ELB CNAME in a browser (and print it to CLI)
- formation:ssh - open a SSH connection to the bastion host (or if its public, the host specified as primary/bastion in the scenario) and print the address to CLI
- formation:describe - query AWS and describe the state of the running formation