General Enhancements

General Enhancements

• Use same hostname for linked containers (PR #836)

  • This enhancement allows for better consistency between convox start and docker-compose up by using the same hostname entry for the linked contaiers.

• Tag Subnets with a "Name" (PR #828)

  • Allows for better organization and identification of subnets used throughout a rack.

• Get a service's URL (PR #825)

  • With the various convox services supported, sometimes you need easy access to that service's endpoint.
  • convox services url <service> makes that happen:

  $ convox services
  NAME             TYPE    STATUS
  s3-111          s3      running
  
  $ convox services url s3-1111
  s3://misc123:misc123@convox-rack-s3-111
  

  Very script friendly!

• BIG thanks to @adamenger for these improvements!

Fix and Improvements

• Fix cron jobs getting associated with the wrong process type (PR #835)
• Fix to better handle public repository during a build with logs (PR #839)

For more information on this release, please see PR #844.

Support for private git submodules

Private Submodules

With this release, building an app (through Console) with private submodules is now supported.

Build output that was triggered by pushing to a GitHub repo:

convox builds info BRCOWEZXEMP
RUNNING: git clone --progress https://github.com/convox/fake-public-repo.git src
RUNNING: git submodule update --init --recursive
Submodule 'fake-private-repo' (https://github.com/convox/fake-private-repo.git) registered for path 'fake-private-repo'
Submodule path 'fake-private-repo': checked out 'dd41c4403cddb598490cbabc31ddd512556ba1ff'
RUNNING: git checkout b9d76b1ced8f7fa80db19ed3840c33448891cda1
RUNNING: git submodule update --recursive
RUNNING: /usr/local/bin/git-restore-mtime .
RUNNING: docker pull httpd
Using default tag: latest
latest: Pulling from library/httpd
51f5c6a04d83: Pulling fs layer
a3ed95caeb02: Pulling fs layer

Also as a security measure, we no longer log the secret tokens during the build process.

New AWS region support, CLI enhancements and more.

Support for the ap-southeast-2 region

With the availability of AWS Lambda in the Asia Pacific (Sydney) region, Rack now fully supports ap-southeast-2!

Thanks to @rjocoleman for bringing it to our attention and for PR #806!

Enhancements

• Ability to wait for a promotion to finish. This can come in handy when running convox commands within a CI/CD pipeline. For more information, see PR #805. Thanks to @ddollar:

$ convox releases promote RPJBPUCVHUQ -a httpd --wait
Promoting RPJBPUCVHUQ... UPDATING
Waiting for RPJBPUCVHUQ... OK

$ convox deploy --wait
# Some build output here
...
Promoting RGAEABKZYTI... UPDATING
Waiting for RGAEABKZYTI... OK

• Get correct container IP for v2 networks. See PR #795 and thanks to @cleblanc87!
• Two related changes. See PR #803 for more information:
  • Ensure the release created by convox env set uses the currently promoted build
  • Ensure convox run --release uses the correct build and environment.

$convox run --release RUCSSAOGITI "echo $RELEASE"
RUCSSAOGITI

$convox run --release RSOUVVZELSO "echo $RELEASE"
RSOUVVZELSO

Parameter Consolidation

This release consolidates parameters in the CloudFormation stacks for applications. Because CloudFormation stacks are limited to a maximum of 60 parameters this
change allows more process types to be defined for an application.

Previously each process type in your application used three parameters to define scale:

"WebCount": "3",
"WebCpu": "0",
"WebMemory": "512"

After this change, these parameters have been consolidated:

"WebFormation": "3,0,512"

You can now run up to 40 unique process types per application.

Thanks to @jamiequint for feedback and for running an enormous application on Convox :)

New AMIs and Docker Engine; Process CPU Config; App Load Balancer Security; Build Cleanup

Upgrade to Docker 1.11

With Docker finally removing the -f option (thanks to @asyazwan for bringing it to our attention with #781), we decided the best route is to upgrade to Docker 1.11.
This includes the the alpine imaged used by rack itself but also an AMI optimize for ECS Instances:

The current Amazon ECS-optimized AMI (amzn-ami-2016.03.c-amazon-ecs-optimized) consists of:

The latest minimal version of the Amazon Linux AMI
The latest version of the Amazon ECS container agent (1.10.0)
The recommended version of Docker for the latest Amazon ECS container agent (1.11.1)
The latest version of the ecs-init package to run and monitor the Amazon ECS agent (1.10.0-1)

See PR #797 for more information.

Process CPU Configuration

Thanks to @beedub you can now configure CPU shares for any process type.

The default is still the same -- not explicitly changing this setting means that all processes share all of the CPU available on an instance. This is still the sensible default to get the most out of CPUs in the cluster.

But the ability to specify this opens up some more powerful use cases. You'd be able to set the number of CPU units to reserve for for a process.

For example, if you run a single-container web process on a single-core instance type with 512 CPU units specified for that container, and that is the only task running on the container instance, that container could use the full 1,024 CPU unit share at any given time. However, if you launched another copy of the same web process on that container instance, each task would be guaranteed a minimum of 512 CPU units when needed, and each container could float to higher CPU usage if the other container was not using it, but if both tasks were 100% active all of the time, they would be limited to 512 CPU units.

More information is available in the AWS documentation.

See #741 for lots of good discussion about the use cases and implementation.

Custom App Load Balancers Security Groups

Convox handles provisioning and configuring a load balancer based on the docker-compose.yml port mappings:

web:
  image: httpd
  ports:
    - 80:80

By default this load balancer will be exposed to the entire internet. Sometimes we want to tighten this to only allow connections from a single computer or single network.

You can now apply additional restrictions to this load balancer by assigning a custom security group to an app:

# get the VPC ID
$ convox rack
Name     convox
Status   running
Version  20160615203231
Region   us-east-1
Count    3
Type     t2.large

$ convox ps --app convox
ID            NAME      RELEASE                  SIZE  STARTED     COMMAND                           
cde9ac958f36  monitor   20160615203231-20160615  64    4 days ago  /go/bin/init sh -c api/bin/monitor
1845eb6ec788  registry  20160615203231-20160615  128   1 day ago   /docker-entrypoint.sh             
79e05a82d06d  registry  20160615203231-20160615  128   4 days ago  /docker-entrypoint.sh             
541d9b8509cb  web       20160615203231-20160615  128   1 day ago   /go/bin/init sh -c api/bin/web    
e9fd0ed66d41  web       20160615203231-20160615  128   4 days ago  /go/bin/init sh -c api/bin/web 

$ convox exec 541d9b8509cb env --app convox | grep VPC
VPC=vpc-f8006b9c

# create a new security group
$ aws ec2 create-security-group --group-name=office --description="office network only" --vpc-id=vpc-f8006b9c
{
    "GroupId": "sg-3bdaaf40"
}

# restrict ingress to a single subnet
$ aws ec2 authorize-security-group-ingress --group-id=sg-3bdaaf40 --protocol=tcp --port=0-65535 --cidr=24.75.239.0/32

# apply the security group to the app load balancers
$ convox apps params set SecurityGroup=sg-3bdaaf40
Updating parameters... OK

After this, the service will only be accessable to IPs from the specified subnet, blocking access from the internet at large.

See @MiguelMoll's patch on #775 for more information.

Build Cleanup

AWS ECR has a default limit of 500 Docker Images, which you can eventually hit after many builds. Thanks to @awsmsrc on #773, convox now automatically deletes the very oldest builds after a successful new build. This both avoids hitting the service limit and saves money on storage costs.

As a reminder, Convox still retains the last 50 builds and releases to facilitate a rollback from a bad release:

$ convox builds
ID           STATUS    RELEASE      STARTED      ELAPSED  DESC
BOFEDWPIEEB  complete  RLBTJENIMQI  3 days ago   6s           
BVFZQPTJJNV  complete  RTAYZHUSGCJ  1 week ago   22s          
BKYKUURZKQU  complete  RONCLAIMWTP  2 weeks ago  49s          
BFUOFQOMAFI  complete  RDJQJEUAPTP  2 weeks ago  10s          
BHGAKQOJCCA  complete  RQOHWZJYDGG  2 weeks ago  5s       

$ convox releases promote RTAYZHUSGCJ
Promoting RTAYZHUSGCJ... UPDATING

CLI Start, Build, Switch Improvements

Convox Start: Better manifest validation

Thanks to @awsmsrc on #763 and #742, convox start has better validation around the manifest yaml format, and the cron label. Improvements like this help cement convox start as both a development tool and a production verification and configuration tool.

As a reminder, configuring a scheduled task in production is as simple as using the convox.cron label:

web:
  build: .
  labels:
    - convox.port.443.protocol=tls
    - convox.port.443.proxy=true
    - convox.cron.jobs=0 * * * ? rake jobs
  links:
    - database
  ports:
    - 80:4000
    - 443:4001
database:
  image: convox/postgres
  ports:
    - 5432

See the Preparing an Application and Scheduled Tasks docs for more info.

Convox Build: Better .dockerignore support

Thanks to @awsmsrc in #757, the tarball of code that is uploaded during convox build handles .dockerignore exactly the same as docker build. This is accomplished by delegating responsibilities to the docker/archive package.

Convox Switch

Thanks to @ddollar and @nzoschke in #691, convox switch finally offers more strategies to manage multiple racks.

Notably you can export CONVOX_RACK=myorg/staging to set the Rack for a particular terminal session.

You can also add the --rack=myorg/production flag to explicitly pick a Rack for any operation.

See the CLI Reference Guide for information about convox switch, as well as setting up bash completion and a PS1 helper.

With improvements like this, we hope that convox makes it easy to manage all your dev, staging and production environments.

General Rack Improvements

Improve rack operations

This enhancement addresses issues where a rack name could be used in the context of an app name. The fix includes better error handling to let the user know what exactly is going on.

For more information see PR #764 and #767

Bug Fix

For a small time window, limited set of users experienced an issue where they weren't able to deploy their apps. This fix address the problem by using GNU tar during builds.

For more information see PR #769 - Use GNU tar for builds

Convox Start Improvements

@awsmsrc fixed a few good bugs that have been reported in the convox start development tool.

The docker-compose.yml file has improved parsing and validation for labels like convox.cron. And .dockerignore is handled better in development file syncing and uploading during convox build.

This update also includes a security enhancement where the convox client now validates the console.convox.com cert.

convox update to get the latest goodies.

See the following PRs for more info:

• #763 validate cron labels
• #762 take .dockerignore into account when using sync
• #757 use docker archiving tool
• #742 dont report bad yaml errors on start
• #753 always verify the certificate for console.convox.com

Logging and Process Output Improvements

Human friendly logging

This enhancement addresses issue #718. It will add logs for mere humans around instance monitoring, failure and scale events. Filter your logs for why:

$ convox logs --app=convox --filter=why --follow=false --since=2h
2016-06-13T20:23:41Z web:20160613194548-monitor/5a0c98f252bf who="EC2/ASG" what="Launching a new EC2 instance: i-a5f0bc3f" why="At 2016-06-13T20:23:03Z an instance was started in response to a difference between desired and actual capacity, increasing the capacity from 3 to 4."
2016-06-13T20:42:54Z agent:0.69/i-a5f0bc3f who="convox/agent" what="marked i-a5f0bc3f unhealthy" why="docker signal: killed"
2016-06-13T20:47:29Z web:20160613194548-monitor/a76ecf0a4d4d who="EC2/ASG" what="Launching a new EC2 instance: i-5a034cc0" why="At 2016-06-13T20:43:33Z an instance was started in response to a difference between desired and actual capacity, increasing the capacity from 2 to 3."
2016-06-13T21:11:47Z web:20160613194548-monitor/a76ecf0a4d4d who="convox/monitor" what="marked instance i-98f86904 unhealthy" why="ECS reported agent disconnected"
2016-06-13T21:12:29Z web:20160613194548-monitor/a76ecf0a4d4d who="EC2/ASG" what="Terminating EC2 instance: i-98f86904" why="At 2016-06-13T21:12:19Z an instance was taken out of service in response to a user health-check."
2016-06-13T21:17:29Z web:20160613194548-monitor/a76ecf0a4d4d who="EC2/ASG" what="Launching a new EC2 instance: i-319f0ead" why="At 2016-06-13T21:12:48Z an instance was started in response to a difference between desired and actual capacity, increasing the capacity from 2 to 3."

For more info see PR #759 - Human friendly logging about why instances are replaced

Improving convox ps reliability

The output of convox ps has been improved, with the goal being to give users accurate and reliable information about running processes. Especially during an application deployment.

For more information see PR #754 - Improving convox ps reliability

DynamoDB, Git and Autoscaling Improvements

DynamoDB Indexes Enhancements

For a majority of users, the decreased provisioned throughput for build and release DynamoDB tables will result in a monthly savings!

Thanks @frsechet for the improvement!
For more info see PR #736: changed provisioned throughput for DynamoDB indexes.

Git Submodules Checkout

This enhancement allows convox build to pull/update a repo's submodules, if any.

Thanks @mfine for the improvement!
For more information see PR #732: Update submodules on git checkout

Autoscaling Improvements

PR #737 made some improvements for scaling a rack. With the rack Autoscale parameter enabled,
there is no need to use the --count option to increase the instance count. A Rack can scale its own instance count based on the needs of the containers it provisions.