Skip to content

Ruby method or shell command runner in container (ECS, docker)

Notifications You must be signed in to change notification settings

reproio/wrapbox

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Wrapbox

Wrapbox runs Ruby method or shell command in a container (ECS, docker).

Installation

Add this line to your application's Gemfile:

gem 'wrapbox'

And then execute:

$ bundle

Or install it yourself as:

$ gem install wrapbox

Usage

Write config.yml

default:
  cluster: wrapbox
  runner: ecs
  region: ap-northeast-1
  container_definitions:
    - image: joker1007/wrapbox
      cpu: 512
      memory: 1024
      essential: true

docker:
  runner: docker
  keep_container: true
  container_definitions:
    - image: joker1007/wrapbox
      cpu: 512
      memory: 1024

ecs2:
  cluster: wrapbox
  runner: ecs
  region: ap-northeast-1
  # Use already existing task definition
  task_definition:
    task_definition_name: foo_task:1
    main_container_name: container_name_where_command_is_executed

run by CLI

$ wrapbox ecs run_cmd -f config.yml \
  -e "FOO=bar,HOGE=fuga" \
  "bundle exec rspec spec/models" \
  "bundle exec rspec spec/controllers" \

run by ruby

Run rake wrapbox:run with CLASS_NAME_ENV and METHOD_NAME_ENV and METHOD_ARGS_ENV

Wrapbox.configure do |c|
  c.load_yaml(File.expand_path("../config.yml", __FILE__))
end

# runs TestJob#perform("arg1", ["arg2", "arg3"]) in ECS container via `rake wrapbox:run`
Wrapbox.run("TestJob", :perform, ["arg1", ["arg2", "arg3"]], environments: [{name: "RAILS_ENV", value: "development"}]) # use default config
# runs TestJob#perform in local docker container (Use docker cli)
Wrapbox.run("TestJob", :perform, ["arg1", ["arg2", "arg3"]], config_name: :docker, environments: [{name: "RAILS_ENV", value: "development"}]) # use docker config

# runs ls . command in ECS container
Wrapbox.run_cmd(["ls ."], environments: [{name: "RAILS_ENV", value: "development"}])

If ECS runner cannot create task, it puts custom metric data to CloudWatch. Custom metric data is wrapbox/WaitingTaskCount that has ClusterName dimension. And, it retry launching until retry count reach launch_retry.

After task exited, Wrapbox checks main container exit code. If exit code is not 0, Wrapbox raise error.

Config

Common

name desc
runner "ecs" or "docker"

for ECS

name desc
cluster target ECS cluster name
region region of ECS cluster
container_definitions see http://docs.aws.amazon.com/sdkforruby/api/Aws/ECS/Client.html#register_task_definition-instance_method
task_role_arn see http://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html
volumes see http://docs.aws.amazon.com/sdkforruby/api/Aws/ECS/Client.html#register_task_definition-instance_method
placement_constraints see http://docs.aws.amazon.com/sdkforruby/api/Aws/ECS/Client.html#register_task_definition-instance_method
placement_strategy see http://docs.aws.amazon.com/sdkforruby/api/Aws/ECS/Client.html#register_task_definition-instance_method
launch_type see http://docs.aws.amazon.com/sdkforruby/api/Aws/ECS/Client.html#run_task-instance_method
network_mode see http://docs.aws.amazon.com/sdkforruby/api/Aws/ECS/Client.html#register_task_definition-instance_method
network_configuration see http://docs.aws.amazon.com/sdkforruby/api/Aws/ECS/Client.html#run_task-instance_method
capacity_provider_strategy see https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/ECS/Client.html#run_task-instance_method
cpu see http://docs.aws.amazon.com/sdkforruby/api/Aws/ECS/Client.html#register_task_definition-instance_method
memory see http://docs.aws.amazon.com/sdkforruby/api/Aws/ECS/Client.html#register_task_definition-instance_method
enable_ecs_managed_tags see https://docs.aws.amazon.com/sdkforruby/api/Aws/ECS/Client.html#run_task-instance_method
tags tags of task definitions. see also https://docs.aws.amazon.com/sdkforruby/api/Aws/ECS/Client.html#register_task_definition-instance_method
propagate_tags specify "TASK_DEFINITION" if you want to propagate tags to tasks. see also https://docs.aws.amazon.com/sdkforruby/api/Aws/ECS/Client.html#run_task-instance_method
launch_instances specify launch_template (required), instance_type, and tag_specifications for Aws::EC2::Client#run_instances. You can also specify wait_until_instance_terminated (default: true)

WRAPBOX_CMD_INDEX environment variable is available in run_cmd and you can distinguish logs from each command like below:

log_configuration:
  log_driver: syslog
  options:
    syslog-address: "tcp://192.168.0.42:123"
    env: WRAPBOX_CMD_INDEX
    tag: wrapbox-{{ printf "%03s" (.ExtraAttributes nil).WRAPBOX_CMD_INDEX }}

for docker

name desc
container_definitions only use image, cpu, memory, and memory_reservation
keep_container If true, doesn't delete the container when the command ends

API

Wrapbox.run

Wrapbox.run(class_name, method_name, args,
  runner: nil, # The "runner" value is used in the configuration  if it is nil.
  config_name: nil, # "default" configuration is used if it is nil.
  cluster: nil, # Available only for ECS runner. The "cluster" value in the configuration is used if it is nil.
  launch_type: "EC2", # Available only for ECS runner. The "launch_type" value in the configuration is used if it is nil.
  task_role_arn: nil, # Available only for ECS runner. The "task_role_arn" value in the configuration is used if it is nil.
  execution_role_arn: nil, # Available only for ECS runner. The "execution_role_arn" value in the configuration is used if it is nil.
  tags: nil, # Available only for ECS runner. The "tags" value in the configuration is used if it is nil.
  propagate_tags: nil, # Available only for ECS runner. The "propagate_tags" value in the configuration is used if it is nil.
  container_definition_overrides: {},
  environments: [],
  timeout: 3600 * 24, # Available only for ECS runner. # Available only for ECS runner.
  launch_timeout: 60 * 10, # Available only for ECS runner.
  launch_retry: 10, # Available only for ECS runner.
  retry_interval: 1, # Available only for ECS runner.
  retry_interval_multiplier: 2, # Available only for ECS runner.
  max_retry_interval: 120, # Available only for ECS runner.
  execution_retry: 0, # Available only for ECS runner.
  keep_container: nil, # Available only for Docker runner. The "keep_container" value in the configuration is used if it is nil.
)

Wrapbox.run_cmd

Wrapbox.run_cmd(*cmd,
  runner: nil, # The "runner" value is used in the configuration  if it is nil.
  config_name: nil, # "default" configuration is used if it is nil.
  cluster: nil, # Available only for ECS runner. The "cluster" value in the configuration is used if it is nil.
  launch_type: "EC2", # Available only for ECS runner. The "launch_type" value in the configuration is used if it is nil.
  task_role_arn: nil, # Available only for ECS runner. The "task_role_arn" value in the configuration is used if it is nil.
  execution_role_arn: nil, # Available only for ECS runner. The "execution_role_arn" value in the configuration is used if it is nil.
  tags: nil, # Available only for ECS runner. The "tags" value in the configuration is used if it is nil.
  propagate_tags: nil, # Available only for ECS runner. The "propagate_tags" value in the configuration is used if it is nil.
  container_definition_overrides: {},
  ignore_signal: false,
  environments: [],
  timeout: 3600 * 24, # Available only for ECS runner. # Available only for ECS runner.
  launch_timeout: 60 * 10, # Available only for ECS runner.
  launch_retry: 10, # Available only for ECS runner.
  retry_interval: 1, # Available only for ECS runner.
  retry_interval_multiplier: 2, # Available only for ECS runner.
  max_retry_interval: 120, # Available only for ECS runner.
  execution_retry: 0, # Available only for ECS runner.
  keep_container: nil, # Available only for Docker runner. The "keep_container" value in the configuration is used if it is nil.
)

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

How to test

The following environment variables are required to run all tests.

Name Description
RUN_AWS_SPECS Set "true" to run tests with aws set to true. You should also set credentials for AWS account to run ECS tasks.
ECS_CLUSTER A cluster used in tests. "default" cluster is used if this variable is not set.
OVERRIDDEN_ECS_CLUSTER A cluster used in tests that ensure cluster parameter.
LAUNCH_TEMPLATE_ID A launch template used in tests that ensure launch_instances configuration.
env \
  RUN_AWS_SPECS=true \
  ECS_CLUSTER='some_cluster' \
  OVERRIDDEN_ECS_CLUSTER='another_cluster' \
  LAUNCH_TEMPLATE_ID=lt-xxxxxxxxxxxxxxxxx \
bundle exec rspec

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/reproio/wrapbox.

About

Ruby method or shell command runner in container (ECS, docker)

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages