Curator in docker

This is dockerized version of elasticsearch curator,tool to manage time-based indices.

the docker image for curator baseon the python:3.6-alpine(this image size only 30mb)

docker hub https://hub.docker.com/r/anjia0532/docker-curator/

Elastic Curator Official Image : untergeek/curator

My Curator Image : anjia0532/docker-curator

Curator is breaking into version dependent releases. Curator 6.x will work with Elasticsearch 6.x, Curator 7.x will work with Elasticsearch 7.x, Curator 8.x will work with Elasticsearch 8.x

Usage

docker-compose.yml

version: '2'

services:

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:x.y.z
    ports:
    - 9200:9200/tcp
    #environment:
    #    - http.host=0.0.0.0
    #    - transport.host=127.0.0.1
    # Uncomment this section to have elasticsearch data loaded from a volume
    #volumes:
    #    - /data/path/to/host/:/usr/share/elasticsearch/data

  # default version :latest 
  # more information https://hub.docker.com/repository/docker/anjia0532/docker-curator/tags?page=1&ordering=last_updated
  curator:
    image: anjia0532/docker-curator
    environment:
        UNIT_COUNT: 10
        UNIT: days
        ES_HOST: elasticsearch
    depends_on:
      - elasticsearch
    volumes:
        - /config/path/to/host/:/config

action_file.yml(default)

env default value

• UNIT_COUNT: 1
• UNIT: months
• ES_HOST: 127.0.0.1

actions:
  1:
    action: delete_indices
    description: >-
       Delete indices older than ${UNIT_COUNT:1} ${UNIT:months} (based on index name), for logstash-
       prefixed indices. Ignore the error if the filter does not result in an
       actionable list of indices (ignore_empty_list) and exit cleanly.
    options:
      ignore_empty_list: True
      timeout_override:
      continue_if_exception: True
      disable_action: False
    filters:
    - filtertype: pattern
      kind: prefix
      value: logstash-
      exclude:
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: ${UNIT:months}
      unit_count: ${UNIT_COUNT:1}
      exclude:
#  2:
#    action: delete_indices
#    description: >-
#       Delete indices older than ${UNIT_COUNT:1} ${UNIT:months} (based on index name), for filebeat-
#       prefixed indices. Ignore the error if the filter does not result in an
#       actionable list of indices (ignore_empty_list) and exit cleanly.
#    options:
#      ignore_empty_list: True
#      timeout_override:
#      continue_if_exception: True
#      disable_action: False
#    filters:
#    - filtertype: pattern
#      kind: prefix
#      value: filebeat-
#      exclude:
#    - filtertype: age
#      source: name
#      direction: older
#      timestring: '%Y.%m.%d'
#      unit: ${UNIT:months}
#      unit_count: ${UNIT_COUNT:1}
#      exclude:
#  3:
#    action: delete_indices
#    description: >-
#       Delete indices older than ${UNIT_COUNT:1} ${UNIT:months} (based on index name), for heartbeat-
#       prefixed indices. Ignore the error if the filter does not result in an
#       actionable list of indices (ignore_empty_list) and exit cleanly.
#    options:
#      ignore_empty_list: True
#      timeout_override:
#      continue_if_exception: True
#      disable_action: False
#    filters:
#    - filtertype: pattern
#      kind: prefix
#      value: heartbeat-
#      exclude:
#    - filtertype: age
#      source: name
#      direction: older
#      timestring: '%Y.%m.%d'
#      unit: ${UNIT:months}
#      unit_count: ${UNIT_COUNT:1}
#      exclude:
#  4:
#    action: delete_indices
#    description: >-
#       Delete indices older than ${UNIT_COUNT:1} ${UNIT:months} (based on index name), for heartbeat-
#       prefixed indices. Ignore the error if the filter does not result in an
#       actionable list of indices (ignore_empty_list) and exit cleanly.
#    options:
#      ignore_empty_list: True
#      timeout_override:
#      continue_if_exception: True
#      disable_action: False
#    filters:
#    - filtertype: pattern
#      kind: prefix
#      value: zipkin-
#      exclude:
#    - filtertype: age
#      source: name
#      direction: older
#      timestring: '%Y.%m.%d'
#      unit: ${UNIT:months}
#      unit_count: ${UNIT_COUNT:1}
#      exclude:

config_file.yml

env default value

• USE_SSL: False
• TIMEOUT: 120
• MASTER_ONLY: True

---
# Remember, leave a key empty if there is no value.  None will be a string,
# not a Python "NoneType"
# old version 5.x-7.x
client:
  hosts:
    - ${ES_HOST:127.0.0.1}
  port: 9200
  url_prefix:
  use_ssl: ${USE_SSL:False}
  certificate:
  client_cert:
  client_key:
  ssl_no_validate: False
  timeout: ${TIMEOUT:120}
  master_only: ${MASTER_ONLY:True}

# curator 8.0+ version
elasticsearch:
  client:
    hosts: ${ES_HOST}
    cloud_id:
    bearer_auth:
    opaque_id:
    request_timeout: ${TIMEOUT:120}
    http_compress:
    verify_certs: False
    ca_certs:
    client_cert:
    client_key:
    ssl_assert_hostname:
    ssl_assert_fingerprint:
    ssl_version:
  other_settings:
    master_only: ${MASTER_ONLY:True}
    skip_version_test:
    username:
    password:
    api_key:
      id:
      api_key:

logging:
  loglevel: INFO
  logfile:
  logformat: default
  #blacklist: ['elasticsearch', 'urllib3']

cron job

every day at 0:00 am exec

0 0 * * * curator --config /config/config_file.yml /config/action_file.yml

Curator's doc

Full reference is available at: https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html

• Configuration
  • Environment Variables
  • Action File
  • Configuration File
• Actions
  • ...
  • Delete Indices
  • ...
• ...
• Examples
  • alias
  • allocation
  • close
  • cluster_routing
  • create_index
  • delete_indices
  • delete_snapshots
  • forcemerge
  • index_settings
  • open
  • reindex
  • replicas
  • restore
  • rollover
  • snapshot
• ...