Skip to content

Deployment with Docker

Jump to bottom
whotwagner edited this page Mar 18, 2022 · 21 revisions

This article describes how to build and run the logdata-anomaly-miner inside a docker-container. It will not explain how to configure the logdata-anomaly-miner. For further information about running the logdata-anomaly-miner, please read the Getting Started.

All docker-run commands in this article use the parameter "--rm". The container will be deleted after a run.

Building the docker-image

Download the repository

$ git clone -b development https://github.com/ait-aecid/logdata-anomaly-miner.git

Build the Image

Please note that in this example the current version of logdata-anomaly-miner was 2.1.0. Use the version-number that you downloaded!

$ cd logdata-anomaly-miner
$ docker build -t aecid/logdata-anomaly-miner:latest -t aecid/logdata-anomaly-miner:2.1.0 .

For newer versions of logdata-anomaly-miner use the included build-script instead:

$ cd logdata-anomaly-miner
$ scripts/build_docker.sh

First run

Next we will simply start the aminer and give out the version string.

First let's copy and prepare the configuration directory:

$ cp -r source/root/etc/aminer/ aminercfg
$ cp aminercfg/template_config.yml aminercfg/config.yml
$ mkdir logs
$ mkdir persistency

Run aminer:

$ docker run -v $PWD/aminercfg:/etc/aminer -v $PWD/persistency:/var/lib/aminer -v $PWD/logs:/logs --rm aecid/logdata-anomaly-miner aminer --version

docker-usage

Please note that we did not configure the aminer properly. It won't parse any logs with this template configuration

Usage

Mountpoints/Volumes

In order to run the logdata-anomaly-miner in production, we need to configure some volumes:

Mountpoint Description Access-Mode
/etc/aminer The configuration directory ro
/var/lib/aminer The directory for the persistency rw
/logs The directory where the logdata-anomaly-miner finds the logresources ro

Run logdata-anomaly-miner with supervisord

If you run logdata-anomaly-miner with supervisord then you might want to mount /var/lib/supervisor in order to access the supervisor-unix-socket:

docker run -v $PWD/aminercfg:/etc/aminer -v $PWD/persistency:/var/lib/aminer -v $PWD/logs:/logs -v $PWD/supervisor:/var/lib/supervisor --rm -it aecid/logdata-anomaly-miner supervisor

It is also possible to run the supervisord using the http-interface. We need a configuration file for that.

#./http.conf
[inet_http_server]
port=:9001

This file needs to be mounted into the logdata-anomaly-miner-container:

docker run -v $PWD/aminercfg:/etc/aminer -v $PWD/persistency:/var/lib/aminer -v $PWD/logs:/logs -v $PWD/supi.conf:/etc/supervisor/conf.d/http.conf --rm -it aecid/logdata-anomaly-miner supervisor

Runmodes

Default

We can run the container without any command:

$ docker run -v $PWD/aminercfg:/etc/aminer -v $PWD/persistency:/var/lib/aminer -v $PWD/logs:/logs --rm aecid/logdata-anomaly-miner

This will execute the following command inside the container:

$ aminer --Config /etc/aminer/config.yml

Aminer with custom parameters

We can run the aminer with custom parameters. In the following example we execute the logdata-anomaly-miner with the command line parameter "--version":

$ docker run -v $PWD/aminercfg:/etc/aminer -v $PWD/persistency:/var/lib/aminer -v $PWD/logs:/logs  --rm aecid/logdata-anomaly-miner aminer --Help
            *     (        )       (     
   (      (  `    )\ )  ( /(       )\ )  
   )\     )\))(  (()/(  )\()) (   (()/(  
((((_)(  ((_)()\  /(_))((_)\  )\   /(_)) 
 )\ _ )\ (_()((_)(_))   _((_)((_) (_))   
 (_)_\(_)|  \/  ||_ _| | \| || __|| _ \  
  / _ \  | |\/| | | |  | .` || _| |   /  
 /_/ \_\ |_|  |_||___| |_|\_||___||_|_\  
   (Austrian Institute of Technology)
       (https://aecid.ait.ac.at)
            Version: 2.1.0

usage: aminer.py [options]
options:
  -c, --Config <config-file>          	path to the config-file
  -d, --Daemon                        	run as a daemon process
  -r, --RunAnalysis                   	enable/disable analysis
  -R, --Remove <persistence-directory>	removes a specific persistence directory
  -C, --Clear                         	removes all persistence directories
  -f, --FromBegin                     	removes RepositioningData before starting the AMiner
  -h, --Help                          	print this print_help screen
  -v, --Version                       	print version-string

AminerRemoteControl

We can also run the aminerremotecontrol with custom parameters:

$ docker run -v $PWD/aminercfg:/etc/aminer -v $PWD/persistency:/var/lib/aminer -v $PWD/logs:/logs --rm aecid/logdata-anomaly-miner aminerremotecontrol --Help
Usage: /usr/lib/logdata-anomaly-miner/aminerremotecontrol.py [arguments]
  --ControlSocket [socketpath]: when given, use nonstandard control socket.
  --Data [data]: provide this json serialized data within execution
    environment as 'remote_control_data' (see man page).
  --Exec [command]: add command to the execution list, can be
    used more than once.
  --ExecFile [file]: add commands from file to the execution list
    in same way as if content would have been used with "--Exec".
  --Help: this output
  --StringResponse: if set, print the response just as string
    instead of passing it to repr.

  For further information read the man pages running 'man AMinerRemoteControl'.

Configuration

The /etc/aminer-directory can be mounted via a volume into the container. The docker container supports all features of the logdata-anomaly-miner and can be configured almost the same as logdata-anomaly-miner. There is only one difference: symbolic links might not work inside the container therefore it might be necessary to copy the files from conf-available to conf-enabled. See Getting Started for further informations about the configuration.

Building the documentation

It is possible to create a documentation of logdata-anomaly-miner using the docker-image:

$ mkdir _build
$ docker run --rm -v _build:/docs/_build aecid/logdata-anomaly-miner mkdocs
Clone this wiki locally