keen_loki_logger - Elixir Logger backend for Loki service

keen_loki_logger is an Elixir logger backend, based on original LokiLogger, which development is dead from 2019, providing support for Logging to Grafana Loki

Known issues

• "works-on-my-machine" level of quality, but we use it in a big enterprise company. Post a feedback in the repo's Github issues.

Features (and TODO)

Some of the features were reworked, some implemented and some are still missing. The main part is there though.

• Elixir Logger formatting
• Elixir Logger metadata
• Loki Scope-Org-Id header for multi-tenancy
• Timezone aware
• Snappy compressed proto format in the HTTP Body
• Async http call to backend. This needs to be properly tested and reviewed, though.
• Proper unit tests.
• HTTP post retry strategy on temporary loki backend failure or network hiccups.
• Authentication with basic auth.
• Authentication with Oauth2 reverse proxy

Installation

The package can be installed by adding keen_loki_logger to your list of dependencies in mix.exs:

def deps do
  [
    {:keen_loki_logger, "~> 0.5.1"}
  ]
end

STRUGGLES ON WINDOWS MACHINES

Caution

IT IS ABSOLUTELY CRUCIAL TO RUN EVERYTHING, INCLUDING THE FIRST mix deps.get IN X64 COMMAND LINE, OTHERWISE YOU'LL BE STUCK IN DEPS BEING DOWNLOADED FOR X86 ENVIRONMENT, BUT LATER TRYING TO BE COMPILED BY rebar3 TO X64, WHICH FAILS WITH NO VISIBLE ERROR AND DIAGNOSTIC=1 HAS TO BE SET TO INVESTIGATE IT IF YOU DOWNLOADED DEPS IN X86 TERMINAL, DELETE BOTH deps and _build FOLDERS

On Windows machines you might have issues compiling your project because of snappyer package that requires C/++ code compilation. Use these steps to be able to compile it without issues:

• Use command line and not PowerShell!
• Install [Microsoft C++ toolset](https://learn.microsoft.com/en-us/cpp/build/building-on-the-command-line?view=msvc-170#download-and-install-the-tools], you don't have to go full C++ development, just the compilation toolset is fine
• Add this c:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\bin\Hostx64\x64\ or similar path (based on your installed version) to PATH environment variable, this will add cl.exe to path
• Before compilation run "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat" which you should have by now. This setups all environment variables needed for compilation of C/++ code, including standard .h files

We have this script in our project to setup the environment.

setup.bat

call "c:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat"

mix deps.get && mix deps.compile

If you still have issues

• run mix in debug mode of mix by setting environment variable MIX_DEBUG=1
• and debug mode of rebar3 by setting environment variable DIAGNOSTIC=1

Configuration

Elixir Project

Loki Logger's behavior is controlled using the application configuration environment:

• loki_host : [OPTIONAL] the hostname of the syslog server, default http://localhost:3100

• loki_path : [OPTIONAL] path to the endpoint, default /loki/api/v1/push

• loki_labels : [OPTIONAL] the Loki log labels used to select the log stream in, default %{application: "loki_logger_library"}

• loki_scope_org_id: [OPTIONAL] optional tenant ID for multitenancy, default fake, which is a standard Loki value when you have just one tenant

• level: [OPTIONAL] logging threshold. Messages "above" this threshold will be discarded. The supported levels, ordered by precedence are :debug, :info, :warn, :error.

• format: [OPTIONAL] the format message used to print logs. Defaults to: "$metadata level=$level $levelpad$message". It may also be a {module, function} tuple that is invoked with the log level, the message, the current timestamp and the metadata.

• metadata: [OPTIONAL] the metadata to be printed by $metadata. Defaults to :all, which prints all metadata to the message.

• max_buffer: [OPTIONAL] the amount of entries to buffer before posting to the Loki REST api. Defaults to 32.

• finch_protocols: [OPTIONAL] the HTTP protocol to use when communicating with Loki, supported values are :http1 and :http2, default :http1.

• finch_pool_size: [OPTIONAL] the amount of HTTP connection pools to create, applicable only for HTTP1, for HTTP2 the value is always set to 1, because HTTP2 is by itself multiplexed, default 16.

• finch_pool_count: [OPTIONAL] the amount of HTTP connection pools to create, default 4.

• finch_pool_max_idle_time: [OPTIONAL] I have no idea what this does :-)

• mint_conn_opts: [OPTIONAL] Mint is the low-level HTTP client library and this is the way to set it up. We use it, for example, to setup CA certificate for internal servers.

For example, the following config/config.exs file sets up Loki Logger using level debug, with application label loki_logger_library.

config.exs

use Mix.Config

config :logger,
       backends: [LokiLogger]

config :logger, :keen_loki_logger,
  level: :debug,
  format: "$metadata level=$level $message",
  metadata: [:erl_level, :application, :file, :module, :function],
  max_buffer: 300,
  loki_host: "https://1.2.3.4:3100",
  basic_auth_user: "lokiuser",
  basic_auth_password: "lokipassword",
  finch_protocols: [:http2],
  mint_conn_opts: [transport_opts: [cacerts: :public_key.cacerts_get()]]

runtime.exs

config :logger, :keen_loki_logger,
  loki_labels: %{app: "dummy_logging_service", env: System.get_env("APP_ENV"), host: System.get_env("COMPUTER_NAME")}

Protobuff lib regeneration

only needed for development

protoc --proto_path=./lib/proto --elixir_out=./lib lib/proto/loki.proto 

License

The source code is released under the Apache v2.0 License.

Check LICENSE for more information.