Skip to content

Commit

Permalink
feat: enhancing loki logging capture (#1310)
Browse files Browse the repository at this point in the history
  • Loading branch information
rodrigo-o authored Sep 30, 2024
1 parent e10c578 commit 4e5afab
Show file tree
Hide file tree
Showing 6 changed files with 42 additions and 5 deletions.
12 changes: 12 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -166,14 +166,26 @@ test-iex:
checkpoint-sync: compile-all
iex -S mix run -- --checkpoint-sync-url https://mainnet-checkpoint-sync.stakely.io/ --metrics

#▶️ checkpoint-sync.logfile: @ Run an interactive terminal using checkpoint sync with a log file.
checkpoint-sync.logfile: compile-all
iex -S mix run -- --checkpoint-sync-url https://mainnet-checkpoint-sync.stakely.io/ --metrics --log-file ./logs/mainnet.log

#▶️ sepolia: @ Run an interactive terminal using sepolia network
sepolia: compile-all
iex -S mix run -- --checkpoint-sync-url https://sepolia.beaconstate.info --network sepolia --metrics

#▶️ sepolia.logfile: @ Run an interactive terminal using sepolia network with a log file
sepolia.logfile: compile-all
iex -S mix run -- --checkpoint-sync-url https://sepolia.beaconstate.info --network sepolia --metrics --log-file ./logs/sepolia.log

#▶️ holesky: @ Run an interactive terminal using holesky network
holesky: compile-all
iex -S mix run -- --checkpoint-sync-url https://checkpoint-sync.holesky.ethpandaops.io --network holesky

#▶️ holesky.logfile: @ Run an interactive terminal using holesky network with a log file
holesky.logfile: compile-all
iex -S mix run -- --checkpoint-sync-url https://checkpoint-sync.holesky.ethpandaops.io --network holesky --log-file ./logs/holesky.log

#▶️ gnosis: @ Run an interactive terminal using gnosis network
gnosis: compile-all
iex -S mix run -- --checkpoint-sync-url https://checkpoint.gnosischain.com --network gnosis
Expand Down
6 changes: 3 additions & 3 deletions config/runtime.exs
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ case Keyword.get(args, :log_file) do
nil ->
# Use custom formatter for prettier logs
config :logger, :default_formatter,
format: {ConsoleLogger, :format},
format: {CustomConsoleLogger, :format},
metadata: [:slot, :root, :bits]

log_file ->
Expand All @@ -241,9 +241,9 @@ case Keyword.get(args, :log_file) do
config :logger, utc_log: true

config :logger, :default_formatter,
format: {LogfmtEx, :format},
format: {CustomLogfmtEx, :format},
colors: [enabled: false],
metadata: [:mfa, :slot]
metadata: [:mfa, :pid, :slot, :root]

config :logfmt_ex, :opts,
message_key: "msg",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
defmodule ConsoleLogger do
defmodule CustomConsoleLogger do
@moduledoc """
Custom logger formatter for console output.
"""
Expand Down
19 changes: 19 additions & 0 deletions lib/lambda_ethereum_consensus/logger/custom_logfmt_ex.ex
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
defmodule CustomLogfmtEx do
@moduledoc """
Custom logger formatter for logfmt output.
"""

alias LambdaEthereumConsensus.Utils

def format(level, message, timestamp, metadata) do
formatted_metadata = format_metadata(metadata)

LogfmtEx.format(level, message, timestamp, formatted_metadata)
end

defp format_metadata(metadata) do
metadata
|> Keyword.replace_lazy(:root, &Utils.format_shorten_binary(&1))
|> Keyword.replace_lazy(:bits, &Utils.format_bitstring(&1))
end
end
2 changes: 1 addition & 1 deletion metrics/loki/loki.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ schema_config:
period: 24h

ruler:
alertmanager_url: http://localhost:9093
alertmanager_url: http://localhost:9093/alertmanager
# By default, Loki will send anonymous, but uniquely-identifiable usage and configuration
# analytics to Grafana Labs. These statistics are sent to https://stats.grafana.org/
#
Expand Down
6 changes: 6 additions & 0 deletions metrics/promtail/promtail.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,18 @@ scrape_configs:
msg:
mfa:
process: registered_name
pid:
slot:
root:
- timestamp:
format: "2006-01-02T15:04:05.000"
source: ts
- labels:
level:
mfa:
process:
pid:
slot:
root:
- output:
source: msg

0 comments on commit 4e5afab

Please sign in to comment.