How to elegantly configure a Logger with a configuration file while supporting log rotation

[jiaopengzi]

Problem Description

• Function: Test1() is a log rotation and cutting library gopkg.in/natefinch/lumberjack.v2 recommended in the official documentation.
• Function: Test2() is a Logger that uses yaml to read configuration based on the basic configuration in the official documentation.

After executing the main function,

in the console output:

2023-05-15T08:49:16.555+0800 | INFO | logger construction succeeded:config from yaml | {"app": "jpz"}

in the log file foo.log output:

{"level":"info","ts":1684111756.5545945,"msg":"logger construction succeeded:lumberjack.Logger"}

These two logs are definitely different.

My current requirements:

1. Both support using the configuration file config_log_zap.yaml so that all configurations can take effect, and let lumberjack complete the log rotation and splitting work.

2. The output of both the console and the log file should be the same, so that I can quickly apply the desired content through the configuration file. The reason why both the console and the log file are needed is because I need to pay attention to and record past output messages during development.

   In the console output:

   2023-05-15T08:49:16.555+0800 | INFO | logger construction succeeded:config from yaml | {"app": "jpz"}

   In the log file foo.log output:

   2023-05-15T08:49:16.555+0800 | INFO | logger construction succeeded:config from yaml | {"app": "jpz"}

3. How should I merge Test1() and Test2() into one function Test0() to meet the above two requirements?

Please give me some help, I've been studying it for a long time.

main.go

package main

import (
	"gopkg.in/yaml.v3"
	"os"

	"go.uber.org/zap"
	"go.uber.org/zap/zapcore"
	"gopkg.in/natefinch/lumberjack.v2"
)

func Test1() {
	// lumberjack.Logger is already safe for concurrent use, so we don't need to
	// lock it.
	w := zapcore.AddSync(&lumberjack.Logger{
		Filename:   "./foo.log",
		MaxSize:    500, // megabytes
		MaxBackups: 3,
		MaxAge:     28, // days
	})
	core := zapcore.NewCore(
		zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig()),
		w,
		zap.InfoLevel,
	)
	logger := zap.New(core)
	logger.Info("logger construction succeeded:lumberjack.Logger")
}

func Test2() {
	var cfg zap.Config
	yamlFile, _ := os.ReadFile("./config_log_zap.yaml")
	if err := yaml.Unmarshal(yamlFile, &cfg); err != nil {
		panic(err)
	}

	logger := zap.Must(cfg.Build())
	defer logger.Sync()

	logger.Info("logger construction succeeded:config from yaml")
}

func main() {
	Test1()
	Test2()
}

config_log_zap.yaml

# 日志级别，支持 "debug", "info", "warn", "error", "dpanic", "panic" 和 "fatal"
level: "debug"

# 是否处于开发模式，开发模式下 DPanicLevel 行为会变化，更多地捕获堆栈跟踪
development: true

# 是否禁用调用者信息，如调用函数的文件名和行号
disableCaller: true

# 是否禁用自动捕获堆栈跟踪
disableStacktrace: false

# 采样设置
sampling:
  initial: 100
  thereafter: 100

encoding: "console"

# 所选编码的详细配置
encoderConfig:
  messageKey: "msg"                # 消息键名
  levelKey: "level"                # 日志级别键名
  timeKey: "ts"                    # 时间戳键名
  nameKey: "logger"                # 日志记录器名称键名
  callerKey: "caller"              # 调用者信息键名
  functionKey: "function"          # 函数名键名
  stacktraceKey: "stacktrace"      # 堆栈跟踪键名
  skipLineEnding: false            # 是否跳过行结束符
  lineEnding: "\n"                 # 行结束符
  levelEncoder: "capital"          # 级别编码器，支持 "lower"、"upper"、"capital" 和 "capitalize"
  timeEncoder: "iso8601"           # 时间编码器，支持 "iso8601"、"millis"、"nanos" 和 "epoch"
  durationEncoder: "string"        # 持续时间编码器，支持 "string"、"nanos" 和 "secs"
  callerEncoder: "full"            # 调用者编码器，支持 "short" 和 "full"
  nameEncoder: "full"              # 日志记录器名称编码器，支持 "short" 和 "full"
  consoleSeparator: " | "          # 控制台日志记录器分隔符


outputPaths:
  - "stdout"                                # stdout（标准输出）控制台输出
  - "./foo.log"                                # 文件输出

# 内部日志记录器错误输出路径列表，默认为 "stderr"
errorOutputPaths:
  - "stderr"                                # stderr（标准错误）控制台输出
  - "./error_logs"                          # 文件输出

# 根日志记录器的初始字段集合
initialFields:
  app: "jpz"

[jiaopengzi]

There is an answer on Stackoverflow .
It meets my needs.