v3 Started

[natefinch]

I started v3 of lumberjack a few days ago. 🎉

Alpha is Available

Already there's a sketch of how it'll look in the /v3 branch (which you can import as "github.com/natefinch/lumberjack/v3" and use go get github.com/natefinch/lumberjack/v3@latest to add it to your current project. There's a v3.0.0-alpha tag on that branch. All the old tests pass, but it's definitely alpha and you should expect the API to change.

Backwards Compatibility is Paramount

My promise to you is that if you currently have an application using lumberjack v2, you'll be able to upgrade to v3, with v3 properly handling any v2 backup files etc. This will require that you configure v3 to mimic the behavior of v2, of course, but I'll make sure that's easy and well-documented.

New API

Lumberjack was one of the first Go libraries I wrote, and the fact that I almost immediately needed to release a v2 is kind of indicative of my immaturity with the language at the time. And the update to v3 mostly is about fixing the API mistakes I made in v2. 😆

For example, MaxSize of the file is just an int64 representing number of bytes (instead of integer number of megabytes), and MaxAge of a backup file is a time.Duration (rather than an integer number of days).

The new API will use a constructor function, so we can validate the configuration and the filename when you create it, rather than waiting for the first Write to find out if you typoed something.

New Features

I do plan to implement most of the feature requests that people have put forth that I previously shot down for being too prone to user error. Far be it from me to stop you from shooting yourself in the foot ;) ... but seriously, I can't always anticipate all the uses someone may have for the library, so I leave it up to you all to keep your feet out of the firing line.

I don't think there's any features that need removing from what exists in v2, so this list of TODOs should be considered purely additive to what's in v2:

• Self-defined date format in the backup filenames
• No date format in the backup filenames
  • using logrotate's schema of _1, _2, etc, and using last modified date to determine backup file age
• Callback after rotate
• Periodic rotation (daily, etc)
• Max total log size
  • this important if you gzip backup files... it's hard to know how much they'll compress, and thus it's hard to know how many backups is a "safe" number.
• Backups stored in a separate directory

Other Requests?

If you have any other requests, please feel free to respond here, rather than in issues, so we can all discuss them in one place. Thanks!

[francislavoie]

Hey! We're using lumberjack.v2 in Caddy, and we've had a user request support for "Self-defined date format in the backup filenames". https://caddy.community/t/log-rotation-template-change/17099

It looks like the v3 branch hasn't seen too much activity since you wrote this post. Just wondering if you have a general idea when you plan to continue work on it? We'd love to upgrade Caddy to v3 once it's a bit more stable 😃

[hvisage]

using the logrotate _1/_2 scheme (either incremental numbers or TZ timestamps appended) would be solving my problem of tracking only the *.log files,

[mholt]

This is exciting! We've been using lumberjack for years in Caddy.

I see you're planning "periodic rotation (daily, etc.)" which sounds great. We got a lot of requests to rotate logs periodically by time instead of size, e.g.: caddyserver/caddy#1096 -- I seem to remember that feature has some technical difficulties though. Do you have a plan to overcome them?

Another issue I recall is having multiple lumberjack.Loggers pointing at the same log file, but from within the same process: caddyserver/caddy#1363 -- we worked around this by pooling them, but I wonder if that's something that could be done under the hood? (I realize it's kind of a niche request, but just thought I'd mention it.)

Also, we'd possibly be interested in multiple compression formats other than gzip. For example, zstd.

[mholt]

Oh, one other thing that's really important to us is the ability to stop a logger or clean up. When Caddy config is reloaded, the old logger should be stopped and the new logger will be started. This happens often during the lifetime of the process, as the logger config may change.

For example, a Stop() or Close() method could be good, or even a context we can pass in that we would cancel should be sufficient!

[natefinch]

I didn't even realize caddy was using lumberjack, that's great :)

Would the Caddy folks be willing to take over maintenance of Lumberjack? One of the reasons I haven't done a lot of work on it is that I'm not really using it anymore. I'd be happy to pass responsibility for it on to your org, if you're up for it.

[mholt]

Ah, thanks for the offer -- although we barely have the bandwidth to keep our own projects going 😅 I can ask around and see if anyone in the community would like to take it up with us, but I personally don't have the time to commit to it right now.

[joydeep15]

Hey @natefinch
Do you need devs for this? I am interested in the MaxFileSize bytes feature and can contribute if you need help