statsdlog

Simple daemon that consumes a syslog udp stream and generates statsd events when certain log lines are encountered.

Configuration

statsdlog sample config:

[main]
#debug = false
# user to drop privs to
#user = nobody
#statsd_host = 127.0.0.1
#statsd_port = 8125
#statsd_sample_rate = 1.0
#listen_addr = 127.0.0.1
#listen_port = 8126
#buffer_size = 8192
#max_line_backlog = 512
#report internals stats about statsdlog to statsd
#report_internal_stats = false
# if enabled report internal stats every 5 seconds
#internal_stats_interval = 5
# file that contains your log search patterns and statsd event names
patterns_file = /etc/statsdlog/patterns.json
# json_pattern_file = true
#
# alternatively instead of using a json based patterns file you can use a
# a plain text format:
# patterns_file = /etc/statsdlog/patterns.conf
# json_pattern_file = false

• Copy etc/statsdlog/patterns.json to /etc/statsdlog/patterns.json
• Edit patterns.json to include the regex patterns for log lines you want to fire events for.
• Theres a few examples included in etc/statsdlog/ (everything from firing events when packages are installed to nginx errors)
• Copy etc/statsdlog/statsdlog.conf-sample to /etc/statsdlog/statsdlog.conf
• Edit the the conf file to point to your statsd host
• Point the relevent syslog udp stream to 127.0.0.1:8126
• sudo python bin/statsdlog-server --conf=/etc/statsdlog/statsdlog.conf start (or use the the included init script)
• Profit!

Its important to note that the first match NO LONGER wins. As of v0.1.1 an event will be fired for EACH line match. This means a single log line can generate multiple events.

The included patterns.json-openstackswift example includes a few patterns for errors commonly encountered when running swift. There are also sample patterns for things like Nginx errors, sudo/ssh failures, or package installs. statsdlog is handy for generating events for all the things you want to track that do not natively support statsd.

Requirements

• eventlet

Installing

via setup.py:

• git clone git://github.com/pandemicsyn/statsdlog.git
• cd statsdlog
• python setup.py install

etc/statsdlog/statsdlog.init is available as a simple init script. You may need to update it to point to /usr/local/bin/ if installing via pip or setup.py.

via pip:

• pip install statsdlog

Building packages

Clone the version you want and build the package with stdeb (sudo apt-get install python-stdeb):

git clone git@github.com:pandemicsyn/statsdlog.git statsdlog
cd statsdlog
python setup.py --command-packages=stdeb.command bdist_deb
dpkg -i deb_dist/python-statsdlog_0.0.6-1_all.deb