Skip to content

Commit

Permalink
updating
Browse files Browse the repository at this point in the history
  • Loading branch information
projx committed Aug 8, 2022
1 parent 91463c7 commit 39482da
Show file tree
Hide file tree
Showing 7 changed files with 686 additions and 25 deletions.
6 changes: 4 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# App and ide specific
# prjx common ...
/etc/*.yml
/*.obj
/downloads/*
Expand All @@ -11,6 +11,8 @@
.idea/
/.idea/
.idea/
data/
/stubs/

# Byte-compiled / optimized / DLL files
__pycache__/
Expand Down Expand Up @@ -112,4 +114,4 @@ ENV/
# mypy
.mypy_cache/

/stubs/

12 changes: 6 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
## Build: docker build --force-rm --no-cache -t "qtool" .
## Run Interactive: docker run -it qtool
## Run Interactive: docker run -it qtool /bin/bash
## Run Daemon: docker run -d qtool
## Build: docker build --force-rm --no-cache -t "qasa" .
## Run Interactive: docker run -it qasa
## Run Interactive: docker run -it qasa /bin/bash
## Run Daemon: docker run -d qasa
## apt-get install --reinstall ca-certificates

## Debian-base build...
FROM python:3.10.6-slim-buster
ADD . /app
RUN pip install -r /app/requirements.txt
CMD ["python3", "/app/qtool.py", "exec", "scheduler"]
CMD ["python3", "/app/qasa.py", "exec", "scheduler"]

##
### Alpine Linux build...
# FROM python:3.10.6-alpine3.16
# RUN set -ex && apk add --no-cache gcc libc-dev
# ADD . /app
# RUN pip install -r /app/requirements.txt
# CMD ["python3", "/app/qtool.py", "exec", "scheduler"]
# CMD ["python3", "/app/qasa.py", "exec", "scheduler"]


661 changes: 661 additions & 0 deletions LICENSE.md

Large diffs are not rendered by default.

18 changes: 8 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[![Publish Docker image](https://github.com/projx/qtool/actions/workflows/build-and-push.yml/badge.svg)](https://github.com/projx/qtool/actions/workflows/build-and-push.yml)
[![Publish Docker image](https://github.com/projx/qasa/actions/workflows/build-and-push.yml/badge.svg)](https://github.com/projx/qasa/actions/workflows/build-and-push.yml)

# QtooL (Query Too Log)
# QASA (Query Anything, Send Anywhere)

A poller for pulling useful data (SNMP, HTTP etc), and sending the results onwards... perhaps to a Log analytics system or syslog server... It really doesn't care!

Expand All @@ -14,7 +14,7 @@ Personally, I wanted to be able to visualise my homelab metrics, such as:
- Internet connection speed-tests
- Query certain Web APIs

Understandably, sane people use Grafana for this, it does it out-of-the-box... but I wanted the freedom of using something else, being able to easily point my data elsewhere, such as Splunk, ELK or even a Cloud service. So I wrote QtooL...
Understandably, sane people use Grafana for this, it does it out-of-the-box... but I wanted the freedom of using something else, being able to easily point my data elsewhere, such as Splunk, ELK or even a Cloud service. So I wrote QASA...

## What is it?

Expand All @@ -34,7 +34,7 @@ Some key requirements were

### There are 3 main components:

## ![connectors.png](https://github.com/projx/qtool/blob/main/docs/connectors.png?raw=true)
## ![connectors.png](https://github.com/projx/qasa/blob/main/docs/connectors.png?raw=true)

The Scheduler creates a thread for each Poller, which runs the Poller > Formatter > Sender, then sleeps for a defined interval.

Expand Down Expand Up @@ -84,13 +84,13 @@ Please see the [README.md](docs/index.md) for installation instructions.
## Getting Started

There is a basic by working configuration (see etc-sample/) which polls yahoo.com and then writes the data to the log/ directory, also there are several examples for other pollers, such polling a Switch and WAP via SNMP.
There is a sample configuration (see etc-sample/), it includes

Please see the [README.md](docs/index.md) for details on the configuration, and a breakdown of all the different options.
Please see the [index.md](docs/index.md) for details on the configuration, and a breakdown of all the different options.

### **Possible Enhancements:**

1. Add "Content-Type" return to each Formatter class, i.e. json or text, for use in the senders
1. Refactor Pollers, to make them easier for others to create new pollers or extend existing ones.
2. Add "Content-Type" return to each Formatter class, i.e. json or text, for use in the senders
2. Update FileSender to do file rotation
3. Consider changing "time" field to "@timestamp"
4. Pollers
Expand Down Expand Up @@ -121,6 +121,4 @@ Please see the [README.md](docs/index.md) for details on the configuration, and

---

I originally started writing this in summer of 2020, I've been using it in my homelab since beginning of 2021, and I intended to release to make it available last summer, but unfortunately life got in the way.

Now I've had an idea where I'd like to build a dashboard, something complimentary to Uptime Kuma, but monitoring SNMP etc, which will make use of QTool for polling, hence I've decided get it to a point I'm happy to make available for others to see.
I started writing this in summer of 2020, I've personally been using it since then and making incremental changes. I never intended to open source it, but some users in r/homelab showed an interest, so I thought why not. You many find references to "QTool" in the code, this was its original name, but I was preparing for a trip to our casa in Spain at the time, and QASA seemed to have a better ring to it.
6 changes: 3 additions & 3 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
version: "3"
services:
qtool:
image: ghcr.io/projx/qtool:v0.1.5
container_name: qtool
qasa:
image: ghcr.io/projx/qasa:v0.1.5
container_name: qasa
restart: unless-stopped
volumes:
- ./etc/:/app/etc
Expand Down
6 changes: 3 additions & 3 deletions etc-sample/senders.yml
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
senders:

## This will output to app/log/qtool_example.log
## This will output to app/log/qasa_example.log
LOCALFILE:
connector_class: lib.senders.syslog.QFileSender
alias: LOCALFILE
directory: logs
file: 'qtool_example.log'
file: 'qasa_example.log'

## You can make multiple senders of the same type, but they must have unique aliases.
LOCALFILE_ALTERNATIVE:
connector_class: lib.senders.syslog.QFileSender
alias: LOCALFILE_ALTERNATIVE
directory: logs
file: 'qtool_example.log'
file: 'qasa_example.log'

# SYSLOG_TCP:
# alias: SPLUNK_SYSLOG_UDP
Expand Down
2 changes: 1 addition & 1 deletion qasa.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ def global_setup(show_output = True):
@click.group()
@click.option('-s', '--suppress', is_flag=True, default=False, help="Suppress most output (Note actual output level is defined in settings.yml")
def main(suppress):
"""QtooL - Is a scheduler for Polling, Formatting and Sending """
"""QASA - Is a scheduler for Polling, Formatting and Sending """

# Handle flags
show_output = False if suppress else True
Expand Down

0 comments on commit 39482da

Please sign in to comment.