4. Architecture
BuffaLogs is developed using different technologies:
- Django - Python framework
- Elasticsearch - Analytics engine
- Kibana - GUI for Elasticsearch data
- PostgreSQL - Object-relational database
- Nginx - Web server
- RabbitMQ - Message Broker
- Celery - Task queue
- Celery Beat - Periodic task scheduler
The general architecture is explained by the diagram below:
Django is a high-level Python application development framework.
Elasticsearch allows you to store, search, and analyze huge volumes of data quickly and in near real-time and give back answers in milliseconds.
Kibana is a visual interface tool mainly used to analyze large volumes of log data massed in Elasticsearch.
Nginx is an open-source web server, also used as a reverse proxy, HTTP cache, and load balancer.
PostgreSQL is an open-source relational database. It supports both SQL (relational) and JSON (non-relational) querying.
RabbitMQ is a message-queueing software to which applications connect in order to transfer messages. A message could have information about a process or task that should start on another application. The queue-manager software stores the messages until a receiving application connects and takes a message off the queue, in order to process it. Message queueing allows web servers to respond to requests quickly instead of being forced to perform resource-heavy procedures on the spot that may delay response time.
Celery is the consumer in a scenario of producer and consumer. It handles any task that is waiting in the message broker (RabbitMQ).
Celery Beat is the scheduler, so it keeps track of when tasks should be executed by Celery. Celery would consume the tasks instantly, instead with Celery Beat the consumption of tasks is periodic.