Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to Python 3.12, closes #156 #157

Merged
merged 4 commits into from
Dec 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# https://editorconfig.org

root = true

[*]
indent_style = space
indent_size = 2
trim_trailing_whitespace = true
insert_final_newline = true
charset = utf-8
end_of_line = lf

[*.{py,md}]
indent_size = 4

[Makefile]
indent_size = 4
indent_style = tab
5 changes: 1 addition & 4 deletions .env
Original file line number Diff line number Diff line change
@@ -1,5 +1,2 @@
HOST=currency-exchange.p.rapidapi.com
HOST=https://currency-exchange.p.rapidapi.com
API_TOKEN=d26533cbe8msh020600a07386d87p18a610jsn31d39f87f802

# Locust
LOCUST_WEB_AUTH=ubuntu:debian
13 changes: 5 additions & 8 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ on:
- master

schedule:
# Every Sunday at 0:37 UTC.
- cron: "37 0 * * 0"
# Run every 15 days at 12:37 AM UTC.
- cron: "37 0 */15 * *"

concurrency:
group: ${{ github.head_ref || github.run_id }}
cancel-in-progress: true
group: ${{ github.head_ref || github.run_id }}
cancel-in-progress: true

jobs:
run-tests:
Expand All @@ -26,7 +26,7 @@ jobs:
# Use matrix strategy to run the tests on multiple Py versions on multiple OSs.
matrix:
os: [ubuntu-latest, macos-latest]
python-version: ["3.8", "3.9", "3.10", "3.11"]
python-version: ["3.9", "3.10", "3.11", "3.12"]

steps:
- uses: actions/checkout@v4
Expand All @@ -50,6 +50,3 @@ jobs:
- name: Run the tests & Generate coverage report
run: |
make test
locust -f src/locustfile.py \
--host https://currency-exchange.p.rapidapi.com/ \
--headless -u 1 -r 5 --run-time=5 --exit-code-on-error 1
24 changes: 24 additions & 0 deletions .prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"proseWrap": "preserve",
"semi": false,
"singleQuote": false,
"useTabs": false,
"tabWidth": 2,
"trailingComma": "all",
"printWidth": 92,
"overrides": [
{
"files": "**/*.md",
"options": {
"tabWidth": 4,
"useTabs": false,
"singleQuote": false,
"trailingComma": "all",
"arrowParens": "avoid",
"printWidth": 92,
"proseWrap": "always",
"embeddedLanguageFormatting": "off"
}
}
]
}
136 changes: 63 additions & 73 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,20 @@

</div>


## Description

[Locust][0] is a distributed and scalable open-source library that
helps you write effective load tests in pure Python. This repository demonstrates a
modular architecture to establish a template for quickly building a scalable stress
testing pipeline with Locust.
[Locust][0] is a distributed and scalable open-source library that helps you write effective
load tests in pure Python. This repository demonstrates a modular architecture that can work
as a template for quickly building a scalable stress testing pipeline with Locust.

## Locust terminology

If you're unfamiliar with the terminologies and the generic workflow of writing
stress-tests with Locust, then you should go through the official [documentation][1]
first. With that out of the way, let's go through a few terminologies that comes up in
this context quite often:
If you're unfamiliar with the terminologies and the generic workflow of writing stress-tests
with Locust, then I recommend going through the official [documentation][1] first. With that
out of the way, let's rehash a few terminologies that comes up in this context quite often:

**Task:** In Locust, a [Task][2] is the smallest unit of a test suite. Usually, it
means, any function or method that is decorated with the `@task` decorator.
**Task:** In Locust, a [Task][2] is the smallest unit of a test suite. Usually, it means,
any function or method that is decorated with the `@task` decorator.

**TaskSet:** A [TaskSet][3] is a class that establishes a contextual boundary between
different groups of Tasks. You can essentially group multiple similar Tasks inside a
Expand All @@ -29,19 +26,19 @@ TaskSet. Then you use the TaskSets from your User class.
**User:** In Locust, a [User][4] is a class that executes the tests either by directly
calling the Task methods or via using TaskSets.

***In more complex cases, the tests can further be organized by arranging them in
multiple test modules. This template groups the Tasks using TaskSets and places multiple
TaskSets in separate test modules to ensure modularity and better scalability.***
**_In more complex cases, the tests can further be organized by arranging them in multiple
test modules. This template groups the Tasks using TaskSets and places multiple TaskSets in
separate test modules to achieve better modularity._**

## Target API

This template uses [Rapid API's][5] currency-exchange [API][6] for showcasing the load
testing procedure. The API converts one currency to another using the current exchange
rate.
Here, we're using [Rapid API's][5] currency-exchange [API][6] to showcase the load testing
procedure. The API converts one currency to another using the current exchange rate.

### API anatomy

It takes three parameters in its query string:

```
1. q : str - quantity
2. from : str - currency to convert from
Expand All @@ -50,13 +47,12 @@ It takes three parameters in its query string:

And returns the converted value.


### Access the API

Sign up for a Rapid API [account][7] and get your token. You can access the API via
cURL like (You need to provide your API token in the header):
Sign up for a Rapid API [account][7] and get your token. You can access the API via cURL
like (You need to provide your API token in the header):

```bash
```sh
curl --request GET \
--url 'https://currency-exchange.p.rapidapi.com/exchange?q=1.0&from=USD&to=BDT' \
--header 'x-rapidapi-host: currency-exchange.p.rapidapi.com' \
Expand All @@ -65,13 +61,12 @@ curl --request GET \

The response will look like this:

```
```txt
84.91925⏎
```

Or, you might want to access it via Python. You can do so using the [HTTPx][8] library
like this:

Or, you might want to access it via Python. You can do so using the [HTTPx][8] library like
this:

```python
import httpx
Expand All @@ -93,12 +88,10 @@ print(response.text)

## Stress testing pipeline

Below, you can see the core architecture of the load testing pipeline. For brevity's
sake—files regarding containerization, deployment, and dependency management have been
omitted.

Below, you can see the core architecture of the load testing pipeline. For brevity's sake,
files regarding containerization, deployment, and dependency management have been omitted.

```
```txt
src/
├── locustfiles # Directir where the load test modules live
│ ├── __init__.py
Expand All @@ -111,86 +104,82 @@ src/
└── settings.py # Read the environment variables here
```

The test suite has three primary components
`setup.py`, `locustfiles`, and the `locust.conf` file.
The test suite has three primary components: `setup.py`, `locustfiles`, and the
`locust.conf` file.

### [setup.py](src/setup.py)

The common elements required for testing, like `auth`, `login`, and `logout` functions
reside in the `setup.py` file.

### [locustfiles](src/locustfiles/)
The load test modules reside in the **`locustfiles`** directory. Test modules import and
use the functions in the `setup.py` file before executing each test.

The load test modules reside in the **`locustfiles`** directory. Test modules import and use
the functions in the `setup.py` file before executing each test.

In the `locustfiles` directory, currently, there are only two load test modules—
`bdt_convert.py` and `rs_convert.py`. You can name your test modules whatever you want
and the load testing classes and functions should reside here.
`bdt_convert.py` and `rs_convert.py`. You can name your test modules whatever you want and
the load testing classes and functions should reside here.

* [bdt_convert.py](src/locustfiles/bdt_convert.py): This module houses a single TaskSet
named `BDTConvert` that has two Tasks—`usd_to_bdt` and `bdt_to_usd`. The first Task
tests the exchange API when the request query asks for USD to BDT currency conversion
and the second Task tests the API while requesting BDT to USD conversion.
- [bdt_convert.py](src/locustfiles/bdt_convert.py): This module houses a single TaskSet
named `BDTConvert` that has two Tasks—`usd_to_bdt` and `bdt_to_usd`. The first Task
tests the exchange API when the request query asks for USD to BDT currency conversion
and the second Task tests the API while requesting BDT to USD conversion.

* [rs_convert.py](src/locustfiles/rs_convert.py): The second test module does the same
things as the first one; only it tests the APIs while the request query asks for USD to
RS conversion and vice versa.
- [rs_convert.py](src/locustfiles/rs_convert.py): The second test module does the same
things as the first one; only it tests the APIs while the request query asks for USD to
RS conversion and vice versa.

The reason that there are two similar test modules is just to demonstrate how you
can organize your Tasks, TaskSets, and test modules.
The reason that there are two similar test modules is just to demonstrate how you
can organize your Tasks, TaskSets, and test modules.

* [locustfile.py](src/locustfile.py): This file works as the entrypoint of the workflow.
It imports the TaskSets from the `bdt_convert` and `usd_convert` modules and creates a
[HttpUser][9] that will execute the tasks.
- [locustfile.py](src/locustfile.py): This file works as the entrypoint of the workflow.
It imports the TaskSets from the `bdt_convert` and `usd_convert` modules and creates a
[HttpUser][9] that will execute the tasks.

### [locust.conf](src/locust.conf)

The **`locust.conf`** file defines the configurations like *hostname*, *number* of
*workers*, *number of simulated users*, *spawn rate*, etc.

The **`locust.conf`** file defines the configurations like _hostname_, _number_ of
_workers_, _number of simulated users_, _spawn rate_, etc.

## Run the stress tests locally

* Make sure you have the latest version of [docker][10] and
docker-compose installed on your machine.
- Make sure you have the latest version of [docker][10] and docker-compose installed on
your machine.

* Clone this repository and go to the root directory.
- Clone this repository and go to the root directory.

* Place your rapidAPI token in the `.env` file.
- Place your rapidAPI token in the `.env` file.

* Run:
- Run:

```bash
```sh
docker compose up -d
```

This will spin up a master container and a single worker container that will do the
testing. If you want to deploy more workers to do the load testing then run:

```bash
```sh
docker compose up -d --scale worker 2
```

* To access the Locust GUI, go to [http://localhost:8089/][11] on your browser. You'll
be prompted to provide a username and a password. Use `ubuntu` as the username and
`debian` as the password. You'll be greeted by a screen like below. You should see that
the fields of the form are already filled in since Locust pulls the values from the
`locust.conf` file:
- To access the Locust GUI, go to [http://localhost:8089/][11] on your browser. You'll be
greeted by a screen like below. You should see that the fields of the form are already
filled in since Locust pulls the values from the `locust.conf` file:

![locust signin][12]
![locust signin][12]

* Once you've pressed the *start swarming* button, you'll be taken to the following page:
- Once you've pressed the _start swarming_ button, you'll be taken to the following page:

![Screenshot from 2020-09-05 03-12-25][13]

* You can start, stop and control your tests from there.

- You can start, stop and control your tests from there.

## Disclaimer

This dockerized application is production-ready. However, you shouldn't expose your
environment file (.env) in production. Here, it was done only for demonstration
purposes.

environment file (.env) in production. Here, it was done only for demonstration purposes.

[0]: https://locust.io/
[1]: https://docs.locust.io/en/stable/
Expand All @@ -204,6 +193,7 @@ purposes.
[9]: https://docs.locust.io/en/stable/writing-a-locustfile.html#making-http-requests
[10]: https://www.docker.com/
[11]: http://localhost:8089/

[12]: https://user-images.githubusercontent.com/30027932/92285103-51988580-ef25-11ea-9155-c9d3f5dcaf42.png
[13]: https://user-images.githubusercontent.com/30027932/92285284-b94ed080-ef25-11ea-9f91-3f972fd844f1.png
[12]:
https://user-images.githubusercontent.com/30027932/92285103-51988580-ef25-11ea-9155-c9d3f5dcaf42.png
[13]:
https://user-images.githubusercontent.com/30027932/92285284-b94ed080-ef25-11ea-9f91-3f972fd844f1.png
4 changes: 2 additions & 2 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ services:
container_name: locust-main
build:
context: ./
dockerfile: ./dockerfiles/python311/Dockerfile
dockerfile: ./dockerfiles/python312/Dockerfile
ports:
- 8089:8089
volumes:
Expand All @@ -21,7 +21,7 @@ services:
- ./:/code/
build:
context: ./
dockerfile: ./dockerfiles/python311/Dockerfile
dockerfile: ./dockerfiles/python312/Dockerfile
command: |
bash -c 'locust -f /code/src/locustfile.py --worker \
--master-host main --config /code/src/locust.conf'
Expand Down
2 changes: 1 addition & 1 deletion dockerfiles/python310/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Use the latest locust image
FROM python:3.10-bullseye
FROM python:3.10-bookworm

# Set working directory
WORKDIR /code
Expand Down
2 changes: 1 addition & 1 deletion dockerfiles/python311/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Use the latest locust image
FROM python:3.11-bullseye
FROM python:3.11-bookworm

# Set working directory
WORKDIR /code
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Use the latest locust image
FROM python:3.8-bullseye
FROM python:3.12-bookworm

# Set working directory
WORKDIR /code
Expand Down
2 changes: 1 addition & 1 deletion dockerfiles/python39/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Use the latest locust image
FROM python:3.9-bullseye
FROM python:3.9-bookworm

# Set working directory
WORKDIR /code
Expand Down
Loading