A prototype for the report a breach service.
The project requires Python 3.11. Backing services are provided by Docker whilst the web app itself is ran as a normal process with Pipenv.
We use Pipenv to manage our virtual environment, dependencies, and environment variables. You can install it with either of the following commands:
# without homebrew
pip install --user pipenv
# OR with homebrew
brew install pipenv
Once installed, we need to install the requirements for the project:
pipenv install --dev
If you're using homebrew, install libmagic:
brew install libmagic
Now we need to activate the virtual environment:
pipenv shell
Install the repos pre commit hooks:
pre-commit install
Set pre-commit to autoupdate:
pre-commit autoupdate
Copy and paste the .env.example file and rename it to .env:
cp .env.example .env
Ask your colleagues to provide the missing values in your .env file.
Configure the AWS CLI:
aws configure
You can use the following dummy credentials that are provided by AWS. Alternatively, you can use your own credentials if you have an AWS account, Localstack does not validate them anyway:
AWS Access Key ID : AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key : wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region : eu-west-2
Use docker-compose to run the backing services
docker-compose up -d
Collect the apps static files:
invoke collectstatic
Add the below to your /etc/hosts file:
127.0.0.1 report-a-suspected-breach
127.0.0.1 view-a-suspected-breach
The first time you clone the repo, you may need to initialise the submodule:
git submodule update --init
After following the setup, use the following to run the web app:
invoke runserver
Django will provide the local url which should be http://127.0.0.1:8000/. As we use the django sites framework, this will no longer resolve by default.
To view the report-a-suspected-breach app navigate to: http://report-a-suspected-breach:8000/
To view the view-a-suspected-breach app, navigate to: http://view-a-suspected-breach:8000/
Along with the above runserver command, while developing on the project,
the following will be handy when making changes to the db model:
invoke makemigrations
invoke migrate
To add a new dependency to the project, use the following command:
pipenv install <package-name>
We use Localstack to emulate AWS services locally, specifically S3. Two buckets are created on container startup:
temporary-document-bucketthat's used to temporarily store uploaded filespermanent-document-bucketthat's used to store uploaded files permanently
Localstack works similarly to the awscli. For example, to see objects inside the temporary-document-bucket bucket, run command:\
awslocal s3 ls temporary-document-bucket
We store the list of Sanction regimes in a private git submodule located in django_app/sanctions_regimes.
The first time you clone the repo, you may need to initialise the submodule:
git submodule update --init
If this list has changed, you can update it from the latest version of the submodule by running the following command:
git submodule update --remote --merge
We use playwright for end-to-end testing. To run the end-to-end tests for report a breach, start the django server using the test config settings:
pipenv run python manage.py runserver --settings=config.settings.test
To run end-to-end tests only:\
pipenv run pytest tests/test_frontend
A useful command for writing end-to-end tests is:
pipenv run playwright codegen http://report-a-suspected-breach:8000/
The app works out of the box with Mock SSO, which is part of the Docker Compose setup. If the OAuth 2.0 flow isn't working,
try setting the AUTHBROKER_URL to docker.for.mac.localhost:8080 or host.docker.internal:8080
(value varies across platforms). This is because the Mock SSO service (configured with the AUTHBROKER_URL)
must be accessible from outside of docker-compose for the authorization redirect, and also from within docker-compose
to make the access token POST request.
We use:
- black to ensure consistent formatting across the project.
- flake8 to lint the project and flag any common code smells.
- isort to ensure imports are ordered correctly.
- djhtml to format and indent our HTML, CSS, and JS files
We use pre-commit to run all of the above before every commit.
Typing is used throughout the app. We have followed the best practices outlined in Typing Best Practices
All branches should be created from the main branch and be named after the JIRA ticket they are related to. e.g. DST-1234
All commits should be made to a branch and not directly to main.
The commit message should contain a clear and concise description of the changes made.
All pull requests should be made to main and should be named after the JIRA ticket they are related to and a short
description of the functionality implemented. e.g. DST-1234 - Implementing S3 buckets
We use python-invoke to run various commands in regard to django and linting/formatting.
These are maintained in tasks.py
All methods in the file can be run by typing invoke {command}.
Note that any command with underscores needs to be called with a - instead.
For example, the following task in tasks.py;
@task
def frontend_tests(context: Any) -> None:
context.run("pipenv run pytest tests/test_frontend")
Will be invoked by calling invoke frontend-tests
MyPy is used to sanity check typing but is not currently enforced.
To run mypy against the django_app, call invoke mypy