Skip to content

Commit

Permalink
Add CodeQL advanced security and mypy (#72)
Browse files Browse the repository at this point in the history
### What kind of change does this PR introduce?

* Adds a YAML workflow for CodeQL security checks
* Adds a hook for `vulture` to find dead code
* Adds a hook for `mypy` for dynamic type-checking with some small
adjustments needed for type narrowing

### Does this PR introduce a breaking change?

No.

### Other information:

This PR adds `mypy`, which is a significant undertaking for most
projects. I'd rather have this added by default and later disabled or
progressively adopted, rather than never used. There are many good
reasons to be using `mypy`, and security (by way of type checking and
enhanced maintainability) is one reason.
  • Loading branch information
Zeitsperre authored Jan 14, 2025
2 parents 63f44fc + 7ffdfaf commit 36ea293
Show file tree
Hide file tree
Showing 8 changed files with 157 additions and 27 deletions.
1 change: 1 addition & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ jobs:
github.com:443
index.crates.io:443
objects.githubusercontent.com:443
proxy.golang.org:443
pypi.org:443
static.crates.io:443
- name: Checkout Repository
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ on:
types:
- closed

permissions: # added using https://github.com/step-security/secure-repo
permissions:
contents: read

jobs:
Expand All @@ -16,7 +16,7 @@ jobs:
actions: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@0080882f6c36860b6ba35c610c98ce87d4e2f26f # v2.10.2
uses: step-security/harden-runner@0080882f6c36860b6ba35c610c98ce87d4e2f26f # v2.10.2
with:
disable-sudo: true
egress-policy: block
Expand Down
101 changes: 101 additions & 0 deletions {{cookiecutter.project_slug}}/.github/workflows/codeql.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# For most projects, this workflow file will not need changing; you simply need to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
name: "CodeQL Advanced"

on:
push:
branches:
- main
pull_request:
branches:
- main
schedule:
- cron: '36 9 * * 1'

permissions:
contents: read

{% raw -%}
jobs:
analyze:
name: Analyze (${{ matrix.language }})
# Runner size impacts CodeQL analysis time. To learn more, please see:
# - https://gh.io/recommended-hardware-resources-for-running-codeql
# - https://gh.io/supported-runners-and-hardware-resources
# - https://gh.io/using-larger-runners (GitHub.com only)
# Consider using larger runners or machines with greater resources for possible analysis time improvements.
runs-on: ${{ (matrix.language == 'swift' && 'macos-latest') || 'ubuntu-latest' }}
permissions:
# required for all workflows
security-events: write

# required to fetch internal or private CodeQL packs
packages: read

# only required for workflows in private repositories
actions: read
contents: read

strategy:
fail-fast: false
matrix:
include:
- language: python
build-mode: none
# CodeQL supports the following values keywords for 'language': 'c-cpp', 'csharp', 'go', 'java-kotlin', 'javascript-typescript', 'python', 'ruby', 'swift'
# Use `c-cpp` to analyze code written in C, C++ or both
# Use 'java-kotlin' to analyze code written in Java, Kotlin or both
# Use 'javascript-typescript' to analyze code written in JavaScript, TypeScript or both
# To learn more about changing the languages that are analyzed or customizing the build mode for your analysis,
# see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning.
# If you are analyzing a compiled language, you can modify the 'build-mode' for that language to customize how
# your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages
steps:
- name: Harden Runner
uses: step-security/harden-runner@0080882f6c36860b6ba35c610c98ce87d4e2f26f # v2.10.2
with:
disable-sudo: true
egress-policy: audit

- name: Checkout Repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@48ab28a6f5dbc2a99bf1e0131198dd8f1df78169 # v3.28.0
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.

# For more details on CodeQL's query packs, refer to:
# https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
# queries: security-extended,security-and-quality

# If the analyze step fails for one of the languages you are analyzing with
# "We were unable to automatically build your code", modify the matrix above
# to set the build mode to "manual" for that language. Then modify this step
# to build your code.
# ℹ️ Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
- if: matrix.build-mode == 'manual'
shell: bash
run: |
echo 'If you are using a "manual" build mode for one or more of the' \
'languages you are analyzing, replace this with the commands to build' \
'your code, for example:'
echo ' make bootstrap'
echo ' make release'
exit 1
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@48ab28a6f5dbc2a99bf1e0131198dd8f1df78169 # v3.28.0
with:
category: "/language:${{matrix.language}}"
{%- endraw %}
22 changes: 19 additions & 3 deletions {{cookiecutter.project_slug}}/.pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ default_language_version:

repos:
- repo: https://github.com/asottile/pyupgrade
rev: v3.17.0
rev: v3.19.1
hooks:
- id: pyupgrade
args: [ '--py39-plus' ]
Expand Down Expand Up @@ -38,12 +38,16 @@ repos:
rev: 24.10.0
hooks:
- id: black
{% if cookiecutter.make_docs %}exclude: ^docs/{% endif %}
{% if cookiecutter.make_docs -%}
exclude: ^docs/
{%- endif %}
- repo: https://github.com/PyCQA/isort
rev: 5.13.2
hooks:
- id: isort
{% if cookiecutter.make_docs %}exclude: ^docs/{% endif %}
{% if cookiecutter.make_docs -%}
exclude: ^docs/
{%- endif %}
{%- endif %}
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.8.2
Expand All @@ -57,6 +61,14 @@ repos:
- id: flake8
additional_dependencies: [ 'flake8-rst-docstrings' ]
args: [ '--config=.flake8' ]
- repo: https://github.com/jendrikseipp/vulture
rev: v2.13
hooks:
- id: vulture
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.14.1
hooks:
- id: mypy
{%- if cookiecutter.use_black == 'y' %}
- repo: https://github.com/keewis/blackdoc
rev: v0.3.9
Expand Down Expand Up @@ -85,6 +97,10 @@ repos:
hooks:
- id: zizmor
args: [ '--config=.zizmor.yml' ]
- repo: https://github.com/gitleaks/gitleaks
rev: v8.21.2
hooks:
- id: gitleaks
- repo: meta
hooks:
- id: check-hooks-apply
Expand Down
3 changes: 2 additions & 1 deletion {{cookiecutter.project_slug}}/docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
#
import os
import sys
from typing import Any

sys.path.insert(0, os.path.abspath('..'))

Expand Down Expand Up @@ -138,7 +139,7 @@

# -- Options for LaTeX output ------------------------------------------

latex_elements = {
latex_elements: dict[str, Any] = {
# The paper size ('letterpaper' or 'a4paper').
#
# 'papersize': 'letterpaper',
Expand Down
8 changes: 8 additions & 0 deletions {{cookiecutter.project_slug}}/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -386,3 +386,11 @@ max-doc-length = 180

[tool.ruff.lint.pydocstyle]
convention = "numpy"

[tool.vulture]
exclude = []
ignore_decorators = ["@pytest.fixture"]
ignore_names = []
min_confidence = 90
paths = ["src/{{cookiecutter.project_slug }}", "tests"]
sort_by_size = true
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ def main():
parser.add_argument('_', nargs='*')
args = parser.parse_args()

print("Arguments: " + str(args._))
print(f"Arguments: {args._}")
print("Replace this message by putting your code into {{cookiecutter.project_slug}}.cli.main")
return 0

Expand All @@ -25,7 +25,7 @@ def main():


@click.command()
def main(args=None):
def main(args=None) -> int:
"""Console script for {{cookiecutter.project_slug}}."""
click.echo(
"Replace this message by putting your code into {{cookiecutter.project_slug}}.cli.main",
Expand All @@ -43,8 +43,8 @@ def main(args=None):
console = Console()


@app.command()
def main():
@app.command() # type: ignore[misc]
def main() -> None:
"""Console script for {{cookiecutter.project_slug}}."""
console.print(
"Replace this message by putting your code into {{cookiecutter.project_slug}}.cli.main",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,31 +5,30 @@
from importlib.util import find_spec
{% if cookiecutter.use_pytest == 'n' -%}
import unittest
{% else %}
import pytest
{%- endif %}
{%- if cookiecutter.command_line_interface|lower == 'click' %}
from click.testing import CliRunner
{%- endif %}
{% if cookiecutter.command_line_interface|lower == 'click' %}import {{ cookiecutter.project_slug }}.cli as cli{%- endif %}
{%- if cookiecutter.command_line_interface|lower == 'click' %}import {{ cookiecutter.project_slug }}.cli as cli{%- endif %}
from {{ cookiecutter.project_slug }} import {{ cookiecutter.project_slug }} # noqa: F401
{%- if cookiecutter.use_pytest == 'y' %}

# import pytest

@pytest.fixture
def response():
"""Sample pytest fixture.
# @pytest.fixture
# def response():
# """Sample pytest fixture.
#
# See more at: https://doc.pytest.org/en/latest/explanation/fixtures.html
# """
# # import requests
# # return requests.get('https://github.com/audreyr/cookiecutter-pypackage')

See more at: https://doc.pytest.org/en/latest/explanation/fixtures.html
"""
# import requests
# return requests.get('https://github.com/audreyr/cookiecutter-pypackage')


def test_content(response):
"""Sample pytest test function with the pytest fixture as an argument."""
# from bs4 import BeautifulSoup
# assert 'GitHub' in BeautifulSoup(response.content).title.string
# def test_content(response):
# """Sample pytest test function with the pytest fixture as an argument."""
# # from bs4 import BeautifulSoup
# # assert 'GitHub' in BeautifulSoup(response.content).title.string
{%- if cookiecutter.command_line_interface|lower == 'click' %}


Expand Down Expand Up @@ -74,9 +73,13 @@ def test_command_line_interface(self):

def test_package_metadata():
"""Test the package metadata."""
project = find_spec("{{ cookiecutter.project_slug }}").submodule_search_locations[0]
project = find_spec("{{ cookiecutter.project_slug }}")

assert project is not None
assert project.submodule_search_locations is not None
location = project.submodule_search_locations[0]

metadata = pathlib.Path(project).resolve().joinpath("__init__.py")
metadata = pathlib.Path(location).resolve().joinpath("__init__.py")

with metadata.open() as f:
contents = f.read()
Expand Down

0 comments on commit 36ea293

Please sign in to comment.