abnf-to-regexp

The program abnf-to-regexp converts augmented Backus-Naur form (ABNF) to a regular expression.

Motivation

For a lot of string matching problems, it is easier to maintain an ABNF grammar instead of a regular expression. However, many programming languages do not provide parsing and matching of ABNFs in their standard libraries. This tool allows you to write your grammars in ABNF and convert it to a regular expression which you then include in your source code.

It is based on abnf Python module, which is used to parse the ABNFs.

After the parsing, we apply a series of optimizations to make the regular expression a bit more readable. For example, the alternations of character classes are taken together to form a single character class.

--help

usage: abnf-to-regexp [-h] -i INPUT [-o OUTPUT]
                      [--format {single-regexp,python-nested}]

Convert ABNF grammars to Python regular expressions.

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
                        path to the ABNF file
  -o OUTPUT, --output OUTPUT
                        path to the file where regular expression is stored;
                        if not specified, writes to STDOUT
  --format {single-regexp,python-nested}
                        Output format; for example a single regular expression
                        or a code snippet

Example Conversion

Please see test_data/nested-python/rfc3987/grammar.abnf for an example grammar.

The corresponding generated code, e.g., in Python, is stored at test_data/nested-python/rfc3987/expected.py.

Installation

You can install the tool with pip in your virtual environment:

pip3 install abnf-to-regexp

Development

• Check out the repository.
• In the repository root, create the virtual environment:

python3 -m venv venv3

• Activate the virtual environment (in this case, on Linux):

source venv3/bin/activate

• Install the development dependencies:

pip3 install -e .[dev]

• Run the pre-commit checks:

python precommit.py

Versioning

We follow Semantic Versioning. The version X.Y.Z indicates:

• X is the major version (backward-incompatible),
• Y is the minor version (backward-compatible), and
• Z is the patch version (backward-compatible bug fix).