Developing language parser plugins

Make sure you've read through the Development page too.

Note: these docs may need some refining. They were written as part of developing the first plugin, the lute-mandarin. As always, working code is the best guideline, so see the lute-mandarin parser files for a working example!

Adding a new parser

When developing a new parser parser, you must create a new folder under plugins, following the naming convention lute-{language}.

The template folder plugins/_template_ has some stub files and things you'll likely need. You can copy the folder plugins/_template_ to a new folder named plugins/lute-{language} to help you get started. See the plugins/_template_/README.md for notes on what to do with the files in your copied folder.

The pyproject.toml

In your folder, create a pyproject.toml file to define the project name and description.

The project name should be lute3-{lang}, to make a clear association with the core Lute package lute3. (At some point, I hope to get a community project in PyPi, and all packages will be published there.)

The pyproject.toml file must define the entry-points as follows, to allow lute to bind it at runtime:

[project.entry-points."lute.plugin.parse"]
lute{lang} = "lute{lang}_parser.parser:{lang}Parser" 

The pyproject.toml must also add Lute itself as a dependency (>=3.4.2, when the plugin capability was added), to have the parser use some of Lute's existing classes.

The parser class

The new parser must be a subclass of lute.parser.base.AbstractParser, and implement, at a minimum, the get_parser_tokens and name methods. Please also include a language definition file in the plugin directory. Each parser also needs unit tests to ensure it actually works.

Developing your parser

When developing your parser, you should install Lute, and it, into your virtual environment.

pip install .
pushd plugins/lute-mandarin
pip install . 
popd

When you start the development server with inv start, you will see that your parser gets loaded, e.g.:

(.venv) $ inv start

Initializing app.
Initializing parsers from plugins ...
Enabled parsers:
  * ... [snip] ...
  * Lute Mandarin Chinese

Things to check during dev:

• you can load the preconfigured definition and story for languages using your parser
• you can create a new language using your new parser

Parser CI

GitHub CI runs your parser tests. It also does a code styling and linting check (using black), so if you open a PR and get failures, please investigate and fix them!

Parser releases

Parsers are released separately from Lute into their own independent packages, so that users can pick and choose what extra parser plugins they want to install.