syntaxgym is a Python package for easy targeted syntactic evaluation of
language models. It replicates the core behavior of the SyntaxGym website,
providing standardized and reproducible processes for targeted syntactic
evaluation.
syntaxgym requires a working Docker installation.
Next, install and update the package using pip:
$ pip install -U syntaxgym
- Package documentation
- SyntaxGym, the centralized repository of targeted syntactic evaluations