Python Legistar Scraper

Scrapes municipal data from Legistar sites.

To install

pip install scraper-legistar

Contributing

Local installation

Install this package and its dependencies. (We recommend working in a virtual environment, e.g., venv or virtualenvwrapper).

pip install -e .

Install the development requirements.

pip install -r requirements.txt

Testing

To run the tests:

pytest -sv

A note on test fixtures

The tests rely on HTML from assorted Legistar properties. As it would be undesirable to hit the live server each time the tests are run, particularly during local development, fixtures are provided in the tests/fixtures/ directory.

A convenience script is provided for refreshing the fixtures, i.e., re-scraping Legistar and storing the resultant HTML.

The Legistar site is fairly stable, however if your patch addresses a change in page structure, update the fixtures as follows:

python tests/refresh_fixtures.py

Be sure to commit the refreshed fixture and submit it with your patch.