HowToTest

« back to DeveloperGuide

Every piece of new code, or significant change to existing code, should be accompanied by a corresponding new test or change in a test. We have two kinds of tests: unit tests, which quickly exercise the code that has minimal dependencies, and system tests, which attempt to exercise every user-facing page and feature.

Our tests are run with pytest. We generally prefer to use plain assert because it's much easier to read. For example, please write assert x == 3 instead of self.assertEquals(x, 3). If the assertion fails, pytest will produce a nice report explaining what the value of x was.

tools/all_tests will run both the unit and server tests.

Unit tests

Each Python module in the app/ directory should have a corresponding test module in the tests/ directory, with a filename starting with test_. For example, app/indexing.py contains the indexing and ranking functions, and there are unit tests for those functions in tests/test_indexing.py.

You can run the unit tests with the script tools/unit_tests. If you want to run the tests in just one module, specify the module name, e.g. tools/unit_tests test_indexing.

System tests

The system tests bring up a running application server with a file-based stub for the datastore, put test data in the datastore, and issue HTTP requests to the server to simulate a user actually loading pages and clicking on buttons.

All of the system tests currently reside in one file, tests/server_test_cases.py. For efficiency, the tests are grouped into classes based on what data they modify. Your tests can create and modify data by manipulating datastore entities just like regular application code. For example, you can create a Person entity by instantiating the Person class and calling put() on it.

You can run all the server tests with the script tools/server_tests. If you want to run the tests in just one class or method, specify the -k option with any substring of the class or method name, e.g. tools/server_tests -k test_subscribe.

To simulate the actions of a real user, your tests can use the scrape module to make HTTP requests, parse HTML, find links and buttons on the page, and submit forms. See http://zesty.ca/scrape for details on how to use the scrape tool.

« back to DeveloperGuide