Cricket should support testing on tox

[hawkowl]

tox is heavily used in many major projects for the ease of testing multiple Python configurations, versions of packages, and the like. Cricket should support using tox, because then it's usefulness is amplified in projects that use tox, as it can put a nice face on running lots of different tests on different tox envs from one spot.

Things that need to be taken into consideration:

• Test discovery will have to be per-environment. Test suites do not stay the same on 2.6, 2.7, 3.3, etc, especially in the case of Twisted. We have entire test files that are not on 3.3, some test classes are deleted because they can't be run, and therefore won't show up on test discovery for all platforms. We also generate some test classes depending on what reactors are available, and not all Pythons have the same reactors available. Therefore, during startup, a "discovery" run will have to happen, possibly in a env called something like "py27-tests-discovery" for the "py27-tests" env, which just executes the test runner in a "dry run" configuration (-n for Twisted Trial).
• Cricket can't just access the raw unittest results -- it'll have to use something like subunit, which is a machine-readable streaming output format. Trial has built-in support for subunit, and there is a py.test plugin available for it. So, cricket will have to understand this subunit output, but that shouldn't be too difficult.
• Cricket will have to execute tox itself, and have some way of knowing when the tests start and what's tox's output. Tox has --result-json but I have no idea if it is streaming (unlikely). Running tests in parallel environments may be desirable, but it may also not be.

[freakboy3742]

Integrating with Tox definitely seems like a good idea to me. It will probably require some UI changes, but they shouldn't be too dramatic.

More generally, I wasn't aware of subunit and it's protocol - from what you describe, that sounds like a much better option than my current bespoke protocol.

[Christena421]

I will try this , then i can think cricket support texting on tox.