-
Notifications
You must be signed in to change notification settings - Fork 63
/
index.html
153 lines (118 loc) · 6.62 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
---
layout: naked
title: PIT Mutation Testing
---
<div class="jumbotron masthead">
<div class="container">
<img src="/images/pit-black-150x152.png" />
<br /><br />
<h1>Real world mutation testing</h1>
<p class="description">PIT is a state of the art <strong>mutation testing</strong> system, providing <strong>gold
standard test coverage</strong> for Java and the jvm. It's fast, scalable and integrates with modern test and
build tooling.</p>
<p>
<a href="/quickstart" class="btn btn-primary btn-large">Get Started</a>
</p>
<ul class="masthead-links">
<li>
<a href="https://www.arcmutate.com">Pro Version</a>
</li>
<li>
<a href="https://github.com/hcoles/pitest/issues">Issues</a>
</li>
<li>
<a href="https://github.com/hcoles/pitest">Source</a>
</li>
<li>
<a href="http://mvnrepository.com/artifact/org.pitest/pitest">Maven Central</a>
</li>
</ul>
</div>
</div>
<div class="container">
<div class="marketing">
<h1>What is mutation testing?</h1>
<p class="marketing-byline">How it works in 51 words</p>
<div class="row-fluid">
<div class="span12">
<p class="marketing-info">
Mutation testing is conceptually quite simple.
<p>
<p class="marketing-info"> Faults (or <strong>mutations</strong>) are automatically seeded into your code, then
your tests are run. If your tests fail then the mutation is <strong>killed</strong>, if your tests pass then
the mutation <strong>lived</strong>. </p>
<p class="marketing-info">The quality of your tests can be gauged from the percentage of mutations killed.</p>
<h1>What?</h1>
<p class="marketing-byline">Really it is quite simple</p>
<p class="marketing-info">To put it another way - PIT runs your unit tests against automatically modified
versions of your application code. When the application code changes, it should produce different results and
cause the unit tests to fail. If a unit test does not fail in this situation, it may indicate an issue with
the test suite. </p>
<h1>Why?</h1>
<p class="marketing-byline">What's wrong with line coverage?</p>
<p class="marketing-info">Traditional test coverage (i.e line, statement, branch, etc.) measures only which code
is <strong>executed</strong> by your tests. It does <strong>not</strong> check that your tests are actually
able to <strong>detect faults</strong> in the executed code. It is therefore only able to identify code that
is definitely <strong>not tested</strong>.</p>
<p class="marketing-info">The most extreme examples of the problem are tests with no assertions. Fortunately
these are uncommon in most code bases. Much more common is code that is only <strong>partially tested</strong>
by its suite. A suite that only partially tests code can still execute all its branches <i>(<a
href="/weak_tests">examples</a>)</i>.</p>
<p class="marketing-info">As it is actually able to detect whether each statement is meaningfully tested,
mutation testing is the <strong>gold standard</strong> against which all other types of coverage are measured.
</p>
<h1>Why PIT?</h1>
<p class="marketing-byline"></p>
<p class="marketing-info">There are other mutation testing systems for Java, but they are not widely used.
<p>
<p class="marketing-info">They are mostly slow, difficult to use and written to meet the needs of academic
research rather than real development teams.
<p>
<p class="marketing-info">PIT is different. It's</p>
<ul class="marketing-info">
<li><strong>fast</strong> - can analyse in <strong>minutes</strong> what would take earlier systems
<strong>days</strong></li>
<li><strong>easy to use</strong> - works with ant, maven, gradle and others</li>
<li><strong>actively developed</strong></li>
<li><strong>actively supported</strong></li>
</ul>
<p class="marketing-info">The reports produced by PIT are in an easy to read format combining <strong>line
coverage</strong> and <strong>mutation coverage</strong> information.</p>
<p><img src="/images/wicket_core_example32.png" />
<p><i>Example snippet taken from coverage report of <a href="http://wicket.apache.org/">Wicket Core</a></i></p>
<p><i>Light green shows line coverage, dark green shows mutation coverage.</i></p>
<p><i>Light pink show lack of line coverage, dark pink shows lack of mutation coverage.</i></p>
<h1>How to use it?</h1>
<p class="marketing-byline"></p>
<p class="marketing-info">The most effective way to use mutation testing is to run it <strong>frequently</strong> against only the code that has been changed.<p>
<p class="marketing-info">Once it has been integrated into the build file, pitest can be run locally by developers, or automatically against pull requests and merge requests using <a href="https://www.arcmutate.com">arcmutate</a>.</p>
<h1>Pro Version</h1>
<p class="marketing-info">Produced by the same team, <a href="https://www.arcmutate.com">arcmutate</a> extends pitest, adding support for Kotlin, Spring, Git and more.
<p>
</div>
</div>
<hr class="soften">
<h1>Success stories</h1>
<p class="marketing-byline"></p>
<div class="row-fluid">
<div class="span12">
<img src="/images/ladders-logo.jpg" />
<p><br />"... While we also used Clover for basic code coverage, as we got our PIT mutation coverage up into the
90s I stopped paying much attention to Clover"</p>
</p>"... This gave us extreme confidence in our tests ... The effects of that confidence were outstanding."</p>
<p><i>Kyle Winter, Lead Software Engineer, The Ladders</i></p>
</div>
</div>
<div class="row-fluid">
<div class="span12">
<img src="/images/sky_summer_logo.png" />
<p>". . . from my own personal experience of using PIT I've found it not only gives me confidence in the quality
of both my own and others unit test quality, but has actually been a design aid in so much that as well as
finding untested code, it can also find redundant code that when deleted still implements the intended
functionality."</p>
<p><a href="/sky_experience">more</a></p>
<p><i>Matt Kirk, Lead Developer, British Sky Broadcasting</i></p>
</div>
</div>
</div>
</div>