Measuring Success

In the days before open source, measuring the success of a software effort was often defined in strictly financial terms. How many boxes—yes, boxes—of software did a company sell? That revenue was offset by the cost of support, authoring, and distribution (again, boxes), and voila! The profit was the measure of success.

In the open source world, it doesn’t quite work like that. For one thing, while there are commercially successful open source projects, the success of those projects is still not strictly tied to just the financials. Nor is success set against the common benchmark of downloads. It is very easy to presume, for instance, that if a software project has millions of downloads, it is doing very well. But just because users are consuming the software steadily does not mean there aren’t internal problems in the project’s community. There could be any number of factors set to bring the project to a halt. Conversely, there are many very successful projects that don’t have the headlines and the huge popularity.

So, if money and fame are not the success factors in open source software projects, what are?

For open source projects, success is not just measured in terms of commercial or consumptive success. Free and open source projects have become much more transparently developed thanks to the popularity of distributed version control and collaboration tools. While these tools, such as the ones based on git, have certainly given contributors a big advantage in putting projects together, another benefit is that they also allow a much more robust and quantitative observation of how projects are doing.

In the past, it was difficult to see in real time if the processes of a given project were successful or not. You would see the output of a project’s efforts, but not the steps the process took along the way to get there. At least, not without a lot of effort. So establishing a project’s health and success was done more with subjective methods, often based on sociological and business-oriented principles that have become ingrained in many cultures. Now, though, you can easily dive into the data shown by open development, and use that data to derive quantitative information about a project’s health. It is these concepts that this section explores.