The good news is that software keeps getting better, with fewer than one error per thousand lines of code. The bad news is that both large open-source projects and small proprietary software projects tend to have worse quality than average.
Development testing service Coverity’s annual scan report, which is based on data from almost 500 software projects with a total of over 450 million lines of code, says that almost 230,000 defects were found and fixed. And while the average defect density per thousand lines of code was almost identical between open source and proprietary, there was an interesting diversion in the results.
Open source projects, Coverity says, tend to have .69 bugs per thousand lines of code, virtually the same as proprietary software, which tends to have .68 errors per thousand lines. But large closed-source projects — over one million lines of code — tend to have 33 percent fewer errors than small closed-source projects, with .66 errors over each thousand lines of larger projects compared to .98 in smaller projects. And small open source projects have a massive 70 percent fewer errors than large open source software, with only .44 defects compared to .75.
The difference, according to Coverity, is that small open source projects are labors of love by individual developers or small teams, who carefully comb through their code to reduce errors. Large open source projects, on the other hand, tend to lack standardized processes to ensure code quality, and so the error rate increases.
In commercial or closed-source software, developers experience almost the opposite conditions. Large projects tend to have well-defined formal testing processes, which ensure higher code quality, and small projects tend to be hasty, quick endeavors that show the effects of growing pains, as no standardized testing is in place.
In other words, if you’re looking for bug-free apps, look for a small open source project or a large proprietary piece of software, because those have the best chance of having few defects and high overall code quality.
All of the data in infographic form:
VB's research team is studying web-personalization... Chime in here, and we’ll share the results.