Open-source software and proprietary software contain similar numbers of bugs per thousand lines of code. Software test tool company Coverity released its annual Coverity Scan: Open Source Integrity Report, in which it compares the codebases of open-source projects against those proprietary projects.
The results of the scan showed that in 37 million lines of open-source code across 45 projects, there was an average of .45 bugs per thousand lines of code. Proprietary code, which was sampled anonymously from Coverity’s customer base, included over 300 million lines of code across 41 projects, where it was found that there were .64 bugs per thousand lines. Coverity claimed the average for all software is 1 bug per thousand lines of code.
It should be noted that there was a significantly larger set of code to be had in the proprietary sample. There, the average project size was 7.5 million lines of code. In Coverity’s open-source sample, the average project size was only 832,000 lines.
In the open-source scan, the most common bugs were bugs that raise potential security issues, such as null-pointer dereferences and uninitialized variables. More overt issues, such as buffer overflows and insecure data handling were less common.
Coverity’s report also dissected the Linux kernel, PHP and PostgreSQL. Each one had exemplary ratios, especially when it came to high-risk bugs. But PHP and PostgreSQL both had the best ratios in the Scan report: PHP 5.3 with .20, and PostgreSQL 9.1 with .21 bugs per thousand lines.
Zack Samocha, the Coverity Scan Project’s director, said that larger projects are at a bit of a disadvantage if you’re judging them entirely based on their defects per thousand lines of code. “The reason you have higher [defect] density in larger projects is that usually those are older, bigger applications that have been out in the market for a while. They just have more defects,” he said.