Coverity Scan: Open Source Code Is Better Quality

Critics of open source programming may use the OpenSSL “Heartbleed” disaster to claim otherwise, but a new report from Coverity says open source code now beats proprietary software in quality. This week, the company released results of the 2013 Coverity Scan Open Source report, which found that, by Coverity’s metrics, open source code has surpassed proprietary code in quality for the first time.

Coverity, which is now part of Synopsys (SNPS), first began scanning open source software for quality in 2006, through a partnership with the Department of Homeland Security. Coverity now runs the scans independently.

In 2013, the Coverity Scan analyzed 750 million lines of open source code that came primarily from C and C++ projects, along with some Java projects. It then compared that code with proprietary software sources gleaned from “an anonymous sample of enterprise projects.”

In what will be welcome news for open source fans, the Coverity Scan analysis found that defect density—the number of defects per 1,000 lines of code—was 0.59 among the open source code it analyzed, compared to 0.72 for proprietary code. Put another way, that means the open source code in the sample had 22 percent fewer errors than its proprietary cousin.

For Linux kernel code in particular, the defect density, at 0.61, was slightly above the open source average. But Coverity says Linux remains “a benchmark for open source quality,” especially since kernel developers have reduced the time they take to fix a defect from 122 to six days in the years since Coverity first began analyzing Linux code in 2008.

To be sure, the debate over which approach to software development—open source or proprietary—is superior is not one that a scan of a limited corpus of mostly C and C++ code can end. And there’s much more to code quality than defect density. Still, Coverity’s findings on open source code quality—which the company is now making publicly available on an ongoing basis as it continues to monitor the code of participating projects—are good news for the open source community during a time when proprietary developers are sure to seize on the Heartbleed fiasco to denounce the “many eyes make all bugs shallow” mantra.