C. MetricsIn addition to counting v

C. Metrics
In addition to counting vulnerabilities, our BuildBot configuration
collected and analyzed the following software
metrics: SLOC, cyclomatic complexity, and nesting complexity.
While SCA is a commercial static analysis tool, our
other metric tools were open source tools packaged with
Ubuntu Linux 10.04. Cyclomatic Complexity and Nesting
Complexity were computed using PHP CodeSniffer 1.10 [6]
with custom classes and a Ruby script to extract only the
data we needed.
In our previous study, we used SLOCCount 2.26 [25]
to measure SLOC. However, this tool returned inconsistent
results when run multiple times on the same source code,
sometimes failing to report results at all. Therefore, we
used Fortify SCA to count code in this study, which returns
consistent SLOC counts for code, and as it is the same tool
that we used to find vulnerabilities, we expect that it counts
the same code that it finds vulnerabilities in. We still run
SLOCCount on each revision as part of a check to ensure
that our BuildBot-based system produced the same results as
our previous study. SLOC metrics from SCA are lower than
those from SLOCCount, as SCA SLOC does not include
lines with only braces or declarations.
We measured vulnerability density using the static analysis
vulnerability density (SAVD) metric [7] We used results
from Fortify SCA version 5.10 to compute SAVD, as SCA
reported both the number of vulnerabilities and SLOC. This
means that both the numerator and denominator of SAVD
differ from the original study, which results in much higher
SAVD values due to both the larger number of vulnerabilities
found and the smaller code size values. Since Fortify SCA
also categorized vulnerabilities into types such as cross-
Figure 1. SLOC vs. Vulnerabilities for All Project Versions
Table II
AGGREGATE DATA
Datum 2006 2007 2008 2009 2010
SLOC 684,654 782,870 864,113 980,029 1,182,917
Vulnerabilities 19,253 17,404 24,529 24,707 23,613
OWASP Top 10 14,742 12,467 17,970 17,623 17,389
SAVD 28.12 22.23 28.38 25.21 19.96
site scripting and SQL injection, we could also measure
vulnerability density for particular types of vulnerabilities.
IV. AGGREGATE FINDINGS
The size of the aggregate code base for all fourteen
projects steadily grew from 684,654 sources line of code
in mid-2006 to 1,182,917 lines of code in mid-2010. The
number of vulnerabilities also grew throughout that time
period from 19,253 to 23,613, though not steadily, as there
was a large decrease in 2007 followed by a large increase
in 2008. We found that the relationship between the number
of vulnerabilities in a version and the code size was roughly
linear and plotted in Figure 1 using a logarithmic scale.
Clusters of data points are typically different versions from
the same application. As the number of vulnerabilities grew
slower than the the size of the code, vulnerability density
decreased strongly from 28.12 vulnerabilities per thousand
lines of code to 19.96. Table II summarizes the findings of
this analysis.
While the overall trend was towards lower vulnerability
density, individual projects evolved differently, with eight
projects decreasing vulnerability density over the study and
six projects increasing. In our previous study, only six
projects had declining vulnerability densities. While SAVD
declined for eight projects, the total number of vulnerabilities
only declined for four of those projects: phpbb, po,
smarty, and squirrelmail.
Figure 2. SAVD Evolution by Project
Vulnerability densities varied widely, from 2.1 vulnerabilities
per thousand lines of code (achievo) to 202 (po)
in June 2006, evolving to 2.7 (achievo) to 206 (po) in
June 2010. Figure 2 shows the evolution of SAVD for each
of the fourteen projects from year to year. Both of the
highest SAVD projects, phpmyadmin and po, contained a
set of vulnerabilities made up mostly of cross-site scripting
vulnerabilities, and had hundreds of commits related to internationalization
efforts, which caused swings of thousands
of vulnerabilities within a week or two at times.
Figure 3 plots the SLOC and vulnerability counts for
WordPress over the four-year period. In early 2007, a release
of WordPress eliminated about a thousand vulnerabilities;
however as new code was added, so were new vulnerabilities.
WordPress contributors demonstrated that they can
write more secure software; however, there are very few
corrections being made after 2007. This approach seems
characteristic of a project that does not have consistent security
processes. However, squirrelmail, as shown in Figure 4,
consistently fixed vulnerabilities over the four year period,
without introducing a large number of new errors, even as
the code grew in size.

結果 (中文) 1:

C.衡量标准除了计数漏洞，我们 BuildBot 配置收集并分析以下软件度量 ︰ SLOC，圈复杂度和筑巢的复杂性。SCA 是一个商业静态分析工具，而我们其他度量工具是开放源码工具打包在一起Linux Ubuntu 10.04。圈复杂度和嵌套复杂性计算使用 PHP CodeSniffer 1.10 [6]使用自定义类和一个 Ruby 脚本，只提取我们所需的数据。在我们以前的研究中，我们使用 SLOCCount 2.26 [25]来衡量 SLOC。但是，此工具返回不一致结果当多次运行在相同的源代码，有时根本未报告结果。因此，我们用于强化 SCA 计数在此研究中，返回代码一致 SLOC 计数的代码，以及它是相同的工具我们用于查找漏洞，我们期待它计数相同的代码，它发现中的漏洞。我们仍然运行SLOCCount 对每个修订作为检查的一部分，以确保我们基于 BuildBot 的系统产生相同的结果我们前期的研究。从 SCA SLOC 度量是低于那些从 SLOCCount，作为 SCA SLOC 不包括只有大括号或声明的行。我们衡量脆弱性密度使用静态分析漏洞密度 (SAVD) 度量 [7] 我们使用结果从巩固 SCA 版本 5.10 计算 SAVD，作为 SCA报告数目的脆弱性和 SLOC。这意味着，分子和分母的 SAVD不同于最初的研究结果在高得多由于这两个较大的漏洞数量 SAVD 值发现和更小的代码大小的值。因为强化 SCA此外进行了分类类型，如跨进漏洞图 1。SLOC 与漏洞项目的所有版本表二聚合数据基准 2006年 2007年 2008年 2009年 2010SLOC 684,654 782,870 864,113 980,029 1,182,917漏洞 19,253 17,404 24,529 24,707 23,613OWASP 顶部 10 14,742 12,467 17,970 17,623 17,389SAVD 28.12 22.23 28.38 25.21 19.96站点脚本和 SQL 注入，我们也可衡量漏洞密度为特定类型的安全漏洞。四.聚合结果基地所有十四聚合代码的大小项目稳步增长从 684,654 源行的代码在 2006 年中期到 2010 年年中代码 1,182,917 行。的安全漏洞数量也增长在这整个期间期间从 19,253 到 23,613，虽然不稳，作为那里在 2007 年之后大增加大幅度降低在 2008 年。我们发现数量之间的关系版本和代码中的漏洞的大小大约是线性和绘制在图 1 中使用对数刻度。成群的数据点是通常不同版本同一应用程序。安全漏洞的数量随着比慢的代码漏洞密度的大小强烈减少了从千分之 28.12 漏洞行至 19.96 代码。表二总结的结果这种分析。虽然总的趋势是降低脆弱性密度，个别项目不同，随着八降低脆弱性密度对其研究项目和增加的六个项目。在我们前面的研究，只有六个项目一度减少脆弱性密度。虽然 SAVD八个工程项目，安全漏洞的总数量下降仅下降四这些项目 ︰ phpbb，大埔，smarty 和 squirrelmail。图 2。SAVD 演化的项目漏洞密度相差很大，从 2.1 漏洞每千行代码 （大展） 到 202 (po)2006 年 6 月，演变到 2.7 （大展） 至 206 (po)2010 年 6 月。图 2 显示 SAVD 的演化为每个一年到一年的 14 个项目。两个最高的 SAVD 项目、 phpmyadmin 及大埔，载组的组成主要的跨站点脚本漏洞漏洞，和数百名提交涉及国际化努力，造成数以千计的波动在一两个星期的时间内安全漏洞。图 3 地块 SLOC 和漏洞计数在四年期间的 WordPress。在 2007 年年初，释放WordPress 的消除关于一千的脆弱性;不过添加了新的代码，因此，是新的漏洞。WordPress 派遣证明他们可以编写更安全的软件;然而，也有极少数2007 年后作了更正。这种方法似乎一个项目，并没有一致的安全特性进程。然而，squirrelmail，如图 4 所示在四年期间，一直固定漏洞而不会引入大量的新的错误，甚至作为代码变得越来越大。