I. INTRODUCTION
Web applications are the source of almost half of all
security vulnerabilities, representing 49% of all vulnerability
disclosures reported in IBM’s 2010 X-Force Trend and
Risk Report [12]. This is probably an undercount as many
organizations develop and deploy web applications in-house
whose vulnerabilities are not reported publicly. MITRE
found that the most common two vulnerability types since
2005 were cross-site scripting and SQL injection [5], which
are primarily found in web applications.
While worms that exploit network or operating system
vulnerabilities have largely disappeared, web applications
are regularly targeted by mass attacks such as the April 2011
LizaMoon mass SQL injection attacks [14]. Furthermore,
popular open source web applications such as WordPress
are the subject of mass attacks designed to inject malware
to infect users of their sites [24].
This study is a partial replication of our previous study of
vulnerabilities in fourteen open source PHP web applications
from 2006-2008 [23]. While replication of experiments is
important in all areas of empirical software engineering, it
is particularly important in studies of software security, due
to the rapidly evolving nature of the field, with new types of
vulnerabilities appearing each year. Software that is thought
to be secure one year is discovered to be insecure the next
due to an absence of measures to prevent a type of intrusion
not known before. There were no worries about cross-site
scripting (XSS) before most browsers supported Javascript
in the 1990s or about clickjacking until 2008.
Like the original study, this paper analyzes relationships
between software metrics and vulnerabilities measured using
a static analysis tool in fourteen of the most widely used
open source web applications, including WordPress and Mediawiki,
the software on which Wikipedia runs. We mined
the source code repositories of these applications to measure
vulnerability density and to collect a variety of software
metrics, including our security resources indicator metric as
well as traditional metrics such as code size and complexity.
This is the largest survey of web application security, both
in terms of code size and number of application users.
There are two important differences between this study
and our previous one. The first is that this study analyzes
the web applications for four years, from 2006-2010, instead
of two. The longer time span allows us to verify whether
the relationships we discovered between complexity and
vulnerabilities were the product of the state of source code in
2006-2008 or whether they hold more generally. The second
is the use of a more recent version of the Fortify Source
Code Analyzer (SCA) tool, which can identify 73 types of
vulnerabilities in PHP code as opposed to only 13 types of
vulnerabilities in the version we used in the original study,
enabling us to find latent vulnerabilities that missed in the
previous study.