Master's Thesis Timo Zandonella

Abstract

New secure software engineering challenges have emerged for large-scale agile development,
such as an increase in malicious attacks and subsequent data breaches. To address them, more
security measures need to be taken, many of which fall under the responsibility of a central
security governance unit. Due to the increased workload, security governance often cannot
keep up with the agile pace of the development teams, which can result in development
teams being limited in how efficiently they can develop. One way to increase efficiency
and security levels in development teams is to give security-capable teams more freedom
and autonomy in their security decisions. Measuring a team’s security capability can be
achieved using security maturity scores, which systematically assess a team and its processes
in different areas. Assessing often relies on manual self-assessments by team members or
external audits by security specialists in the form of questionnaires, which can be skewed
and time-consuming to complete.
We propose secure software engineering metrics as a complementary team security maturity
tool to address the existing issues with assessments. Secure software engineering metrics
measure team-wide or product-specific attributes, such as the number of unmitigated vulner-
abilities or the team’s knowledge of security policies. Measurement is continuous and occurs
at all stages of the software development life cycle. However, there is no recognized collection
of high-quality metrics, and those proposed to date are often described in an unstructured
way. In addition, existing security maturity models do not use secure software engineering
metrics to calculate a security maturity score.
To address this research gap, we first create a structured catalogue of security metrics. We
establish rigorous qualification criteria for the metrics and collect the information using a
standardized catalogue format for software metrics. Additionally, we research which tools
can automatically measure the catalogued secure software engineering metrics. Second, we
integrate the cataloged metrics into a maturity model to complement the (self-)assessments
and propose a procedure to calculate a maturity score from the assessments and the secure
software engineering metrics. Third, we implement a prototype web application that en-
ables a development team and decision makers to track security metrics and their maturity
throughout the software development lifecycle.

Research Questions

• Research question 1 (RQ1): Which security metrics exist and how can they
  automatically be captured with the support of security tools?
• Research question 2 (RQ2): How can security metrics be used to assess the
  security maturity of an agile development team?
• Research question 3 (RQ3): How can a team’s security maturity be calculated,
  represented and visualized in a self-assessment tool?