Title

Goals and KPIs in Large-scale Agile Development: A Systematic Literature Review

Abstract

Although agile software development was originally designed for use in small, single-team projects [1], the adoption of agile software development in large-scale projects has increased over the last years [2]. Within the adoption of large-scale agile development, coordination in multi-team environments and hierarchical management and organizational boundaries have proven to be challenging [3]. With implementing agile at large organizations, also comes the need to introduce software measurements into the development process. Software measurement has been widely studied in traditional software development [4], and over the last years also some literature reviews about agile and lean software development have been performed [5], [6]. However, in a systematic mapping study about large-scale agile software development, Uludat et al. [7] identified that so far, there is no research stream about measurements in large-scale agile software development. Since no systematic literature review about metrics in large-scale agile software development has been performed, this master’s thesis aims to close this research gap. A systematic literature review was conducted to identify goals and metrics in large-scale agile software development and the challenges and success factors of introducing and using metrics. With the search terms, 17042 publications were identified. Out of those, 109 primary studies, about goals and metrics in large-scale agile software development were identified. More than 80% of studies were published in conferences, and 33% of sources were experience reports, indicating a lack of maturity. We found that most measurements were taken on team and program level. Even though multiple studies indicate the use of metrics at the portfolio level, a lack of actually described metrics has been found. Furthermore, 21 challenges of using and implementing metrics in large-scale agile software development have been identified that could be grouped into six categories. The most frequently mentioned challenges were problems with data availability and data collection. Moreover, 22 success factors grouped into ten categories were found. The most prominent success factors were automated data collection, making metrics visible to everyone, and introducing metrics in an iterative fashion.

Research Questions

Research question 1 (RQ1): What are goals and KPIs of large-scale agile development in scientific literature?

Research question 2 (RQ2): What are the challenges of using and implementing metrics in large-scale agile software development?

Research question 3 (RQ3): What are the success factors of using and implementing metrics in large-scale agile software development?

Sources

[1] D. Cohen, M. Lindvall, and P. Costa. “An introduction to agile methods.” In: Adv.Comput.62.03 (2004), pp. 1–66.

[2] 10th annual ”state of agile development” survey. 2016. URL:http://www.agile247.pl/wp-content/uploads/2016/04/VersionOne-10th-Annual-State-of-Agile-Report.pdf.

[3] K. Dikert, M. Paasivaara, and C. Lassenius. “Challenges and success factors for large-scale agile transformations: A systematic literature review”. In: Journal of Systems andSoftware119 (2016), pp. 87–108.

[4] A. Meidan, J. A. Garcıa-Garcıa, I. Ramos, and M. J. Escalona. “Measuring software process: a systematic mapping study”. In: ACM Computing Surveys (CSUR)51.3 (2018), pp. 1–32

[5] E. Kupiainen, M. V. Mäntylä, and J. Itkonen. “Using metrics in Agile and LeanSoftware Development–A systematic literature review of industrial studies”. In: Information and Software Technology62 (2015), pp. 143–163

[6] M. Feyh and K. Petersen. “Lean software development measures and indicators-a systematic mapping study”. In: International Conference on Lean Enterprise Software and Systems. Springer. 2013, pp. 32–47.

[7] Ö. Uludag, P. Philipp, A. Putta, M. Paasivaara, C. Lassenius, and F. Matthes. “Reveal-ing the State-of-the-Art in Large-Scale Agile Development: A Systematic MappingStudy”. In: arXiv preprint arXiv:2007.05578(2020).