Design and Implementation of an Expert Recommendation System for Making Design Decisions

Motivation

Several empirical studies indicate that software architects and designers do not always take rationalistic approach but favor a naturalistic approach to decision making (Tang, Razavian, Paech, & Hesse, 2017) (Zannier, Chiasson, & Maurer, 2007). In this context, the personal experience of architects influences the design decision-making process. The mental representation of a system by a designer is crucial, however cognitive limitations to handle complex systems or the usage of first hand good enough solutions limit the decisions. Furthermore, decisions for various parts of software systems are done by single persons or small groups (Tang, Razavian, Paech, & Hesse, 2017) (van Vliet & Tang, 2016). Design decisions made by these decision-makers are seldom explicitly stated in a formal documentation and if they are, accessibility remains a challenge (Jansen & Bosch, 2005). People need to rely on these so-called experts and their tacit knowledge. As the modern working environment drifts to collaboration of distributed teams, access to such knowledge is even more important to build consistent systems based on sustainable design decisions to achieve specified quality metrics. This work aims to address the aforementioned challenges by designing an expert recommender which enables to access relevant knowledge in a simplified and automated way.

Research Questions

In this master thesis, the following questions should be answered to solve the named issues:

• How to quantify/measure the expertise of software architects and developers in the context of design decision-making process?
• What are the separate roles taken during the decision-making process to address design decisions?
• How to automatically identify experts to address design decisions and allocate roles to the recommended experts?

Related Work

In the proposed solution, we examine the so-called existence design decisions with their sub classification behavior, structural and non-existence (Kruchten, 2004). The existence classification describes decisions which specify if an element will be present within a system design or not. These are decisions we expect to find within code repositories and task management tools.

Concentrating on the existence design decisions, Baht et al. provided in their work a dataset with 781 design decisions. Furthermore, they split this dataset into the sub classifications provided by Kruchten mentioned before (Bhat, Shumaiev, Biesdorf, Hohenstein, & Matthes, Automatic Extraction of Design Decisions from Issue Management Systems: A Machine Learning Based Approach, 2017). These results provide for both the role development and the data processing a valid starting point.

Furthermore, as architecture knowledge management is not just a project related issue, a possible implementation into the AMELIE system can be considered. The AMELIE system collects from various sources as repositories or task management tools and projects and maps them to a meta-model (Bhat, et al., 2016).

An example for an expertise browser deliver Mockus & Herbsleb, 2002. They analyzed an existing software repository and visualized them into an expertise browser for finding key personnel. Hereby, they describe how to present such an expertise browser to the user and several necessary functionalities as a search functionality.

Capilla et al. describe common benefits, barriers and possible improvements for Architecture Knowledge Management. Additionally, they describe in research recognized different usage contexts based on the type of a project (agile, OSS, worldwide collaboration,…) of architecture knowledge management (Capilla, Jansen, Tang, Avgeriou, & Babar, 2016). Creating an architecture knowledge expert browser, these results should be taken into consideration.

Several evaluations strategies for validating a recommender system in software engineering exist. Dimensions as correctness, diversity, risk or even privacy should be considered and eventually measured (Avazpour, Pitakrat, Grunske, & Grundy, 2014).

Approach

The approach for answering these questions is separated into two main tasks:

First, answer above research questions in a theoretical analysis based on existing literature and common design principles that leads to a creation of a theoretical framework solving the issues.

Second, evolve the theoretical framework in an agile manner into a working prototype implementation for the existing AMELIE system.

The first step concentrates mainly on tasks building the theoretical framework.

• Identify & define common roles of architecture decision makers within software projects
• Analyze extracted and already available issues from Baht et al. and map them to roles (e.g. extended data model necessary)
• Define functional and non-functional requirements for an expertise browser
• Design an expertise browser based on the defined requirements

The second step focuses on the implementation.

• Implement a processing pipeline for mapping design to roles
• Implement the expertise browser

As a last step, the implementation of a design decision expert browser needs to be evaluated about its benefits.

Outlook

The main target of this master thesis is to create an example based implementation for an design decision browser for experts. This tool should support the benefits and limit known barriers for architecture knowledge management. Key decision-makers for design decisions should be easy findable and accessible, employees need to document less through the automatic processing and overall software quality should increase as common design principles are easier to follow.

Sources

Avazpour, I., Pitakrat, T., Grunske, L., & Grundy, J. (2014). Dimensions and metrics for evaluating recommendation systems. In M. Robillard, W. Maalej, R. Walker, & T. Zimmermann, Recommendation systems in software engineering (S. 245-273). Heidelberg: Springer.

Bhat, M., Shumaiev, K., Biesdorf, A., Hohenstein, U., & Matthes, F. (2017). Automatic Extraction of Design Decisions from Issue Management Systems: A Machine Learning Based Approach. European Conference on Software Architecture (S. 138-154). Springer.

Bhat, M., Shumaiev, K., Biesdorf, A., Hohenstein, U., Hassel, M., & Matthes, F. (2016). Meta-model based framework for architectural knowledge management. Proccedings of the 10th European Conference on Software Architecture Workshops (S. 12-19). ACM.

Bhat, M., Shumaiev, K., Biesdorf, A., Hohenstein, U., Hassel, M., & Matthes, F. (2017). An Ontology-based Approach for Software Architecture Recommendations. 23rd Americas Conference on Information Systems (AMCIS) 2017. Boston, MA, USA.

Capilla, R., Jansen, A., Tang, A., Avgeriou, P., & Babar, M. A. (June 2016). 10 years of software architecture knowledge management: Practice and future. Journal of Systems and Software, S. 191-205.

Galster, M., Tamburri, D. A., & Kazman, R. (05. 09 2017). Towards Understanding the Social and Organizational Dimensions of Software Architecting. (N. U. ACM New York, Hrsg.) ACM SIGSOFT Software Engineering Notes, 42(3), S. 24-25.

Jansen, A., & Bosch, J. (2005). Software architecture as a set of architectural design decisions. WICSA 2005. 5th Working IEEE/IFIP Conference on Software Architecture (S. 109-120). IEEE.

Kruchten, P. (2004). An ontology of architectural design decisions in software intensive systems. 2nd Groningen workshop on software variability, (S. 54-61).

Mockus, A., & Herbsleb, J. D. (2002). Expertise browser: a quantitative approach to identifying expertise. Proceedings of the 24th international conference on software engineering (S. 503-512). ACM.

Tang, A., Razavian, M., Paech, B., & Hesse, T. M. (2017). Human Aspects in Software Architecture Decision Making: A Literature Review. 2017 IEEE International Conference on Software Architecture (ICSA) (S. 107-116). IEEE.

van Vliet, H., & Tang, A. (2016). Decision making in software architecture. Journal of Systems and Software, 117, S. 638-644.

Ven, J. S., & Bosch, J. (2013). Making the Right Decision: Supporting Architects with Design Decision Data. European Conference on Software Architecture: ECSA 2013: Software Architecture (S. 176-183). Berlin, Heidelberg: Springer.

Zannier, C., Chiasson, M., & Maurer, F. (2007). A model of design decision making based on empirical results of interviews with software designers. Information and Software Technology, 49(6), S. 637-653.