Improving the Developer Experience of API Consumers using Usage Scenarios and Examples

Abstract

Working with Application Programming Interfaces (APIs) has become an important part of software developers everyday work. Programmers or API Consumers heavily rely on the API documentation and its different components. A key resource to learn APIs are examples and usage scenarios. The lack of examples, low quality in documentation, and benefits of samples are thoroughly researched topics. Previous work has shown that these problems significantly hamper productivity and developer experience. While there is a substantial body of knowledge regarding API documentation, only a fraction of it is dedicated to examples and usage scenarios. There is a lack of research regarding the individual contribution of these crucial resources. Further, the challenges in this field are mostly studied from a developers perspective. Recent papers call for studies evaluating API documentations with regards to developer experience.
We aim to fill the mentioned gaps by conducting a study including API providers and consumers. We carried out 14 interviews with IT professionals at a large German software vendor and elicited knowledge types to be included in the documentation. Further, we design two API documentations out of which one is optimized for developer experience and contains features found in previous research and our interviews. Our practical evaluation at the same company includes two groups of six software developers who we oppose with the same tasks but with different API documentations to study the impact of the differences. We observed and interviewed the study participants during the study to make implications about effects on the developer experience.
Our findings reveal that improvements to usage scenarios and examples of an API documentation show significant effects on performance, task success, and perceived usability. Based on our quantitative and qualitative analysis, we give recommendations concerning features in API documentation that seemingly yield improvements to the developer experience. Our results uncover that examples alone do not necessarily help. Often, it is the complexity, coverage, or the references to other resources that API consumers value while working with documentations. Further, the results indicate dependencies between API characteristics such as complexity and the usefulness of API documentation features like the glossary.\\

Keywords: API, Documentation, Usage Scenarios, Examples, Developer Experience, Case Study, Requirements

Research Questions

RQ1: What are the approaches and concepts to create and publish API usage scenarios and examples?

RQ2: What are the knowledge types that must be included in API usage scenarios and examples?

RQ3: How can usage scenarios and examples be leveraged to improve the developer experience for API consumers?

References

Hoffman, Daniel; Strooper, Paul (2000): Prose+test cases=specifications. 

Meng, Michael; Steinhardt, Stephanie; Schubert, Andreas (2018): Application Programming Interface Documentation: What Do Software Developers Want? In: Journal of Technical Writing and Communication.

Meng, Michael; Steinhardt, Stefanie; Schubert, Andreas (2019): How Developers Use API Documentation: An Observation Study. In Communication Design Quarterly.

Nasehi, Seyed Mehdi; Maurer, Frank (2010): Unit tests as API usage examples.

Robillard, Martin P. (2009): What Makes APIs Hard to Learn? Answers from Developers.

Sohan, S. M.; Maurer, Frank; Anslow, Craig; Robillard, Martin P. (2017): A study of the effectiveness of usage examples in REST API documentation.