Design and Implementation of a Conversational Health Question Answering System

Abstract

Conversational agents offer a more accessible method of retrieving information than traditional search engines. In contrast to these more impersonal tools, such as Google or Wikipedia, conversational agents offer a conversation-like interaction which enhances the user experience by asking follow-up questions and creating a sense of dialogue. This interactive approach is particularly valuable in healthcare, where users often include older adults less familiar with technology and individuals facing loneliness.

This thesis presents the development of a voice-based conversational agent that answers consumer health questions. At its core, this agent uses a similarity-based approach, employing a dataset of question-answer pairs as its knowledge base. As such medical question-answering datasets barely exist in the literature, this work addresses the gap by developing a high-quality dataset. For this purpose, a pipeline consisting of several steps is created, which automatically generates question-answer pairs with the help of a large language model.

Moreover, the agent contains a recommendation system, suggesting new questions to the user and therefore enhancing the conversation-like feeling. The efficacy as well as the user experience offered by the agent were assessed with two different strategies. The results of the evaluation confirm that the presented approach improves accessibility of health information and maintains a pleasant conversational experience.

Research Questions

1. Which existing approaches are there in building conversational question answering agents in the context of health-related topics?
2. ​How to construct a dataset with question-answer pairs?​
3. How to develop a pipeline for conversational question answering using the scraped dataset?​
4. Which evaluation methods can be used to assess the performance and effectiveness of the developed system?​