Augmenting Knowledge-Based Conversational Search Systems With Large Language Models

 

Motivation

Conversational Interfaces are increasingly used by websites and apps, with notable examples like Microsoft 365 Copilot or the language learning app Duolingo. This trend reflects a shift towards natural language in human-computer interaction, facilitated by utilizing Large Language Models (LLMs) to generate responses aligned with user intents. Although this approach is effective for tasks like text summarization and text generation, there are still some shortcomings for conversational search. Issues include hallucination, outdated information, data lineage, and reliability. Integrating a knowledge base as a grounding mechanism into conversational search can mitigate these problems.

 

Objective

This thesis aims to assess the potential of using LLMs to enhance the Natural Language Understanding (NLU) and Natural Language Generation (NLG) components of knowledge-based conversational search systems. Specifically, we want to investigate the suitability of LLMs and different prompting techniques for semantic parsing, which involves translating a user's intent into a database query such as SPARQL. In addition, we want to evaluate the use of LLMs for data-to-text generation. This process includes generating a response based on the user's intent and the retrieved data from the database.

Research Questions

1. Which previous studies have investigated using Large Language Models for the tasks of semantic parsing and text generation?
2. What selection of Large Language Models and Prompting techniques are suitable for a comparative analysis of the considered tasks?
3. How capable are the selected Large Language Models and prompting strategies for semantic parsing and triples-to-text generation based on automatic and human evaluation?