Intelligent Channel Navigation in Customer Service Using Large Language Models

 

Abstract

This thesis touched upon three topics in the area of customer service in collaboration with a large European insurance company: channel decision factors, Large Language Model (LLM)-powered channel preference prediction and generation of reasons for such preferences. A structured literature review was conducted to uncover potentially relevant customer choice determinants for channel selection and combined with a workshop and interviews in the case study company. An LLM-based communication channel classifier for incoming requests in customer service was developed and various prompt and input strategies tested for appropriateness. The inclusion of many user related data points, examples (few-shots) and Chain-of-Thought prompting proved to be a strong combination and delivered on-par performance with simple learning-based approaches such as regression, albeit both scored moderate results in general. The LLM was further tested for its capability of generating similar reasons to those provided by real humans which worked well for simple reasons but seems to be challenging overall.

 

Research Questions

1. In customer service centers, what are relevant factors for deciding the optimal channel for customer service requests?

2. How do different input factors and prompt strategies influence the effectiveness of LLMs in selecting appropriate communication channels for customer service requests?

 

3. How well do LLMs predict the reasons for choosing a customer service channel?