Master's Thesis Felix Weißl

Effects of integrating assistance features in a customer support system

Motivation

Customer service is a central element to stay competitive in the online market. The desired customer satisfaction is only possible with acceptable response times on e.g. customer emails.
This can lead to cost and scaling problems (find enough suitable employees, provide offices, coordination problem). Repetitive, laborious work aggravates this problem and does not contribute to the satisfaction of the employee. Using the right software tools is evident and has to be chosen specifically for the required workflow.

Machine Learning and Information Extraction (ML and IE) allow you to classify free text and extract important data from emails, whereupon assistance and automation systems can be built. The thesis is focused on observing the effects of integrating those assistance features in the customer support department of a large German comparison and contract management portal for energy providers (gas, electricity).

Approach

The use of such an assistance system offers opportunities and risks in terms of cost-effectiveness. By displaying context-relevant information (e.g., contract data) and offering further actions (e.g., cancel button), time and thus cost can be saved. Likewise, the display of false information (caused by misclassification) can cause additional work which will have the opposite effect.

User studies in handling such a system and determining utility metrics (e.g., time savings, number of steps in the process of processing a customer in the form of clicks) are intended to elucidate here.
It is possible to look at the customer support work without, as well as with the described functions on concrete test scenarios. The obtained test data can help in determining the usefulness in terms of cost-effectiveness, but also in terms of user acceptance. The user in this case is the customer support emloyee, which might feel subjected to paternalism.

Within his Master's thesis Michael Legenc developed an API supported by machine learning to classify mails for their purpose and priority. This work was done at the energy department of the German comparison portal CHECK24. Their customer suppor's current email client is, for various reasons, unsuitable for the extension of the described test environment (outdated framework, UI not suitable). It therefore makes sense to re-prototype it with sufficient functionality and to use it as a measuring instrument. The evaluation is concluded with a questionnaire to measure the user acceptance of the client.

Research Questions

• RQ1: Does the integration of NLP-based assistance features improve the processing time of email customer support?
• RQ2: How can email customer support performance be measured?
• RQ3: How does customer support handle false classifications?
• RQ4: How does customer support react to automation in its accustomed workflow?