Master's Thesis Ihsan Soydemir

Abstract

Detecting and correcting hallucinations in text generated by large language models remains a challenging task today. Even the most advanced and powerful language models often struggle with this issue, and hundreds of approaches have been proposed to address it. This study will focus on post-hoc approaches, which aim to identify and correct hallucinations after the text has been generated.

While current post-hoc techniques demonstrate some success with short or one-sentence claims, they fall short regarding longer-form content. This research aims to analyze existing methods for reducing hallucinations in long-form claims and develop a taxonomy of hallucinations. The strengths and weaknesses of current approaches will be evaluated, and necessary enhancements will be made to improve their performance in long-form content.

Goal

This research aims to analyze and improve post-hoc approaches for detecting and correcting hallucinations in long-form text generation. The following three research questions will be addressed:

1. What is an appropriate taxonomy for categorizing hallucinations in large language models (LLMs), such as numeric and semantic hallucinations?
2. How can the Retrieval-Augmented Generation (RAG) model be effectively applied to handle long contexts without compromising performance or efficiency?
3. How can we optimally balance editing and faithfulness when refining generated text?