Abstract:
Hallucination has been widely recognized to be a significant drawback for large
language models (LLMs). There have been many works that attempt to reduce the
extent of hallucination. These efforts have mostly been empirical so far, which
cannot answer the fundamental question whether it can be completely eliminated.
In this paper, we formalize the problem and show that it is impossible to eliminate
hallucination in LLMs. Specifically, we define a formal world where hallucina-
tion is defined as inconsistencies between a computable LLM and a computable
ground truth function. By employing results from learning theory, we show that
LLMs cannot learn all of the computable functions and will therefore always hal-
lucinate. Since the formal world is a part of the real world which is much more
complicated, hallucinations are also inevitable for real world LLMs. Furthermore,
for real world LLMs constrained by provable time complexity, we describe the
hallucination-prone tasks and empirically validate our claims. Finally, using the
formal world framework, we discuss the possible mechanisms and efficacies of
existing hallucination mitigators as well as the practical implications on the safe
deployment of LLMs.
LLMs are bad for the uses they've been recently pushed for, yes. But this is legitimately a very good use of them. This is natural language processing, within a narrow scope with a specific intention. This is exactly what it can be good at. Even if does have a high false negative rate, that's still thousands and thousands of true positive cases that were addressed quickly and cheaply, and that a human auditor no longer needs to touch.