llm-hallucination-survey

Hallucination refers to the generated content that while seemingly plausible, deviates from user input (input-conflicting), previously generated context (context-conflicting), or factual knowledge (fact-conflicting).

📰News

😎 We have uploaded a comprehensive survey about the hallucination issue within the context of large language models, which discussed the evaluation, explanation, and mitigation. Check it out!

Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models

If you think our survey is helpful, please kindly cite our paper:

@article{zhang2023hallucination,
      title={Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models}, 
      author={Zhang, Yue and Li, Yafu and Cui, Leyang and Cai, Deng and Liu, Lemao and Fu, Tingchen and Huang, Xinting and Zhao, Enbo and Zhang, Yu and Chen, Yulong and Wang, Longyue and Luu, Anh Tuan and Bi, Wei and Shi, Freda and Shi, Shuming},
      journal={arXiv preprint arXiv:2309.01219},
      year={2023}
}

🚀Table of Content

• LLM-Hallucination-Survey
  • News
  • Table of Content
  • Evaluation
  • Source
  • Mitigation
  • Contact

🔍Evaluation of LLM Hallucination

Input-conflicting Hallucination

This kind of hallucination denotes the model response deviates from the user input, including task instruction and task input. This kind of hallucination has been widely studied in some traditional NLG tasks, such as:

• Machine Translation:

  • Hallucinations in Neural Machine TranslationDownload Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fannjiang, David Sussillo [paper] 2018.9
  • Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translation Nuno M. Guerreiro, Elena Voita, André F.T. Martins [paper] 2022.8
  • Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better David Dale, Elena Voita, Loïc Barrault, Marta R. Costa-jussà [paper] 2022.12
  • HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translation David Dale, Elena Voita, Janice Lam, Prangthip Hansanti, Christophe Ropers, Elahe Kalbassi, Cynthia Gao, Loïc Barrault, Marta R. Costa-jussà [paper] 2023.05

• Data-to-text:

  • Controlling Hallucinations at Word Level in Data-to-Text Generation Clément Rebuffel, Marco Roberti, Laure Soulier, Geoffrey Scoutheeten, Rossella Cancelliere, Patrick Gallinari[paper] 2021.2
  • On Hallucination and Predictive Uncertainty in Conditional Language Generation Yijun Xiao, William Yang Wang[paper] 2021.3
  • Faithful Low-Resource Data-to-Text Generation through Cycle Training Zhuoer Wang, Marcus Collins, Nikhita Vedula, Simone Filice, Shervin Malmasi, Oleg Rokhlenko[paper] 2023.7

• Summarization:

  • On Faithfulness and Factuality in Abstractive Summarization Joshua Maynez, Shashi Narayan, Bernd Bohnet, Ryan McDonald[paper] 2020.5
  • Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization Meng Cao, Yue Dong, Jackie Chi Kit Cheung[paper] 2021.9
  • Summarization is (Almost) Dead Xiao Pu, Mingqi Gao, Xiaojun Wan[paper] 2023.9
  • Hallucination Reduction in Long Input Text Summarization Tohida Rehman, Ronit Mandal, Abhishek Agarwal, Debarshi Kumar Sanyal[paper] 2023.9
  • Lighter, yet More Faithful: Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarization George Chrysostomou, Zhixue Zhao, Miles Williams, Nikolaos Aletras[paper] 2023.11
  • TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarization Liyan Tang, Igor Shalyminov, Amy Wing-mei Wong, Jon Burnsky, Jake W. Vincent, Yu'an Yang, Siffi Singh, Song Feng, Hwanjun Song, Hang Su, Lijia Sun, Yi Zhang, Saab Mansour, Kathleen McKeown[paper] 2024.02

• Dialogue:

  • Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding Nouha Dziri, Andrea Madotto, Osmar Zaiane, Avishek Joey Bose[paper] 2021.4
  • RHO: Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding Ziwei Ji, Zihan Liu, Nayeon Lee, Tiezheng Yu, Bryan Wilie, Min Zeng, Pascale Fung[paper] 2023.7
  • DiaHalu: A Dialogue-level Hallucination Evaluation Benchmark for Large Language Models Kedi Chen, Qin Chen, Jie Zhou, Yishen He, Liang He[paper] 2024.3

• Question Answering:

  • Entity-Based Knowledge Conflicts in Question Answering Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, Sameer Singh[paper] 2021.9
  • Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering Vaibhav Adlakha, Parishad BehnamGhader, Xing Han Lu, Nicholas Meade, Siva Reddy [paper] 2023.7

Context-conflicting Hallucination

This kind of hallucination means the generated content exhibits self-contradiction, i.e., conflicts with previously generated content. Here are some preliminary studies in this direction:

1. Knowledge Enhanced Fine-Tuning for Better Handling Unseen Entities in Dialogue Generation Leyang Cui, Yu Wu, Shujie Liu, Yue Zhang[paper] 2021.9

2. A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation Tianyu Liu, Yizhe Zhang, Chris Brockett, Yi Mao, Zhifang Sui, Weizhu Chen, Bill Dolan[paper] 2022.5 (not only limited to context-conflicting type)

3. Large Language Models Can Be Easily Distracted by Irrelevant Context Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Schärli, Denny Zhou[paper] 2023.2

4. HistAlign: Improving Context Dependency in Language Generation by Aligning with History David Wan, Shiyue Zhang, Mohit Bansal[paper] 2023.5

5. Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation Niels Mündler, Jingxuan He, Slobodan Jenko, Martin Vechev [paper] 2023.5

Fact-conflicting Hallucination

This kind of hallucination means the generated content conflicts with established facts. This kind of hallucination is challenging and important for practical applications of LLMs, so it has been widely studied in recent work.

1. TruthfulQA: Measuring How Models Mimic Human Falsehoods Stephanie Lin, Jacob Hilton, Owain Evans [paper] 2022.5

2. A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation Tianyu Liu, Yizhe Zhang, Chris Brockett, Yi Mao, Zhifang Sui, Weizhu Chen, Bill Dolan [paper] 2022.5

3. A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, Pascale Fung [paper] 2023.2

4. HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, Ji-Rong Wen [paper] 2023.5

5. Automatic Evaluation of Attribution by Large Language Models Xiang Yue, Boshi Wang, Kai Zhang, Ziru Chen, Yu Su, Huan Sun [paper] 2023.5

6. Adaptive Chameleon or Stubborn Sloth: Unraveling the Behavior of Large Language Models in Knowledge Clashes Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, Yu Su [paper] 2023.5

7. LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond Philippe Laban, Wojciech Kryściński, Divyansh Agarwal, Alexander R. Fabbri, Caiming Xiong, Shafiq Joty, Chien-Sheng Wu [paper] 2023.5

8. Evaluating the Factual Consistency of Large Language Models Through News Summarization Derek Tam, Anisha Mascarenhas, Shiyue Zhang, Sarah Kwan, Mohit Bansal, Colin Raffel [paper] 2023.5

9. Methods for Measuring, Updating, and Visualizing Factual Beliefs in Language Models Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal, Srinivasan Iyer [paper] 2023.5

10. How Language Model Hallucinations Can Snowball Muru Zhang, Ofir Press, William Merrill, Alisa Liu, Noah A. Smith [paper] 2023.5

11. Evaluating Factual Consistency of Texts with Semantic Role Labeling Jing Fan, Dennis Aumiller, Michael Gertz [paper] 2023.5

12. FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, Hannaneh Hajishirzi [paper] 2023.5

13. Measuring and Modifying Factual Knowledge in Large Language Models Pouya Pezeshkpour [paper] 2023.6

14. KoLA: Carefully Benchmarking World Knowledge of Large Language Models Jifan Yu, Xiaozhi Wang, Shangqing Tu, Shulin Cao, Daniel Zhang-Li, Xin Lv, Hao Peng, Zijun Yao, Xiaohan Zhang, Hanming Li, Chunyang Li, Zheyuan Zhang, Yushi Bai, Yantao Liu, Amy Xin, Nianyi Lin, Kaifeng Yun, Linlu Gong, Jianhui Chen, Zhili Wu, Yunjia Qi, Weikai Li, Yong Guan, Kaisheng Zeng, Ji Qi, Hailong Jin, Jinxin Liu, Yu Gu, Yuan Yao, Ning Ding, Lei Hou, Zhiyuan Liu, Bin Xu, Jie Tang, Juanzi Li [paper] 2023.6

15. Generating Benchmarks for Factuality Evaluation of Language Models Dor Muhlgay, Ori Ram, Inbal Magar, Yoav Levine, Nir Ratner, Yonatan Belinkov, Omri Abend, Kevin Leyton-Brown, Amnon Shashua, Yoav Shoham [paper] 2023.7

16. Fact-Checking of AI-Generated Reports Razi Mahmood, Ge Wang, Mannudeep Kalra, Pingkun Yan [paper] 2023.7

17. Med-HALT: Medical Domain Hallucination Test for Large Language Models Logesh Kumar Umapathi, Ankit Pal, Malaikannan Sankarasubbu [paper] 2023.7

18. Large Language Models on Wikipedia-Style Survey Generation: an Evaluation in NLP Concepts

    Fan Gao, Hang Jiang, Moritz Blum, Jinghui Lu, Yuang Jiang, Irene Li [paper] 2023.8

19. ChatGPT Hallucinates when Attributing Answers Guido Zuccon, Bevan Koopman, Razia Shaik [paper] 2023.9

20. BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Models *Zican Dong, Tianyi Tang, Junyi Li, Wayne Xin Zhao, Ji-Rong