This repository is the official implementation of CARE-MI. It contains the codes for reproducing the benchmark construction procedure and experiment related information. Examples of the benchmark can be found in examples.tsv.
The paper is currently on arXiv. The benchmark will be made public at a later stage.
Authors:
- Tong Xiang (tongxiang@is.ids.osaka-u.ac.jp)
- Liangzhi Li (liliangzhi@xiaoyouzi.com)*
- Wangyue Li (liwangyue@xiaoyouzi.com)
- Mingbai Bai (baimingbai@xiaoyouzi.com)
- Lu Wei (weilu@xiaoyouzi.com)
- Bowen Wang (wang@ids.osaka-u.ac.jp)
- Noa Garcia (noagarcia@ids.osaka-u.ac.jp)
*Corresponding author.
The benchmark is and only is for evaluating the misinformation in long-form (LF) generation for Chinese Large Language Models (LLMs) in the maternity and infant care domain; it is constructed on top of existing knowledge graph (KG) datasets and multiple-choice (MC) question-answering (QA) datasets. Theoretically speaking, it is easy to transfer our benchmark construction pipeline to other knowledge-intensive domains or low-resourced languages. An illustration of our benchmark construction pipeline is shown below.
We construct two types of questions in the benchmark:
- True/False (TF) Question: Given a question, the LLM is required to judge whether the claim in the question is correct or not.
- Open-End (OE) Question: Given a question, the LLM is allowed to provide a free-form, open-ended answer in response to the question. Unlike TF questions, the answers to OE questions are not limited to True/False judgments.
We utilize word lists to filter out samples that are related to maternity and infant care topic. The word list that we use are listed below:
| Source | Language | Size |
|---|---|---|
| The Women's Health Group | EN | 87 |
| Department of Health, State Government of Victoria, Australia | EN | 99 |
| Maternal and Infant Care Clinic | EN | 57 |
| Having a Baby in China | ZH | 267 |
| Aggregated Word-list (Deduplicated & Filtered) | ZH | 238 |
To setup the environment for the codes in this repository, run:
git clone https://github.com/Meetyou-AI-Lab/CARE-MI
conda create -n care-mi python=3.8
conda activate care-mi
pip install -r requirements.txt
First, enable the required environment by:
cd care-mi
conda activate care-mi
To get domain-filtered data for all datasets, run:
python preprocess.py
True statements are the fundamental part of the benchmark construction since the rest of the data generation rely on the generated statements heavily. True statements are declarative statements that are directly built upon KG triples or QA pairs without modifying their factualness.
To generate true statements for KG samples such as BIOS and CPUBMED, run (here we use BIOS as an example):
python triple2d.py --dataset BIOS
To generate true statements for MC samples such as MEDQA and MLECQA, run (here we use MEDQA as an example):
python qa2d.py --dataset MEDQA
We offer two methods to generate factually incorrect statements on top of the true statements: negation and replacement.
Negation
Negated statements are constructed by directly negating the true statements. These negated statements are naturally false answers to the TF questions.
To generate negated statements, run (here we use MEDQA as an example):
python negation.py --dataset MEDQA
Replacement
Replacement generates false statements by replacing the originally correct answer, usually an entity, in the statement with a randomly selected wrong one. These statements are naturally false answers to the OE questions.
To generate fakse statements using replacement, run (here we use MEDQA as an example):
python replacement.py --dataset MEDQA
We generate questions that are utilized to test the misinformation of LLMs. To enable the usage of ChatYuan, beyond the required settings mentioned here, we refer the readers to its github page.
To generate questions, run (here we use MEDQA as an example):
python qg.py --dataset MEDQA
Before knowledge retrieval, we first aggregate all data we have for now. Assume we have already generate everything for all datasets, including BIOS, CPUBMED, MEDQA and MLECQA, we can run the following command:
python benchmark.py --datasets BIOS CPUBMED MEDQA MLECQA
Do the knowledge retrieval by running:
python retrieval.py --corpus textbook --retriever BM25Okapi --n 3
--corpus: Corpus that is utilized for retrieval. Options includewikipediaandtextbook.--retriever: Retrieval algorithms. Options includeBM25Okapi,BM25L,BM25Plus. Default toBM25Okapi.--n: Number of the most relevant documents selected. Default to 3.
We do the retrieval for both wikipedia and textbook in our benchmark construction. We select the top 3 documents from both sources for each question. Resources regarding the wikipedia and textbook can be found in the corpus folder. Note that the wikipedia we utilize here is the domain-filtered and preprocessed version of the Chinese Wikipedia Dump.
We refer the user to the appendix of our paper about the details of expert annotation.
We evaluate the following models (:heavy_check_mark: means available, :x: indicates not available, :o: indicates that the weights can be obtained only with permission):
| Model | Huggingface | Github |
|---|---|---|
| MOSS-16B-SFT | ✔️ [link] | ✔️ [link] |
| ChatGLM-6B | ✔️ [link] | ✔️ [link] |
| BELLE-7B-2M | ✔️ [link] | ✔️ [link] |
| BELLE-7B-0.2M | ✔️ [link] | ✔️ [link] |
| GPT-4 | ❌ | ❌ |
| GPT-3.5-turbo | ❌ | ❌ |
| LLaMA-13B-T | ⭕ [link] | ✔️ [link] |
Note that we further pretrain and fine-tune the original LLaMA-13B with Chinese corpus and selected instruction following tasks to obtain LLaMA-13B-T. We refer the readers to its original github page and fastchat for further details about pretraining and fine-tuning a LLaMA model.
The following tables present the human evaluation results of LLMs tested on our proposed benchmark, on two different metrics, e.g., correctness and interpretability. For the evaluation, each annotator is required to assign a scalar between 0 and 1 for each metric on each sample. Best performanced models are bolded and the second best ones are underlined. More details can be found in the paper.
Correctness
| Model | All | BIOS | CPubMed | MLEC-QA | MEDQA |
|---|---|---|---|---|---|
| MOSS-16B-SFT | 0.671 |
0.930 |
0.925 |
0.644 |
0.639 |
| ChatGLM-6B | 0.610 |
0.928 |
0.748 |
0.579 |
0.599 |
| BELLE-7B-2M | 0.647 |
0.843 |
0.928 |
0.631 |
0.605 |
| BELLE-7B-0.2M | 0.670 |
0.947 |
0.942 |
0.624 |
0.646 |
| GPT-4 | 0.867 |
0.958 |
0.967 |
0.851 |
0.858 |
| GPT-3.5-turbo | 0.824 |
0.973 |
0.948 |
0.799 |
0.815 |
| LLaMA-13B-T | 0.709 |
0.871 |
0.922 |
0.678 |
0.689 |
| Human Baseline* | 0.938 |
1.000 |
1.000 |
0.945 |
0.908 |
Note: We only randomly select 200 questions from the benchmark for human baseline evaluation.
Interpretability
| Model | All | BIOS | CPubMed | MLEC-QA | MEDQA |
|---|---|---|---|---|---|
| MOSS-16B-SFT | 0.746 |
0.920 |
0.883 |
0.726 |
0.731 |
| ChatGLM-6B | 0.730 |
0.929 |
0.779 |
0.705 |
0.734 |
| BELLE-7B-2M | 0.728 |
0.839 |
0.930 |
0.723 |
0.694 |
| BELLE-7B-0.2M | 0.645 |
0.716 |
0.746 |
0.609 |
0.650 |
| GPT-4 | 0.928 |
0.973 |
0.981 |
0.921 |
0.922 |
| GPT-3.5-turbo | 0.883 |
0.977 |
0.960 |
0.864 |
0.880 |
| LLaMA-13B-T | 0.816 |
0.836 |
0.935 |
0.797 |
0.808 |
Since the human evaluation can be time-consuming and expensive, we explore the usage of judgment models as a proxy for human supervision. To this end, we fine-tune judgment models using the data labeled by human annotators as well as generated synthetic data. We try four models:
- BERT-Large
- GPT-3-350M
- GPT-3-6.7B
- LLaMA-13B-T
For the fine-tuning, for each sample, we concatenate the question, corresponding knowledge, and the model output together and feed them to the model. Each sample has a label given by the annotators (for LLM output) or generated (for synthetic data). For fine-tuning GPT-3-350M and GPT-3-6.7B, please check the instruction given by OpenAI. For fine-tuning LLaMA-13-T, please check fastchat.
The performance of each judgment model is shown below; to calculate the accuracy,we cast the scalar scores into binary labels by setting the threshold to be 0.5:
Pearson correlation score
| BERT-Large | GPT-3-350M | GPT-3-6.7B | LLaMA-13B-T | |
|---|---|---|---|---|
| Correctness | 0.020 | 0.783 | 0.803 | 0.868 |
| Interpretability | 0.013 | 0.565 | 0.634 | 0.683 |
Accuracy
| BERT-Large | GPT-3-350M | GPT-3-6.7B | LLaMA-13B-T | |
|---|---|---|---|---|
| Correctness | 0.560 | 0.835 | 0.858 | 0.898 |
| Interpretability | 0.794 | 0.822 | 0.828 | 0.835 |
We select the LLaMA-13B-T as our judgment models for both aspects. The model checkpoint for correctness and interpretability are publicly available. To apply the models, we suggest using the following prompt with questions from the benchmark (the models should and only should be used to evaluate samples from the proposed benchmark):
听取某AI助手对一个医学问题的回答，并在
$\text{[aspect]}$ 方面对其进行打分，不用解释原因。
问题:$\text{[question]}$
参考资料:$\text{[knowledge]}_1$ $\text{[knowledge]}_2$ ...$\text{[knowledge]}_n$
某AI助手:$\text{[answer]}$
你的评分:
If the paper, codes, or the dataset inspire you, please cite us:
@article{xiang2023care,
title={CARE-MI: Chinese Benchmark for Misinformation Evaluation in Maternity and Infant Care},
author={Xiang, Tong and Li, Liangzhi and Li, Wangyue and Bai, Mingbai and Wei, Lu and Wang, Bowen and Garcia, Noa},
journal={arXiv preprint arXiv:2307.01458},
year={2023}
}