ComplexWebQuestions[1] is a famous extension of the WebQuestions dataset, introduced to solve the second issue of few complex multi-relational questions in the WebQuestions. It is created by adding more constraints and multi-hop relations into the SPARQL queries from WebQuestionsSP, and then automatically generating questions via templates and Amazon Mechanic Turk.
This dataset can be downloaded via the link.
| Model / System | Year | F1 | Hits@1 | Accuracy | Language | Reported by |
|---|---|---|---|---|---|---|
| GMT-KBQA | 2022 | 77.0 | - | 72.2 | EN | Hu et al. |
| CBR-KBQA | 2022 | 70.0 | - | 67.1 | EN | Hu et al. |
| chatGPT | 2023 | - | - | 64.02 | EN | Tan et al. |
| GPT-3.5v3 | 2023 | - | - | 57.54 | EN | Tan et al. |
| GPT-3.5v2 | 2023 | - | - | 53.96 | EN | Tan et al. |
| GPT-3 | 2023 | - | - | 51.77 | EN | Tan et al. |
| FLAN-T5 | 2023 | - | - | 46.69 | EN | Tan et al. |
| BART-large | 2022 | 68.2 | - | - | EN | Hu et al. |
| DECAF (BM25 + FiD-3B) | 2022 | - | 70.4 | - | EN | Yu et al. |
| CBR-KBQA | 2022 | - | 70.4 | - | EN | Yu et al. |
| DECAF (BM25 + FiD-large) | 2022 | - | 68.1 ± 0.5 | - | EN | Yu et al. |
| DECAF (DPR + FiD-large) | 2022 | - | 67.0 ± 0.4 | - | EN | Yu et al. |
| HGNet | 2022 | - | 58.1 | - | EN | Yu et al. |
| ProgramTransfer | 2022 | 58.7 | 58.1 | - | EN | Cao et al. |
| ProgramTransfer-o | 2022 | 55.8 | 54.7 | - | EN | Cao et al. |
| ProgramTransfer-pa | 2022 | 54.5 | 54.3 | - | EN | Cao et al. |
| NSM+h | 2022 | - | 53.9 | - | EN | Yu et al. |
| REAREV | 2022 | - | 52.9 | - | EN | Costas Mavromatis and George Karypis |
| QNRKGQA+h | 2022 | - | 51.5 | - | EN | Ma et al. |
| SR+NSM | 2022 | - | 50.2 | - | EN | Yu et al. |
| QNRKGQA | 2022 | - | 50.5 | - | EN | Ma et al. |
| NSM-distill | 2022 | - | 48.8 | - | EN | Costas Mavromatis and George Karypis |
| Rigel | 2022 | - | 48.7 | - | EN | Costas Mavromatis and George Karypis |
| TransferNet | 2022 | - | 48.6 | - | EN | Costas Mavromatis and George Karypis |
| PullNet | 2022 | - | 45.9 | - | EN | Costas Mavromatis and George Karypis |
| MRP-QA-marginal_prob | 2022 | 49.9 | - | - | EN | Wang et al. |
| ProgramTransfer-f | 2022 | 45.9 | 45.2 | - | EN | Cao et al. |
| KBIGER | 2022 | 45.5 | 50.2 | - | EN | Du et al. |
| TERP | 2022 | - | 49.2 | - | EN | Qiao et al. |
| TeacherNet | 2022 | 44.0 | 48.8 | - | EN | Cao et al. |
| NSM | 2022 | 44.0 | 48.8 | - | EN | Du et al. |
| NSM+h | 2022 | - | 48.8 | - | EN | Ma et al. |
| TransferNet | 2022 | - | 48.6 | - | EN | Du et al. |
| NSM+p | 2022 | - | 48.3 | - | EN | Ma et al. |
| NSM | 2022 | - | 47.6 | - | EN | Ma et al. |
| PullNet | 2019 | - | 47.2 | - | EN | Yu et al. |
| PullNet | 2022 | - | 45.9 | - | EN | Ma et al. |
| EmbedKGQA | 2022 | - | 44.7 | - | EN | Qiao et al. |
| QGG | 2020 | 40.4 | 44.1 | - | EN | Lan and Jiang et al. |
| Topic Units | 2019 | 36.5 | - | - | EN | Lan et al. |
| KBQA-GST | 2022 | 36.5 | - | - | EN | Wang et al. |
| TextRay | 2019 | 33.9 | 40.8 | - | EN | Bhutani et al. |
| HR-BiLSTM | 2022 | 31.2 | - | - | EN | Wang et al. |
| KGT5 | 2019 | - | 36.5 | - | EN | Yu et al. |
| GraphNet | 2022 | - | 32.8 | - | EN | Ma et al. |
| KV-Mem | 2022 | - | 21.1 | - | EN | Ma et al. |
| UHop | 2019 | 29.8 | - | - | EN | Chen et al. |
| GRAFT-Net | 2022 | 26.0 | - | - | EN | Wang et al. |
| ProgramTransfer-p | 2022 | 2.3 | 2.1 | - | EN | Cao et al. |
[1] Talmor, Alon, and Jonathan Berant. The web as a knowledge-base for answering complex questions. arXiv preprint arXiv:1803.06643 (2018).