This repository contains the files and code used in the analytical study of ChatGPT 4.0's ability to understand and engage with Italian dialects.
The purpose of this study is to evaluate ChatGPT-4’s language proficiency in Italian dialects. The skills tested, which served as foundation for test design, include comprehension and translation, dialect recognition, analysis of the distinctive features, error detection, text production, interaction, theoretical background, and self-assessment. Different tests were crafted to mimic situations requiring these competencies, trying to emulate authentic ChatGPT-User interactions.
data/- This directory houses the dataset used in this study, including text samples in different Italian dialects, the prompts used for each test, and ChatGPT's outputs to each prompt.scripts/- This folder contains all the scripts and code used to analyze the data and generate results.results/- This folder contains the output of the tests, and the relative statistics.
To get started with this project:
- Clone the repository to your local machine using
git clone <https://github.com/SilviaLilli/ChatGPT-and-Italian-Dialects/tree/main>. - Navigate to the
scripts/directory to view and run the analysis scripts. - Explore the
data/directory to understand the dataset structure and details.
- Python 3.11.4
- ChatGPT 4.0 [https://chat.openai.com/]
- Bird, S., Klein, E., & Loper, E. (2009). Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit. O'Reilly Media, Inc. [https://www.nltk.org/]
This work is licensed under a Creative Commons Attribution 4.0 International License
We welcome contributions to improve and build upon this study. Feel free to submit pull requests or open issues to discuss potential enhancements.
For any queries or feedback, please reach out to [silvialilli@hotmail.it].