ChatGPT-VR Documentation

ChatGPT-VR is a Unity VR project that allows users to interact with a virtual assistant using voice commands. The virtual assistant is powered by OpenAI's GPT-3 model (can be changed to any other model) and features STT (speech-to-text) and TTS (text-to-speech) services from Azure Cognitive Services, a lipsync engine (modified and stripped version of Oculus lipsync), an eye movement script and a simple head movement script. The connection to ChatGPT is handled by OpenAI package via RESTful API. The 3D avatar model is taken from ReadyPlayerMe and can be easily replaced with your own.

Installation and Setup

• To run the ChatGPT-VR project, you will need to have Unity version 2021.3.15f1 or later installed.
• Additionally, you will need to have an OpenAI API key and an Azure Cognitive Services subscription key to use the STT and TTS services. To provide your API keys to Azure and OpenAI, navigate to \Assets\Resources and paste them directly into the respective scriptable objects.

• The project is using Git LFS.
• Oculus (Meta) Quest headset is required (if you want to run it on a VR device). Make sure to install any other XR plugins if you're going to use other HMDs rather than Oculus.

Usage and Functionality

Build the project for Meta (Oculus) Quest or run it directly in the editor. Once you run the project, you can interact with the virtual assistant using voice commands. The virtual assistant will respond to your commands using its voice. The avatar will move its lips and eyes to simulate natural conversation.

Contributing

Contributions to the ChatGPT-VR project are welcome. To contribute, please fork the repository, make your changes, and submit a pull request.

License

ChatGPT-VR is released under the MIT License. See the LICENSE file for more information.