ControllerGPT

An AI controller that uses text prompts to control your robot.

ROS2 is interfaced via WebSockets through rosbridge_suite.
LangChain is used to create an interface with ChatGPT.

Prerequisites

→ You should have your own working OpenAI API key.

How to use

Cloning the package

git clone https://github.com/kalashjain23/ControllerGPT.git
cd ~/ControllerGPT

Install the dependencies in your virtual environment

python3 -m venv env
source env/bin/activate

pip install -r requirements.txt

Start your robot (for the showcase, I'll be using Turtlesim)

ros2 run turtlesim turtlesim_node

Run rosbridge_server to establish a connection with ROS

source /opt/ros/humble/setup.bash # source your ROS distribution
ros2 launch rosbridge_server rosbridge_websocket_launch.xml

Running the main script along with the key

python3 main.py --key (OpenAI API Key)    # Run "python3 main.py -h" for help

Now you'll be asked for the topic that you want ControllerGPT to control along with your goals

Topic for publishing messages (leave blank if not any) → {/publisher_topic}
Topic for using services (leave blank if not any) → {/service_topic}

What do you want your robot to do? --> {your_prompt}

Sit back, relax, and watch ControllerGPT complete the tasks for you!! :D

Some more example prompts for you to try:

→ Move forwards for 2 seconds and then stop the robot. (on /cmd_vel)
→ Move forwards for 2 seconds and then stop the robot. Also, spawn another robot at (1,2). (on /turtle1/cmd_vel and /spawn)

Adding custom interfaces

You can add your own custom interfaces in the respective messages and services directory following a certain format.

The required format:
Messages (.msg)

{"message_type": message_type, "format": {format_of_your_message}}

Example, {"message_type": "std_msgs/String", "format": {"data": _}}

Services (.srv)

{"service_type": service_type, "format": {format_of_your_service}}

Example, {"service_type": "turtlesim/Spawn", "format": {"x": _, "y": _, "theta": _}}

Note: The values of the interfaces are to be replaced with '_', which will be filled by ChatGPT.

Visuals

ControllerGPT.Demo.mp4

This project is inspired by ChatGPT_TurtleSim by Mhubii.