Develop and test changes to the pgai extension

Want to contribute to the pgai project? Start here.

This page shows you how to:

Set up a pgai development environment in Docker: all necessary software and extensions to develop pgai in a container.
Make changes to pgai: edit the pgai source and reflect the changes in the database
Test your pgai changes: use the tests.sql script to test your pgai changes.

pgai Prerequisites

Before you start working with pgai, you need:

The pgai source on your local machine:

git clone git@github.com:timescale/pgai.git

Docker
If you will be working with OpenAI, you will need an OpenAI API Key.
If you will be working with Ollama, you will need it running somewhere accessible.
- pull the llama3 model
- pull the llava:7b model
You may want a Postgres client on your host machine like Psql or PopSQL

The pgai Docker container has all the software you need preinstalled. To build and run the pgai Docker container, then connect to it:

Navigate to the folder you cloned this pgai repository to.
Build the Docker image:
```
make docker_build
```
Run the container:
```
make docker_run
```
The repo directory is mounted to /pgai in the running container.
To get a shell inside the container, run
```
make docker_shell
```
You are logged in as root.
Connect to the database:

To connect from outside the container:
```
psql -d "postgres://postgres@localhost:9876/postgres"
```
To connect from a shell within the container:
```
su postgres -
psql
```
Later, to stop and delete the container, run:
```
make docker_stop
```
```
make docker_rm
```

The repo is mounted to /pgai in the docker container. You may edit the source from either inside the docker container or from the host machine.

If you have updated the unit tests accordingly, you may simply test your changes, or you can manually update the database with your changes.

To reflect your changes in the database manually, do the following from within the docker container/virtual machine.

Copy the edited sources to the appropriate postgres directory:
```
make install_extension
```

From a psql session, run:

DROP EXTENSION IF EXISTS ai CASCADE;
CREATE EXTENSION ai CASCADE;

The tests directory contains unit tests in psql scripts. The test.sql script drives test runs.

Create a .env file to store environment variables needed for testing:
- ENABLE_OPENAI_TESTS - a boolean flag to enable/disable OpenAI unit tests
- ENABLE_OLLAMA_TESTS - a boolean flag to enable/disable Ollama unit tests
- OPENAI_API_KEY - an OpenAI API Key to use for OpenAI unit testing
- OLLAMA_HOST - the URL to the Ollama instance to use for testing (e.g. http://host.docker.internal:11434)
Run the tests
```
make test
```
This will:
1. run make install_extension to copy the sources from the /pgai directory to the correct postgres directory
2. drop the "test" database if it exists
3. create a "test" database
4. create a database user named "tester" if it doesn't exist
5. run test.sh to execute the unit tests against the "test" database

Best practice is to add new tests when you commit new functionality.