OpenAI + Redis Vector Search Demo Application

This demo showcases the vector search similarity (VSS) with huggenface and OpenAI-GPT4 embedings.

Datasets

The dataset was taken from the the following Kaggle links.

• Large Dataset
• Smaller Dataset

Running the App

Before running the app, please install Docker first.

Local Development

1. build redis-vector-db image

   $ cd redis-vector-db
   $ docker build -t redis/redis-vector-db .

2. start the vector-db

   $ docker-compose up -d redis-vector-db

3. Create python virtual env

   $ python3 -m venv .venv
   $ source .venv/bin/activate
   $ pip install -r data/data-requirements.txt
   $ pip install -r app/requirements

4. Replace keys & path

   • Replase the text "your openai api key" in the code to your openai key
   • Config proper proxy server address or set to empty string if it's not needed

5. prepare data

   • Download https://www.kaggle.com/datasets/paramaggarwal/fashion-product-images-small to gui/public/kk
   • Update resources path and run prep_data.ipynb, this will generate data/procuct_metadata.json, and data/product_vectors.openai.json

6. start the backend service

   $ cd app
   $ PYTHONPATH='.' python vecsim_app/load_data.py
   $ PYTHONPATH='.' python vecsim_app/main.py

7. start the frontend

   $ cd gui
   $ yarn
   $ yarn start

8. Navigate to http://localhost:3000 in a browser.

Application

This app was built as a Single Page Application (SPA) with the following components:

• Redis Stack: Vector database + JSON storage
• FastAPI (Python 3.8)
• Pydantic for schema and validation
• React (with Typescript)
• Redis OM for ORM
• Docker Compose for development
• MaterialUI for some UI elements
• React-Bootstrap for some UI elements
• Pytorch/Img2Vec and Huggingface Sentence Transformers for vector embedding creation

Some inspiration was taken from this Cookiecutter project and turned into a SPA application instead of a separate front-end server approach.

Notes:

This is project is forked from https://github.com/RedisVentures/redis-product-search, the major changes are:

• Fix issues and make the code runable in local
• Removed unused codes
• Add OpenAI embeddings from comparison