❗❗ This repo will no longer be maintained, please visit https://github.com/milvus-io/bootcamp ❗ ❗

Building a video search system based on Milvus

This article shows how to use the image feature extraction model VGG and the vector search engine Milvus to build an video search system.

1 System Introduction

The entire workflow of the video search system can be represented by the following picture:

When importing video, first use the OpenCV algorithm library to cut a frame of a video in the incoming system, then use the feature extraction model VGG to extract the vectors of these key frame pictures, and then import the extracted vectors into Milvus. For the original video, Minio is used for storage, and then Redis is used to store the correspondence between video and vector.

When searching for video, first use the same VGG model to convert the uploaded image into a feature vector, then take this vector to Milvus to perform similar vector search, find the most similar vectors, and then use the vectors stored in Redis. The corresponding relationship with the video is to take the video from Minio and return it to the front-end interface.

2 Data preparation

This article uses Tumblr's approximately 100,000 animated gifs as an example to build an end-to-end solution that uses video search video. Readers can use their own video files to build the system.

3 System Deployment

Step1 image build

The entire video search system needs to use Milvus0.10.0 docker, Redis docker, Minio docker, front-end interface docker and background API docker. The front-end interface docker and background API docker need to be built by the reader, and the remaining three dockers can be directly pulled from the docker hub.

# Build front-end interface docker and api docker images
$ cd search-video-demo & make all

Step2 environment configuration

This article uses docker-compose to manage the five containers mentioned earlier. The configuration of the docker-compose.yml file can refer to the following table:

name	default	detail
MINIO_ADDR	192.168.1.38:9000	Can't use 127.0.0.1 or localhost
UPLOAD_FOLDER	/ tmp	Upload tmp file
MILVUS_ADDR	192.168.1.38	Can't use 127.0.0.1 or localhost
VIDEO_REDIS_ADDR	192.168.1.38	Can't use 127.0.0.1 or localhost
MILVUS_PORT	19530
MINIO_BUCKET_NUM	20	The larger the amount of data, the more buckets are needed

The ip address 192.168.1.38 in the above table is the server address used in this article to build the video search system of the map. The user needs to modify it according to his actual situation.

Milvus, Redis and Minio require users to manually create storage directories and then perform the corresponding path mapping in docker-compose.yml. For example, the storage directories created in this article are:

/mnt/redis/data /mnt/minio/data /mnt/milvus/db

So the configuration part of Milvus, Redis and Minio in docker-compose.yml can be configured according to the following figure:

Step3 system startup

Use the docker-compose.yml modified in Step 2 to start the five docker containers needed to search the video system:

$ docker-compose up -d

After the startup is complete, you can use the docker-compose ps command to check whether the five docker containers have started successfully. The result interface after normal startup is as shown below:

Up to now, the entire Yisou video system has been built, but there is no video in the system's base library.

Step4 video import

Under the deploy directory of the system code repository, there is a video import script named import_data.py. Readers only need to modify the path of the video file in the script and the video import time interval to run the script for video import.

data_path: The path of the video to be imported.

time.sleep (0.5): indicates the time interval for importing video. In this paper, the server built by the video search system has 96 CPU cores. The time interval for importing video is set to 0.5 seconds. If there are fewer cpu cores, the time interval for importing video should be properly extended, otherwise it will cause the cpu to take up too much and generate a zombie process.

The startup command is as follows:

$ cd deploy
$ python3 import_data.py

The import process is shown below:

After waiting for the video to be imported, the entire Yisou video system is all set up!

4 Interface display

Open the browser and enter 192.168.1.38:8001 to see the interface of searching video with pictures, as shown below:

Click the settings icon in the upper right corner, you can see the video in the bottom library:

Click the upload box on the left, you can upload a picture you want to search, and then search for videos containing similar shots on the right interface:

Next, let's enjoy the fun of searching video with pictures!