Skip to content

brantnej/JordanSearch

Repository files navigation

JordanSearch

JordanSearch is my final project for CS 499 that focuses on parsing and indexing audio and visual content within videos so users can locate content within a large corpus of videos.

Dependencies

JordanSearch uses ffmpeg, Vosk, ElasticSearch, and ImageAI.

Instructions

  • Download your desired Vosk Model and your ImageAI model and place it in the models directory. I used vosk-model-en-us-0.42-gigaspeech and YOLOv3 in my development. If you use different models, you will need to change vosk_model_path, imageai_model_path, and the ImageAI model type in audio_parser.py and image_parser.py respectively.

  • Place all video files in the input folder. They must be .mp4 files.

  • Run ElasticSearch with docker-compose up -d.

  • Run main.py with the -p flag to parse all video files.

  • Now, you can search for queries and if will return the top results including the file name and timestamp.

  • As long as you are using the same ElasticSearch instance, re-running main.py without the -p flag will skip parsing and begin query entry.

  • If you use the -f flag, you will search for full videos rather than finding specific timestamps within those videos.

Todo

  • Implement opening up the source files at the chosen timestamps

  • Implement a GUI

  • Use an LLM to generate keywords summarizing what the audio is about, so users don't have to search by exact match of content

  • Use an audio model that can also parse sound effects, rather than just dialogue

  • Use an image model that can identify more than just 80 items

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages