3Blue1Brown Translation Project

AI-Powered Translations for 3Blue1Brown's Youtube Channel
20+ supported languages

Explore the docs »

Table of Contents

1. Getting Started
   • Prerequisites
   • Installation
   • Example Usage
2. Roadmap

Getting Started

Prerequisites

Ensure you have the following installed before moving onto installation.

Necessities

• Python (tested with 3.9.17)

  python --version

Recommendations

• Anaconda Package Manager

  conda update conda
  conda update anaconda

• Capable NVIDIA GPU (For Faster Inference)

  nvidia-smi

Installation

1. Get a free API Key at https://https://elevenlabs.io/
2. Clone the repo

   git clone https://github.com/shihabsarar29/3b1b-translations.git

3. Install Necessary Packages

• Anaconda

  conda install --yes --file conda_requirements.txt

• Pip

  pip install -r requirements.txt

4. Enter your API key in ./.env

   ELEVEN_LABS_API_KEY = <API_KEY_HERE>

Example Usage

A common use case involves converting text to speech using the ElevenLabsWrapper, and making this speech more realistic using an RVC Model. To create these realistic audios, make a new IPYNB or PY file in the project's root directory.

from scripts.ElevenLabsAPI.elevenLabsAPI import elevenLabsAPI
from scripts.RVCAPI.RVCAPI import infer
import dotenv

# Load API_KEY from environment file
dotenv.load_dotenv()
API_KEY = os.getenv('ELEVEN_LABS_API_KEY')

# Initialize API Wrapper
elevenLabs = elevenLabsAPI(API_KEY)

# Get the voice_id for the custom model
elevenLabs.get_voice_id('3b1b', inPlace=True)

# Get the text to be converted
text = "Hello, World!"

# Convert the text to speech
elevenLabs.TTS_to_file("TTS_OUTPUT.mp3", text)

# Infer the RVC Model, save the output to TTS_OUTPUT.mp3
infer("sample_rvc.pth", "TTS_OUTPUT.mp3", "SAMPLE.mp3", "logs/sample_rvc/rvc_idx.index", 0, -2, "rmvpe", 160, 3, 0, 1, 0.95, 0.33)

For more information and code examples, please refer to the Documentation. The documentation was last updated 2/24/2024.

Roadmap (as of 2/28)

• TTS Testing
• ElevenLabs TTS
  • Preprocessing
  • ElevenLabs Model Development
  • Wrapper
• Azure TTS
  • Data Collection
  • Training
  • Code Implementation
• RVC
  • RVC Wrapper
  • Data Preprocessing
  • Training
  • Individual Testing
  • Integration + Testing over ElevenLabs TTS
• Translated Speech Time Estimations
  • Collecting Character Averages
  • Code Implementation
  • Adding Constants
• GPT Adjustments
  • Prompt Engineering
  • Code Implementation
• Aligning Translated Audio w/ Original Audio
  • Pause Audios
  • Speedups
  • GPT Adjustments
• ElevenLabs Fully Automated Pipeline
  • Validating Translations
    • GPT Adjustments Integration
  • Code Implementation
    • RVC Integration
    • Audio Synchronization Integration