- Image translation from Chinese to English.
- Mainly for aiding understanding during presentations, when slides are in Chinese.
| Solution | Benefits | Drawbacks |
|---|---|---|
| Google Lens/Translate with Camera | - Real-time translation for presentation | - Manually hold phone - Small display - Introduce distortion in images (e.g. blur) |
- Receive video feed from meeting app
- When translate button is pressed, image is sent for translation
- Translated text is overlaid onto original image for display. Will attempt to preserve word and background color.
virtualenv venv
source venv/bin/activate
# Screen translated_overlay related
pip3 install pyautogui pygetwindow
# Image related manipulation and text overlay
pip3 install opencv-python
# For OCR
pip3 install google-cloud-vision
# https://stackoverflow.com/questions/52455774/googletrans-stopped-working-with-error-nonetype-object-has-no-attribute-group
# Free google translate
pip3 install googletrans==3.1.0.a0Note: Ensure that this app has the relevant permissions (screenshot). Your PC should prompt you when you start the program.
First, add your API key and activate your virtual environment:
export GCLOUD_API_KEY=YOUR_API_KEY
source venv/bin/activate
Running screen translator:
- After you have started, select the desired window in your terminal.
- Press C to translate.
python3 main.py
For testing purposes:
python3 test.py -i assets/test_image.png
- Desired window must be in the foreground.
- Button has to be pressed [c] to translate the image.
- If window name keeps changing, we are unable to update the position of the window. Make sure that the window you wish to translate is positioned properly first.
- Chinese meetings only for now.
- Speech to text translation (e.g. subtitles)
- Extend detected text regions if there not much features detected.
- Scene change detection to automatically send for translation
- Could use underlying libraries (Quartz) and remove dependencies for pyautogui and pygetwindow